I recently completed a week-long project to analyze the performance of today's dominant search engines when presented with questions from the famous Jeopardy! game show. The originator of the idea, Stephen Wolfram, used the results in a blog post about the similarities and differences between (where I work), and IBM's intriguing new question-answering system .

If you have already seen one of IBM's television spots, you'll know that in mid-February the system (dubbed "Watson") will compete on a special episode of Jeopardy against the two top Jeopardy champions Ken Jennings and Brad Rutter.

This event is likely to go down as an iconic example of the advance of AI technology into a realm previously reserved for human judgement, a touchstone that is similar in many ways to IBM's successful challenge to the reigning chess world champion with its computer in the late 90s.

For my part, I looked at how well traditional search engines allow one to narrow down the vast corpus of online information to just a page of potential answers to a Jeopardy clue. Not badly, it turns out, although Watson will surely advance the state of the art in text-corpus question answering.

You can find more information on , but I'll reproduce the main bar chart here, along with a fun little word-cloud I made (with the help of ) of the types of entities that occur as the answers to roughly 200k Jeopardy clues. For one thing, it's interesting how close all the major engines are now becoming, as powerful web search increasingly becomes a commodity we take for granted.

( download )

The inventor of , probably also one of the greatest mathematicians of the 20th century, was a rather astounding character named . When he wasn't throwing fabulous parties in Princeton or working on the atomic bomb, Von Neumann was laying some of the groundwork for the science of the late 20th century. Among his many creations was one of the first electronic digital computers, and associated with it the stored-program architecture that still underlies computers today.


A slightly less well known creation of his was the so-called . Something of an alter ego to the digital computer, the cellular automaton is a distributed computational device composed of an array of many simple processors instead of , as in the computer. Unlike Neumann's other ideas, which continued to flourish, cellular automata fell into obscurity until the late 70s, when they experienced something of a renaissance after the of (who happens to be the CEO of the I work for).
More recently, a strange blend of these two ideas has arisen in the guise of rather long-winded . Various academics have what happens when, instead of the disembodied entities of traditional game theory (who one imagines lurking in imaginary war rooms plotting ), the game players are embodied agents situated in physical space on a grid or lattice, interacting with their neighbors.

It turns out that this can make a huge difference to the dynamics of such games. For example, in the traditional game of Prisoner's Dilemma , the only evolutionarily stable strategy (a strategy that a group of agents can play without being undermined by the appearance of mutants in their midst) is for every player to defect -- in other words, a lose-lose situation where everyone mistrusts everyone and we are all unhappy. This is nature red in tooth and claw, pre- Leviathan .

However, when we consider that these agents can be on a grid, and can, so to speak, huddle together for comfort, we notice that "tribes" of co-operators can form that offset the attacks from defectors on all sides by forming many positive relationships amongst themselves (see pictures below, the top panel is where the co-operators (blue) have successfully fought off invaders, in the bottom not so much). In fact, this point gets to the heart of one of the puzzles of evolutionary biology: why altruism exists and how it evolved.

This last result was discovered only recently with the help of computer experiments, and there are sure to be many such discoveries waiting to be made in the field of evolutionary spatial game theory. To this end (yup, all of this post so far has been background -- it links up too beautifully to not mention the history), I've been working on a software library to allow experiments in ESGT to be conducted elegantly and efficiently. To leave you with a taste of what such experiments to look like, here are some videos of the game of Rock Paper Scissors being played on grid (at various 'temperatures', or degrees of randomness):

I'll post more such videos in the future as I do further experiments, and hopefully delve a little more into the details of the science that is going on. If you're interested in the code and seeing some more examples, or even playing around with it yourself if you have a copy of Mathematica , you can visit the project page on GitHub:

Long live spatial game theory!

Warning ! This post is super-long kind of "Adventures of a Mad Scientist" thing. Probably TL;DR.

I’ve been quite busy these past few weeks since the exams ended. I realised long ago that a hobby a day keeps the blues away - but its taken me quite a while to doing anything about it! Like I said, tho, these holidays have been crammed with project-related excitement, and writing in layman’s language so as to keep you in the know, I will now recount my latest technological escapades conducted from the comfort and safety of my bedroom flat.

On the dawn of my first exam-free day, when the sun coaxed me through the window into bounding out of bed, I decided to make a list of Things I Wanted to Actually Do. Such lists had been worked out before during the euphoric optimism that normally follows the start of a term break. Instead of trumpeting the start of the holiday with a call to arms, most To Do lists ended up mocking me with all the great ideas I knew I wouldn't have the energy to pursue. For the first two years of varsity, most of my winter breaks have dripped past from behind my bedroom window. I’ll be lucky if I can make my breakfast before 12pm, and usually things grow more and more boring until my only reprieve comes in a plastic box from Rent-A-DVD.

This time, however, I had the greatest hopes that things would be different. As a positive start, I went back through Christmas last’s list, cutting out unfeasible or redundant things and adding a few new ideas. The items were divided into sections: Electronics , Programming , Exercise , Identity Stuff (clothes, furniture and decorations for my room), Reading , and so on. The sections I endowed with multiple ideas, each worthy enough to consume the whole holiday if things turned out well.

First of the sections on my list was ‘Electronics’.

Electronics has been my neglected darling during the last three years, mainly because it is such a time consuming hobby. All the electronic parts, gadgets and gizmos that looked cool were proudly arrayed in boxes and trays on my dusty Electronics Shelf in the lounge. They ones that did not were banished to a jumbled existence in a cupboard. The parts on display had collected furry coats of dust since I had set them out so long ago, and were looking decrepit and abandoned. Even the rubber-bands I had piled in a corner of my shelf had cracked and shrivelled from months of exposure to the sunlight that slants in through the window each afternoon.

Work was to be done, and the long green trestle-table, the battleground of my competing interests and ideas, had to be made ready. With a decisive swipe I cleared my table of eraser shavings, old files and orphan paper-clips and made way for my chunky second-hand oscilloscope. In the middle of the table I plonked down my 'breadboard' - the mysteriously named device that allows prototype circuits to painlessly be assembled without the help of a soldering iron. Last of all, I hoisted a large whiteboard into place above the table - a catchment area for ideas and circuit diagrams that overspilled the limits of my mind's eye.

In my list, ‘Electronics’ was subdivided into three main projects. The first project was to get back into a Lego robotics kit I had kept from years back, which had made it all the way here from Joburg along with some other mostly useless junk. New things could be done, and old things redone for the benefit of amazed friends. However, a problem lurked, or rather didn’t lurk, from within the ‘RCX’ – the boxy yellow brain that twitched and spun the Lego motors and probed the diminutive Lego sensors.

An array of 6 AA batteries failed to stare up at me when I opened the bottom of the suspiciously light yellow brain. Hmm. Cogs spun. Batteries were expensive, and didn’t last long anyway. Another solution was needed. I hunted around in this postmaster’s shelf that is divided into dozens of little wooden sections, in which I keep on assortment of electronic beasts like transistors and LEDs. There I found, not the one component that was exactly what I was looking for, but two components that were not quite what I was looking for.

No problem, cabrone! I soldered the two doodas (voltage regulators) together and after a bit of electronic jiggery-pokkery I had a robust, healthy 9V power supply – the exact voltage I would need to pretend that 6 AA batteries were powering the yellow behemoth from beneath. Sure enough, when the yellow brain was jump-started from the circuit I had built, powered by the wall through a long wire, it beeped into life, and on the small LCD screen appeared a jerky black-and-white animation of a Lego man doing a little dance.

However, another surprise lay in wait that would doom my whole enterprise to failure. A frantic hunt through the big box full of Lego parts failed to reward me with a vital part. This part was the cable for the miniature signal-tower that allowed the computer and the brain to speak to each other, communicating such useful information as how much battery life was left, and what the darned thing was supposed to do. Without this vital connection my yellow brick was as useful as a golem with no scroll. Until I find it (or buy another), I had what amounted to a glorified digital clock. A new cable would be expensive, and it appeared I had left the old cable in Joburg.

Not to worry! Unperturbed by this sudden fatal weakness in my plan, I returned to the list to see what the second item was. “Bring Coilgun Back From The Dead”. This project, like the last, was about doing something I had already done, but better and with the hindsight of a bit of physics education. Many years ago, when I was writing A-levels, I had built a series of ‘coilguns’ – electrical contraptions that can launch a nail from a coil of wire using a simple circuit. Coilgun Mark I wasn’t built at all – it comprised a couple of totally random components rigged together according to a dodgy internet tutorial. It looked terrible, but the Mark I worked. Its performance was pretty dismal, though ­– when the coilgun was turned upright the nail could, with luck, make it all the way up and hit the ceiling of my bedroom with an amusing ‘ding’

The Mark II and III used components I had either scrounged or built specially using wire obtained from my high school equipment cupboard. The coils were wild and uneven, and one of them wasn’t totally straight. But the performance was good enough (punching through a coke can) to easily warrant their making! The Mark IV was the beaut, though. A real beaut. The coil was wound professionally in two stages. The windings were even and neat, and the centre of the coil wide enough to accommodate beefier nails than of the previous models could handle.

Coupled with some parts I scrounged from a friend’s uncle, a formidable beast was born. It could punch straight through metal cans and glass bottles, thin steel plates and an old clipboard. It was fun while it lasted, serving as a great distraction from my boring A-level lectures, but eventually its time was over. It was disassembled and boxed for my move to Capetown, and apart from one solo performance briefly in 1 st year to a couple of proto-friends, it was never to see any action again.

Rebuilding it wouldn’t be an enormous challenge, but once rebuilt the real work could begin. I would diagnose every part of the coilgun, calculating and measuring all important variables about its electronic makeup and design. These would be fed into a computer program and used to predict its performances – and better yet, I could program the computer to update the design in order to improve the performance. Virtual changes could be made without the real coilgun being tediously disassembled and rebuilt. Science and engineering to the rescue of the fiddly art of coilgun-making!

The real fun of embarking on this project lay in something other than the coilgun itself. It lay in the fact that all the extra care it was going to take required me to design and play with quite a few other circuits and solve other problems that were related only tangentially to the coilgun. This is the best part of a Real Project – there are many spin-off projects that provide their own entertainment and their own challenges. To begin with, I had to diagnose the components inside the coilgun. This was no easy task; my trusty multimeter (the 20-th century equivalent of the Star Trek tricorder) could only measure some of the important things. One particular property of the coil, called the inductance, was totally beyond its ken.

When I designed the coilgun all those years ago, I had tinkered with a home-built system to find the inductance, but in this regard I only went so far. In 2 nd year, however, I had used the same approach to do an experiment required for the Physics Lab course. I had used it to get more accurate data than what I could get by hand – and I had been able to measure the inductance of a coil I selected from my spare parts bin. I intended to refine the procedure and eventually create a very general purpose tool.

The way the thing worked was quite novel. A computer programme devised a series of sounds. The sounds started low and got higher and higher, and if you listened to it through a speaker you heard something that resembled a slow whoop. Disconnecting the speakers from the computer and connecting them across an electronic component, the ‘sound’ remained a mere electrical signal and was sent shooting through the component. Each ‘sound’ made the component ‘ring’ at a different volume, and the way the component sang when excited by the sound was carefully recorded just as if it was itself a sound, using the ‘Line In’ of the computer to listen to it.

The idea then was to program the computer to calculate properties of the component by seeing how the electrical ‘echo’ varied with pitch. Simple electrical formula provided the desired properties of the component when many of these echoes had been taken into account. It was this programming effort that I decided to embark on – the first step was to provide a tool I could use to measure all manner of properties of the coilgun. Once it was complete, I would begin work on the coilgun itself using the program as stethoscope to listen to the heartbeat of my coil!

The program I wrote from three years ago wasn’t a good place to start. It was an abomination – what programmer’s like to call ‘spaghetti code’ – a big mess of uncommented lines of code, with cryptic workarounds and hacks making the whole thing impossible to understand and re-engineer, even to the person who wrote it! I blame this not on my own sloppiness – although I could have made it easier to modify by following some simple principles – but on the god-awful but ubiquitous language it was written in, the workhorse (and a horse of a language it truly is) of modern programming, known as ‘C’.

My new program would be written in the cleanest, most elegant prose. Any programmer worth his salt knows that this can only be accomplished in the great dynamo of a modern language, known as ‘Python’ (okay, this is a silly thing to say, but got to keep the prose spicy). Python was designed in the early 90’s by a researcher working at the Stichting Mathematisch Centrum in the Netherlands. He wanted a clean, modern programming language to serve as the ‘glue’ that connected together older, messier bits of computer recipes that were used to process the vast quantities of information – in the form of the mangled wreckage of quarks and gluons – that the particle accelerators at CERN spat out ever year.

He named the language after Monty Python’s Flying Circus, and the Python tutorial that comes with every copy of Python (which is free) proclaims that “making references to Monty Python skits in documentation is not only allowed, it is encouraged!”. Open-minded programmers all over the world adopted Python fast , because it threw chunks of useless crap out of most other modern languages, retaining some familiar features, as well as introducing some beautiful new concepts.

For example, Python requires you to be neat in laying out the code on the screen. If you fuck up your indenting, your Python program will crash! This forces you to be clear in the way you arrange your programming recipes and also gets rid of these stupid ‘{‘ angle brackets that almost every other language uses to group together lines of code. I’ve been a convert to Python since about four years ago, gradually refining my ‘Pythonese’ until I can say in a stanza of Python what would require a novel in C or C++.

Things, however, were not all hunky dory. Python has an astounding collection of ‘modules’ – chunks of machinery bolted on to the basic language to enable you to do cool stuff – like build your own web server, decode zip files, or read mp3s. Playing and recording sounds, however, is not one of those modules. And so before I could build my coilgun, I had to build this tool, and before I could program the tool in Python, I had to tinker around with a new Python ‘module’ that would connect my tool, living happily inside a Python playing ground, to the very guts of the computer, where sounds can be played and received at a hardware-level.

Writing modules isn’t easy, and the description of what to do and how to do it is buried under pages and pages of documentation, which luckily does actually come with Python. My first efforts came to naught, but after wrestling with out-of-date advice and cryptic commands for a day or two, I eventually had a working Python module, that I called ‘soundio’ (for input/output), that could accomplish the very basics: playing sounds and recording them simultaneously through the soundcard of the computer.

Now the real work began, and with it came the trance-like experience I call ‘being In The Zone’. I’m sure it happens to writers and artists and mechanics, to all artisans, but with computers it is especially strange. From the outside, all it looks like is one person glued to a computer for a couple of days. From the inside, a new world has overtaken me, and I am exploring hills and mountains, striking out new routes and rehashing familiar ones. I am totally ensconced in a world of my own making. The slopes of some problems are easy enough that with persistence (and occasional breaks for me to pace up and down thinking while I smoke a cigarette), I can surmount them. Other difficulties are too steep that I must abandon my path and try another.

All in all, I spend days in a trance. I focused on almost nothing but what I was immediately doing. Everything fell by the wayside – cooking, cleaning, sleep. I didn’t eat more than one meal a day, and I didn’t ever go outside unless I needed something from the shops or a breath of fresh air and a cigarette. Each day would start with a scampering out of bed to the computer, greedy to make more progress, and after no time at all each day would end with a deepening darkness and a sudden pang of hunger, telling me it was time to emerge from my trance to see to basic bodily needs.

It was this trance that I became locked in day after day. I must have spent about a week in it, although I can’t actually remember. By the end, I had a polished, shiny new computer tool. Before it could divine any unknown parameters, the program needed to eliminate any sources of error due to the soundcard itself - a brief 'calibration' did the trick (>>> precedes commands that I typed in on the Python prompt).

>>> calibrate()
calibrating, please wait...
left channel zero = 7.646 m
right channel zero = 7.278 m
delay = 90 mS
max volume = 574.4 m

I would plug an unknown component (in this case a capacitor, indicated by ‘C’) into a simple circuit connected to the computer, provide some initial information by typing it in, and lo and behold the unknown values (the capacitance and resistance of the capacitor) would be calculated. The results of the strange sing-song between the computer and the circuit were analysed and summarised in the section titled ‘Analysis Results’.

>>> RC()
value of R? 5.256
Analysis Results:
C
= 93.11 uF
Rc
= 280.8 mOhm

More complicated circuits could be analysed too, leading to a wealth of useful information about the components in my coilgun:

>>> RLC()
Analysis Results:
L
= 63.44 uH
C
= 93.11 uF
Rl
= 5.236 Ohm
Rc
= 280.8 mOhm

So like I said, after a week I had achieved about all I was going to achieve. But go on programming I did – the work was so rewarding, and so addictive, that eventually I began imagining flaws to be ironed out where really there were none. The technique has limits, of course - it can’t measure very small capacitances or very large inductances - but I tried my best to make it accurate in these extremes. Eventually, tho, I realized that my tool was satisfyingly complete, and I moved on to the next stage of my master plan: building the Mark V coil gun.

That story, however, will have to wait until another post.