selectparks - Camspace: Markerless Tracking that's Free (as-in-beer)

A small team in Tel Aviv, Israel has been working on what they describe as a paradigm shift in gaming: rather than using an input device like a WiiMote or mouse+keyboard (almost) any object in the room can be trained to be tracked by a standard webcam and their software.

Check out their impressive demo here. They just pick up a brightly coloured object, train and then track.

To be fair, this technology has been around for quite a while in computer vision, popularised most of all by Intel's OpenCV library which allows for defining and tracking many different Regions Of Interest (to unpack a geeky acronym).

Read on for more about this great technology (and why it won't displace the WiiMote any time soon)..

This kind of tracking is given the name markerless tracking, the idea that instead of using specially designed images (markers), you use objects already in the scene as your trackable form. In the case of Camspace the R.O.I they're tracking is that of closed, clearly defined forms that stand-out in relation to the background (from the software's perspective).

While this is a very exciting area of research - something I've been sinking my teeth into lately - it certainly doesn't signal the end for standard marker tracking (like that seen in Reactivision (and therefore Reactable) and ARToolkit. Fiducial markers (black and white, bordered asymmetrical images AKA 'tags') still have several advantages over markerless tracking, at the expense of needing a very specific image and looking far less cool: they provide the ability to easily determine angular tilt on 3 axes (a square or round shape is no good for determining angular tilt when markerless tracking). Moreso it isn't possible with markerless tracking to take a rectangular shape (like a book) and reliably track 360 degrees of rotation without getting into using asymmetrical image content on the cover of that book as a surrogate marker. Training a computer to track a complex image rather than a closed form or simple black-and-white marker is really CPU expensive stuff and takes time to train..

Finally, it's also worth mentioning that the great many blogs claiming this as "Wii Killer" need to be kept in perspective. The Wii offers a very different kind of input, largely in that the WiiMote passes values relating to 3D acceleration rather than a form's position or orientation in space as such (full arm-swing tennis with computer-vision based input would be pretty rough to say the least). Moreso the WiiMote is not bound to what a camera can see and so doesn't suffer from the out-of-frame or low lighting problems that computer-vision based input typically struggles with.

All said, it's great to see someone finally getting this technology out of the lab and into the hands of actual gamers. Camera based input of this kind will suit a heap of games. Fine work, I look fwd to trying it!