A Show Of Hands

Tesla CEO and SpaceX founder Elon Musk is using it to build space rockets. Surgeons can wave bloody gloved hands in front of it to bring up 3D images on screen of the body part they want to slice into. High school students are virtually dissecting frogs and learning notes for the piano on it. And children and gamers are playing Cut the Rope and Sugar Rush on it.

What is it? It’s the Leap Motion controller, a device the size of a cigarette lighter that, when hooked on to your computer, converts it into a motion-controlled machine. Set it about an inch in front of the keyboard, in line with the middle of your screen and it tracks your hand movements at about 200 frames per minute — with a flick of your wrist, you can move objects, with a swipe in the air, you can grab them and even move them about on the screen. The $80-device debuted in the US in July and is already available in Canada, South Africa and many European countries. It will soon be launched in India, Japan, South Korea and China, among other geographies. And the team behind it? Two childhood friends from Florida, who started Leap Motion in San Francisco in 2010. 

A leap of imagination

Mathematician David Holz was working on his PhD in applied math at the University of North Carolina-Chapel Hill, conducting research for NASA on fluid mechanics and in his spare time, trying to devise a new way, using gestures, to interact with computers. The reason: right from his school days, he had found building 3-D models on the computer particularly aggravating. What would take minutes with actual clay would take hours on the machine. His friend Michael Buckwald, who had recently sold his start-up Zazuba.com, understood that. “There is a barrier between people and computers because existing interfaces are all about pushing or not pushing a button. Interactions should be much more nuanced,” says Buckwald. The two got together — Holz ditching his PhD — to set up a company that would help remove that barrier. 

The initial prototype involved two bags of electronics and took half an hour to assemble. Still, chief executive officer Buckwald and chief technical officer Holz managed to impress enough investors with their idea, and Leap Motion (called OcuSpec in its stealth days) was in business. So far, the company has raised more than $44 million in seed, series A and B funding from Founders Fund, Highland Capital Partners, Andreessen Horowitz and SOSventures International. “Leap is founding a platform that can change the human-machine interface. Once that happens, there will be a whole new set of experiences because of gesture-based technologies,” says Manish Patel, partner, Highland Capital Partners.

The funding has helped Leap do over a dozen iterations of the controller, focusing not just on the software but also design and appearance. “We want people to feel a sense of magic, that they are in the future. The product on their desk should be consistent with that sense of magic,” says Buckwald. The final product, then, is something Steve Jobs would probably have approved of, for all that it has off-the-shelf components: rounded, brushed aluminium sides, a black body that’s just 3 inches long and weighs under 50 gm. 

And the magic? The two cameras and three infrared LEDs inside this tiny controller can detect movements of both hands and all 10 fingers individually within an eight-foot cubic cone around the sensors; the software, using some really advanced mathematics devised by Holz, converts these movements to 3-D input. 

That’s where the Leap Motion controller differs from the other big motion-control device so far — Microsoft Kinect is made for whole-body movements, where Leap Motion detects movements of up to 1/100th of a millimetre. Which lends itself to some great gaming opportunities. 

Not child’s play

When the controller was launched in July, Leap Motion also opened its captive app store, Airspace, with 75 apps. Before the launch, it had sent out over 10,000 free devices to developers to create apps using gesture-control technology. Currently, there are over 120 apps in the store, half of which are paid (ranging from 99 cents to $29.99); the company has a 70:30 revenue split with developers. 

Not surprisingly, many of the apps in the store are games and entertainment related. There are everyday games such as Boom Ball, Balloon Buzz, Fruit Ninja and Cut the Rope, as well as more sophisticated offerings such as Dropchord, which combines music and gaming, and music apps such as AirHarp and Chordion Conductor. Google Earth’s latest version is also compatible with Leap technology — the whole world at your fingertips, literally.

“Touch is incremental. This is revolutionary. And it’s not just about people playing games,” says Patel. Indeed, it is the other ways the technology can be used that has Buckwald truly excited. “Our long-term vision for Leap Motion includes embedding the technology not just in laptops [as is already being done with HP], but tablets, cellphones and even cars. There is also great scope for this in industrial applications,” says the 24 year old.

Games are just the beginning

What makes Leap Motion interesting is its near-limitless tech applications

What could some of those applications be? The most obvious seems to be education — 3-D tours of the world, human anatomy and teaching musical instruments, for instance. Already, the Airspace store has apps such as virtual dissection of a frog and a guide to the Milky Way, with details of planets, the length of their days and years etc. For special needs students, too, relying on natural hand movements rather than traditional spoken or written communication may open up new avenues of learning.

Then there’s healthcare, where worries of maintaining sterility can be done away with by opening reports and X-rays with a wave of a hand, or the technology could be used to assist in rehabilitation and physical therapy, to improve hand-eye coordination and dexterity. In retail, think of interactive kiosks and 3-D displays of products, while shopfloor applications could include manipulating industrial robots. And defence and aerospace applications would include controlling vehicles, aircraft and missiles as well as information systems.

Buckwald points out that Leap Motion has also received much attention from the financial sector for applications in biometric screening, given Leap’s ability to capture the unique structure of an individual’s hand, as well as 3-D visualisation of big data. “We are building industrial use cases for our technology and exploring the idea of embedding Leap in industrial equipment,” he adds. 

Teething troubles

Right now, though, users aren’t feeling the love. Reviews of the controller have been mixed to unfavourable and Leap Motion appears to be suffering from a bad case of first-mover disadvantage, with users still figuring out how to work the gesture control device and, more importantly, deciding whether they want to bother with it at all. 

The biggest hurdle centres on usability. Keeping your hands in the air for long periods is uncomfortable, if not downright painful. Then, there’s a lack of standard gestures — if objects move with a left to right swipe in some apps, it is upward or downward sweeps in others, making the controller difficult to use. Then, several antivirus software packages don’t recognise Leap and the device is also hypersensitive to light. 

To be fair to the company, it is working hard to overcome many of these issues. The software is being constantly updated, Leap is working on standardising several interactions across apps and is also ensuring better prediction of finger placement and improving pinch-and-grab. What will also help is greater third-party involvement in the technology, which is where continued investor support, especially when it’s targeted at expanding the Leap ecosystem, will come in handy. In June, even before the controller was up for general sale, Highland Capital announced a $25-million Leap Fund that will invest in companies using Leap Motion’s technology. It has made its first investment, in Syntellia, which has developed a virtual keyboard that will operate on Leap technology.

What will also help is greater awareness of gesture-control technology, which should come as more companies enter the fray. German company pmdtechnologies is working on commercial launch of its CamBoard Pico, while MYO has a motiondetecting armband and the Oculus Rift is an upcoming virtual reality headset. 

Most ground-breaking innovations struggle to create a market for themselves. Leap Motion doesn’t have that problem — for the most part, reviewers seem to want to love the product, having been primed by Hollywood to appreciate gesture control (note to self: avoid references to Minority Report and Iron Man). For now, Leap Motion is still clearly work in progress. If it lives up to its potential, though, it promises to be a work of art.