Jump in… to a virtual meeting?

One of the biggest threats to major airlines isn’t a low-cost competitor, but video conferencing. Why spend the time and money on business travel when the video feed coming through is as good as being there? Then imagine throwing one of the most advanced sensors on the planet into the mix. We’ve spent plenty of time pondering some use cases for Kinect that go above and beyond gaming.  With Avatar Kinect now being released to the public, is it time to start seriously thinking about building apps that make use of 3D input and the depth of ability to recognise minute movements, for communication purposes?

Avatar Kinect

Imagine having a virtual meeting space for up to 8 people where you don’t just hear audio, or have multiple video feeds going on at once that requires a massive screen to make sense of; but rather one you can set up from your lounge for free (with an Xbox Live Gold subscription), with multiple backgrounds to suit the conversation – and harness the facial recognition abilities of Kinect to understand when your friend is frowning, lifting an eyebrow, or gesturing wildly. That’s the premise behind Avatar Kinect:

There are 24 different themed 3D stages and users can record their chats to refer back to later – ideal for lounge debates about the scores over the weekend, but could you really imagine running a business meeting with your avatar in Xbox Live attire?

improvements required (and made)

The age-old mantra that human communication consists of 93% body language and paralinguistic clues means the ability to see really in-depth physical responses are key to the difference between video and audio conferencing. So the Xbox team did some work to improve both facial recognition and hand gestures.

Says Craig Mundie, Chief Research Officer at Microsoft:  “Another thing we’re working on, and we did some work in Avatar Kinect, is tracking the hands. In the major games, we stopped at the wrist, because at that distance, there’s not enough sensor resolution to do the individual digits of your hand. But, when you get a little up close, and you’re not moving so fast, we can basically even get down to the hands and finger movements. So, I think all of these things will prove, obviously, the silicon environment will improve the sensor technology, and we’ll keep moving that along, too.”

Check out this Singapore research project for an overview of the technical complexities that need to be overcome to make hand gestures really stand out.

why choose an avatar over video?

Standard protocol involves sitting in front of a PC webcam with your headset on, doing either a 1:1 video chat, or having multiple small windows open with each participant. You can only see their faces. Or a business could invest in a telepresence room that takes these feeds and assigns them to massive screens, but the cost is enormous and implementation requires a decent amount of space. By using a device that’s familiar and has an install base of over 10 million homes around the world, it makes mobile working more of a reality. Then think about meetings as they exist today – where gestures get across points, people get up and draw things on the whiteboard, and the priceless sense of actually being in the same place and it becomes a very different proposition to managing video feeds of peoples faces.

Craig Mundie outlined the broader goals for NUI technology: “There’s a number of things you can do with the Avatar Kinect. One of them is, it has a Kinect videoconference facility. And what’s interesting about it is that, while it’s just more traditional videoconferencing, the camera tracks you as you move around. So, you’re now sitting some significant distance away from the screen. So, many of the problems that you have with traditional sort of PC-based videoconferencing, where when you’re so close the angular displacement of the camera from where your gaze is gives you that very weird sensation. When you’re far back, that angle becomes just a couple of degrees, and the gaze problem is sort of automatically corrected.”

Mundie did an interview with the Wall Street Journal’s Maria Bartiromo sharing his thoughts around some non-gaming uses of Kinect and yes, some of the interview was conducted using Kinect Avatar. It’s well worth a look here.

Get the Kinect for Windows SDK Beta here