Getting started with the Kinect for Windows SDK

The Microsoft Research Kinect for Windows SDK beta provides academic and enthusiast developers with a simple toolkit to access the raw sensor streams of the Kinect and skeletal tracking functionality. The SDK is offered under a non-commercial license currently, with an expectation of greater functionality and a commercial license later this year. If you’ve used other SDKs already with Kinect you’ll notice the huge speed and performance improvements the Microsoft SDK delivers.

clip_image002

With the SDK installed it’s extremely easy to access the Kinect’s video, depth and audio streams for your own analysis, but it’s even easier to use these streams with the provided skeletal tracking and advanced audio capabilities. The four-element microphone array of the Kinect enables sophisticated acoustic noise suppression, while using the beam formation capabilities enables the programmer to identify the direction of the current sound source. It’s these capabilities of the Kinect; enabling computers to ‘see’ and ‘hear’ better than anything else to date, that are exciting developers all over the world. For me, the prospect of using the Kinect to deliver a ‘single camera navigation and recognition system’ to my Whitebox Robotics PC-Bot 914, is as exciting as it is now real. By combining image recognition technology with the Kinect’s depth stream, ‘A1-DW’ (my PC-bot 914), can not only recognise a location in my home but also easily determine how far from the objects in the view, it is. Combining several ‘compass point’ views and ‘A1’ can start to perceive its location. ‘A1’ can also use face recognition technology to identify people it sees. Coupling this with powerful speech recognition capabilities of the Windows platform and we have the start of some interesting computer/human interactions.

clip_image004

A1-DW plus Kinect: ’What can I get you sir?’

To take advantage of the audio capabilities for speech recognition you’ll need to install the Microsoft Speech Platform Runtime, a suitable Language Pack and the Microsoft Speech Platform SDK. Then it’s just a case of defining the grammars your application or game will listen for. This is very easily done as demonstrated by the ShapeGame sample included in the SDK.

One area lacking direct support in the current release of the SDK is gesture recognition. We hope to see this supported in future SDK releases. For now, there are a number of ways in which gesture support can be provided.

First up, there are already some simple gesture recognition examples published on Codeplex. Many of these use alternative Kinect SDKs to the Microsoft one, but converting them to utilise the Microsoft SDK isn’t very challenging. If you are looking for control interaction rather than broad gesture support, the guys at IdentityMine have published KinectPaint on Codeplex. This not only demonstrates skeletal tracking but also incorporates some simple WPF control modifications that work well with the Kinect – button, listbox, checkbox etc.

clip_image007

David Catuhe’s Gesture Viewer – KinectToolkit on Codeplex.

The most successful gesture recognition systems use a learning machine approach. There are several such implementations in development at the moment and we hope to see them on Codeplex very soon. These enable a developer to record several attempts at a gesture to define the recognition parameters of that gesture type. These are then used with a generic gesture recognition engine within the application to recognise the defined range of gestures. This approach greatly reduces the Kinect developers work load – so I expect this area of functionality will get a lot of effort early on by the NUI guru community, so that the rest of us more normal developers can get on applying this technology in our everyday world.

Download the Microsoft Kinect SDK

Kinect for Windows SDK Quickstart Video Tutorials

Get David Catuhe’s Kinect Toolkit on Codeplex

Read David Catuhe’s post about his gestures and toolkit library for Kinect

Paul Foster

Paul has been programming computers and building bits of kit for over 27 years, of which 15 have been with Microsoft in a variety of technology evangelist roles. Always looking to build with the latest and greatest technology available, Paul spends his time sharing his activities and creations online via his blog or in person. Paul is a keen dinghy sailor, and for a short time was a member of a circus flying trapeze troupe.

Published by Ubelly

Ubelly is an author for ubelly. We don't have a bio filled out for Ubelly yet, but we'll have that up for you soon - don't worry!

2 Comments So Far, what do you think?

  1. thunderboltcutter

    Have you checked out the ASUS Xtion?

  2. Code4food

    Great article. I just purchased my first 914 PC-Bot, and I am waiting for it to arrive. In the meantime, I have been scouring the web for interesting things to do with it. Purchasing the platform sure beats building it all from scratch, although this approach was much more expensive than my other robotic endeavors. One of the attractions of using the 914 is that now I can use Kinect with it! Coupled with ANN coding in the .Net programming space, this will make the reality of true, domestic service bots a reality. Gotta get the price down after that!

Leave a Reply

Your email address will not be published. Required fields are marked *

You may use these HTML tags and attributes: <a href="" title=""> <abbr title=""> <acronym title=""> <b> <blockquote cite=""> <cite> <code> <del datetime=""> <em> <i> <q cite=""> <strike> <strong>