[wptabs style=”wpui-alma” mode=”vertical”] [wptabtitle] TRIPOD & MOUNT [/wptabtitle]

[wptabcontent]

1.  Setup the scanner tripod.  The tripod doesn’t have to be completely level but you want it to be stable so some degree of leveling is required.

2.   Prep the scanner mount by pulling the black lever out, pushing up the gold pin, and then releasing the lever.  This widens the slot where the scanner mount rests. 

When done correctly, the lever will continue to stick out.

[/wptabcontent]

[wptabtitle] CONNECT SCANNER [/wptabtitle] [wptabcontent]

3.  Place the scanner on the tripod until it locks into place (you will hear a ‘click’ sound when the mount has locked).  Once the scanner has been mounted, ensure that your setup is stable.

4.  Remove the power cable and gray  SCSI cable with the PCMCIA card adapter from the small black case.  Plug the scanner power cable in and connect the SCSI cable.  DO NOT TURN THE SCANNER ON.

5.  Remove the card insert from the PCMCIA slot on the laptop and plug the PCMCIA card into the laptop.

[/wptabcontent]

[wptabtitle] SETUP TURNTABLE [/wptabtitle] [wptabcontent]

6.  Remove the turntable, black power box, and all cables from the black case.  Set the two, black and white calibration charts to the side for now.  Place the turntable approximately 1 meter (or slightly less) from the scanner with the corded end facing towards the front i.e. towards the scanner. The suggested scan range for the tele and mid lenses is .6 to 1 meter.  When you begin scanning, the distance parameter will be displayed in the scan window.  We generally suggest to keep the distance in the range of 700-800 mm to avoid data loss that can occur when an object is either too close or too far from the scanner.

7.  Plug the gray cable from turntable into the black power box; also plug the power cable into the black power box. Next, take the blue serial cable and plug the green end (with 4 prongs) into the COM1 port on the black box.  NOTE:  Theprongs on the green end are directional and will face down.

[/wptabcontent]

[wptabtitle] CONNECT LAPTOP [/wptabtitle] [wptabcontent]

8.  Next, take the other end of the serial cable and plug it into the serial connection on the laptop.  To work with a laptop, you will probably have to remove the black case around the plug end using a small screwdriver.

[/wptabcontent]

[wptabtitle] POWER UP SCANNER [/wptabtitle] [wptabcontent]

9.  Once everything has been plugged in and you have the desired lens in place (see the Changing Lenses section in the final slides of this post if you need to change the scanner lens). Next power on the scanner. It will take a minute to load, when it is complete, it will display the following message on screen “Please open laser barrier and press any key.”

10.  Next, pull the large round cap off of the front of the scanner (bottom) and remove the lens cap (top), and then press any button on the back of the scanner to continue.

[/wptabcontent]

[wptabtitle] OPEN POLYWORKS [/wptabtitle] [wptabcontent]

11.  Next power on the laptop and log on.  Note:  It is important to power the scanner first and then the laptop so the connection is established.

12.  Once everything has been turned on, open Polyworks on the laptop.  Once the Polyworks Workspace Manager opens, open the IMAlign Module.  Next, go to Scanners menu, and select Minolta – VIVID 9i.  In the window that comes up,  you should see a live camera view from the scanner (connection successful – hooray.)  You are now ready to continue to the Konica-Minota Vivid 9i – Scan Settings post.

[/wptabcontent]

[wptabtitle] VIVID 9i LENSES [/wptabtitle] [wptabcontent]

Changing the Lenses on the VIVID 9i

The VIVID 9i comes with a set of three interchangeable lenses: tele, middle, and wide.  The scanner’s camera array is fixed 640X480 pixels.  Each lens offers a different field of view with the tele being the smallest.  The FOV for each lens is provided below.

The tele lens offers the best resolution and is ideal for scanning objects smaller than a baseball or larger objects with a higher resolution.  The mid lens works well for objects that are basketball size and was the lens predominantly used for scanning pottery vessels in the Virtual Hampson Museum project.  CAST researchers generally do not recommend use of the wide angle lens.

[/wptabcontent]

[wptabtitle] CHANGING LENSES I [/wptabtitle] [wptabcontent]

A.  To change the lens on the VIVID 9i, first make sure the scanner is turned off.

B.  Next remove the lens that you wish to use from the lens box and its plastic bag.

C.  Next remove the lens protector and unscrew the lens that is currently on the scanner.  IMMEDIATELY place the lens cap (located on the lens box) on the back of the lens.

NOTE: Every lens should always have 2 lens caps (one of the front and back of the lens) when the lens is not in use or is being stored.

[/wptabcontent]

[wptabtitle] CHANGING LENSES II [/wptabtitle] [wptabcontent]

D.  Remove the back lens cap from the desired lens and screw the new lens into the scanner by first lining up the red dots found on both the lens and the scanner.

E. Replace the black lens protector on the scanner.  Replace both lens caps on the newly removed lens, place in a plastic bag, and store in the lens box.

F.  You are ready to begin scanning.  Power on the scanner and continue with Step 9 on the ‘Power up Scanner’ Slide  above.

[/wptabcontent]

[wptabtitle] CONTINUE TO… [/wptabtitle] [wptabcontent]

Continue to Konica-Minota Vivid 9i – Scan Settings.

[/wptabcontent]

[/wptabs]

Mesh vertex spacing circa 1cm.

[wptabtitle] Cloud Compare[/wptabtitle] [wptabcontent]

The open source software Cloud Compare, developed by Daniel Girardeau-Montaut, can be used to perform this kind of assessment. The assessment method described here is based on the application of a series of perceptual metrics to 3D models. In this example we compare two 3D models of the same object, one derived from a C10 scanner and one from from a photogrammetric model developed using Agisoft Photoscan.[/wptabcontent]

[wptabtitle] Selecting Test Features[/wptabtitle] [wptabcontent]

Shallow but broad cuttings decorating stones are common features of interest in archaeology. The features here are on the centimetric scale across (in the xy-plane) and on the millimetric scale in depth (z-plane). In this example we assess the resolution at which a characteristic spiral and circles pattern, in this case from the ‘calendar stone’ at Knowth, Ireland is legible, as recorded by a C10 scanner at a nominal 0.5cm point spacing, and by a photogrammetric model built using Agisoft’s photoscan from 16 images.

[/wptabcontent]

[wptabtitle] Perceptual and Saliency Metrics[/wptabtitle] [wptabcontent]

Models from scanning data of photogrammetry can be both large and complex. Even as models grow in size and complexity, people studying them continue to mentally, subconsciously simplify the model by identifying and extracting the important features.

There are a number of measurements of saliency, or visual attractiveness, of a region of a mesh. These metrics generally incorporate both geometric factors and models of low-level human visual attention.

Local roughness mapped on a subsection of the calendar stone at Knowth.

Roughness is a good example of a relatively simple metric which is an important indicator for mesh saliency. Rough areas are often areas with detail, and areas of concentrated high roughness values are often important areas of the mesh in terms of the recognizability of the essential characteristic features. In the image above you can see roughness values mapped onto the decorative carving, with higher roughness values following the edges of carved areas.

[/wptabcontent]

[wptabtitle] Distribution of Roughness Values[/wptabtitle] [wptabcontent]The presence of roughness isn’t enough. The spatial distribution, or the spatial autocorrelation of the values, is also very important. Randomly distributed small areas with high roughness values usually indicate noise in the mesh. Concentrated, or spatially autocorrelated, areas of high and low roughness in a mesh can indicate a clean model with areas of greater detail.

High roughness values combined with low spatial autocorrelation of these values indicates noise in the model.

[wptabtitle] Picking Relevant Kernel Sizes[/wptabtitle] [wptabcontent]

To use the local roughness values and their distribution to understand the scale at which features are recognizable, we run the metric over our mesh at different, relevant, kernel sizes. In this example, the data in the C10 was recorded at a nominal resolution of 5mm. We run the metric with the kernel at 7mm, 5mm, and 3mm.

Local roughness value calculated at kernel size: 7mm.

Local roughness value calculated at kernel size: 5mm.

Local roughness value calculated at kernel size: 3mm.

Visually we can see that the distribution of roughness values becomes more random as we move past the effective resolution of the C10 data: 5mm. At 7mm the feature of interest -the characteristic spiral- is clearly visible. At 5mm it is still recognizable, but a little noisy. At 3mm, the picture is dominated by instrument noise.
[/wptabcontent]
[/wptabs]

[wptabs style=”wpui-alma” mode=”vertical”] [wptabtitle] THE OPTECH PARSER[/wptabtitle]

[wptabcontent]

Introducing the Optech ILRIS-3D Parser:

The Parser program processes raw scanner data in a variety of output formats recognized by PolyWorks, GIS softwares, and other 3D modeling suites.

Using ILRIS Parser

Open the Parser program by double-clicking on the icon on your desktop.  The Parser window displays as follows:

The Optech Parser window

[/wptabcontent]

[wptabtitle] ADD DATA[/wptabtitle] [wptabcontent]

Add your data

Add your .i3d file by clicking the “Add” button.

Browse to the scan file location and select the desired scan.

Adding data in Optech Parser

The .i3d file is added to the file window in the Parser.  A photograph of the scanned area acquired by the laser scanner is also viewable in the Parser window.

[/wptabcontent]

[wptabtitle] OUTPUT FILES & GENERAL SETTINGS [/wptabtitle] [wptabcontent]

Next, specify your desired output file type. For more information on output file types and their respective settings, please refer to the ILRIS-3D Operation Manual (Section 9).  The PIF file type is the default output file type and is a PolyWorks specific format for use in the Polyworks Suite.  The PIF output options are discussed in the following slides.

Click on the “Settings” button.

General settings in Optech Parser

In the Output File Name textbox, specify the name and location for the output PIF file.

The Reduction option allows you to subsample your data (recommended subsampling: 1).

Filtering – You also have the option to “filter” your data by specifying a Range Gate and/or an Intensity gate.

The Range Gate option specifies a desired data range based on distance from the scanner (in meters)

The Intensity Gate option specifies a desired data range based on the intensity readings returned from the scanner.  While it is good practice to range grade your data to remove outliers etc…, it is not required because the data can also edited in the PIFEdit software.

[/wptabcontent]

[wptabtitle] MORE GENERAL & ADVANCED SETTINGS [/wptabtitle] [wptabcontent]

General Settings:

It is recommended to leave the Generate Bitmap, Generate Log, and Output pf Intensity options checked.

Generate Bitmap creates a separate .bmp file of the scan photograph

Generate Log creates a report containing the scan statistics

Output pf Intensity adds the intensity values returned from the scanner to the PIF file.

General Details and Advanced Settings:

For more on these options and those available in the Advanced Settings tab, please refer to the ILRIS-3D Operation Manual (Section 9).

[/wptabcontent]

[wptabtitle] PARSE DATA [/wptabtitle] [wptabcontent]

When the desired options have been selected in the “Settings” menus, click “Ok,” and “Parse” the data.  A progress report is displayed to show the steps and completion of the parsing process

The progress window shows as the data is being parsed

[/wptabcontent]

[wptabtitle] CONFIRM PARSING IS SUCCESSFUL[/wptabtitle] [wptabcontent]

Note that a green check appears beside the file name in the file window indicating that that the file has been parsed.

To view the resulting PIF file, click the “View output” button.

Confirm parsing was successful and view output file

[/wptabcontent]

[wptabtitle] VIEW PARSED DATA [/wptabtitle] [wptabcontent]

Once you click ‘View Output’ in ILRIS Parser, the PIF file is opened in the PIFEdit viewing window

The parsed file can now be viewed in PIFEdit

[/wptabcontent]

[wptabtitle] FILES CREATED IN PARSING [/wptabtitle] [wptabcontent]

Note that a PIF file (.pf), a report log (.txt) and a bitmap of the scan photo (.bmp) are created in the parsing process.

The files created in the parsing process

[/wptabcontent]

[wptabtitle] CONTINUE TO…[/wptabtitle]

[wptabcontent] Continue to Polyworks PIFEdit: Cleaning Point Cloud Data[/wptabcontent]
[/wptabs]

1.  Set up the tripod as levelly as possible

2.  Attach scanner to tripod, be sure to tighten screw securely

3.  Turn on laptop and connect it to scanner using either the cable or wireless network. Depending on the desired configuration, please set up accordingly:

Ethernet/Wired Connection

– In Control Panel > Network Connections > Select Internet Protocol (TCP/IP)
– Use the following IP address: 192.9.202.1 and Subnet Mask: 255.255.255.0
– In I3dNet Software > Go to tools > Prefs > Communication > Enter 192.9.202.248
– Polyworks dongle

Wireless Connection

– In Internet Protocol (TCP/IP), enter the following IP address: 192.168.0.6 and Subnet Mask: 255.255.255.0
– In I3dNet Software > Enter IP address 192.168.0.5
– Note that to reset the IP address you can log onto the internet > In Control Panel > Network Connections > (TCP/IP) > Specify ‘Obtain IP Address Automatically’

4.  Connect batteries to I-Bar in a T-shape. This is the only way that they will provide power to scanner

5.  Connect batteries to scanner using battery cable

6.  Start the I3DNet Program

7.  Go to the Communication Menu > Click ‘Connect’

8.  Click icon with red outline to place scanning area on screen and re-size red box to define the scan area

9.  Click ‘Acquire’ button to obtain the distance to the object to be scanned

10. Set the parameters of the scan – First will return readings from the first object/surface that is encountered while Last will return readings from the last object/surface that is encountered. Use Last to scan ‘through’ objects such as trees, fences, etc.

11. Set point spacing of laser (resolution)

12. Press the apply button

13. Start scan and save the file in the desired location

14. Parse scan to check data quality

15. Continue to Optech ILRIS Parser: Pre-Processing Scan Data

• 1. Tripod – DO NOT FORGET!!!

• 2. ILRIS 3D Scanner

• 3. Scanner Batteries – fully charged (4 total)

• 4. I-Bar/Battery Holder

• 5. Gray Battery Cable

• 6. Copy of Operation Manual

• 7. Operating Options :

• • Laptop Option (Recommended) Requires:
  • – Laptop with I3DNet, PIF Edit, and Parser software installed
  • – External lithium battery – fully charged (2 options)
  • – Modified Ethernet cable to connect to scanner OR two wireless cards (PCM 352 and LCM 352)
  • – Polworks software (optional)
  • – Polyworks dongle
  • Palm Option Requires:
  • – Handheld Palm Unit – fully charged with ILRIS 3D Palm software installed (different than I3DNet)
  • – Serial data cable to connect to scanner
  • – Flash card, flash card holder (to store data)

   

• 8. Optional Equipment:

• – Extra scanner batteries
• – Extra external laptop batteries
• – Battery charger(s) for scanner batteries, laptop batteries, etc…
• – AC/DC Car Charger
• – Scan targets (optional)
• – Digital Camera
• – Metal rods (triangular)used for stabilizing tripod on hard surfaces

 

NOTE: Even though we are downloading the “Java” Eclipse IDE you can easily add plugins to use this same piece of software with Python, C/C++, and many other applications.

Additionally, we are assuming that you have already gone through the OpenNI installation located here.

You also need to have the Java JDK installed (www.oracle.com).

Finally, to gain access to one of the best open-source computer vision libraries available, you will need to download and install OpenCV(http://opencv.org/) and the JavaCV(http://code.google.com/p/javacv/). The installation instructions located on each of these sites are excellent.
[/wptabcontent]

[wptabtitle] Setting Up Eclipse with OpenNI: Before You Start[/wptabtitle] [wptabcontent]

Important Note: As you may already be aware, these tutorials are focused on the Beginner Level user, not only to using the Kinect but also to programming. Before going any further I should also remind you that if jumping “head first” into the new domain of programming isn’t something for which you have the interest or the time, there are many things you can accomplish with the “ready to use software” solutions located here.

Also, before starting, make sure that you are using the same platform (32 –bit to 32-bit/64 to 64) on the Eclipse IDE, Java JDK, and OpenNI installation.

[/wptabcontent]

[wptabtitle] Eclipse with OpenNI: Starting a New Java Project [/wptabtitle] [wptabcontent]

Starting a New Java Project …..
Once you have downloaded Eclipse, installed the java JDK and the OpenNI/Primesense, you will need to start a new Java Project. Following the wizard is the easiest way to do this.

Check the box that says “public static void main(String[] args)” so that Eclipse will add a few lines of code for us.

NOTE: For this tutorial I have kept the names fairly vague – be sure to use names that you will remember and understand. Remember that if you use a different naming convention than shown here, you will need to make corrections in the sample code to fit to your specifications.

[/wptabcontent]

[wptabtitle] Eclipse with OpenNI: Adding the OpenNI Libraries Part 1[/wptabtitle] [wptabcontent]Adding the OpenNI libraries…

Next we will need to add the OpenNI libraries to the project. This is a pretty straight forward process in Java and Eclipse, simply being a matter of adding the pre-compiled JAR file from the “bin” folder of your OpenNI installation directory.

NOTE: If you plan on using User Tracking or another Primesense middleware capability you will need to add the JAR in the Primesense directory

To do so right-click on the project we just created:

And select the “Properties” menu item.
Then we will want to select the “Java Build Path” and “Add External Jar’s” button.

Repeat the same steps as above for the JavaCV JAR’s that you previously installed somewhere on your machine.

[/wptabcontent]

[wptabtitle] Eclipse with OpenNI: Adding the OpenNI Libraries Part 2[/wptabtitle] [wptabcontent]

Navigate to the “bin” folders of the install directories for OpenNI and Primesense.
On my Windows 7 64-bit machine with the 32-bit install it is located here:

Note: There are TWO OpenNI “JAR” files – one in the bin folder of the OpenNI install directory as well as one in the Primesense directory. I haven’t noticed any difference in using one over the other; as long as your environment paths in Windows are set up to locate the needed files, they should both work.

After this, you should see these files in the “Referenced Libraries” directory on the “Package Explorer” tool bar in Eclipse.

[/wptabcontent]

[wptabtitle] Eclipse with OpenNI: Projects [/wptabtitle] [wptabcontent]

We should now be able to access the Kinect via Java and Eclipse.

In the following projects we will introduce and attempt to explain the necessary steps for initializing the Kinect via OpenNI and for getting basic access to its data feeds in Java.

Each project goes through setting up the Kinect in OpenNI and includes comments to explain line-by-line what is going on.
[/wptabcontent]

[wptabtitle] For information on Visual Studio 2010 & Microsoft C# SDK….[/wptabtitle] [wptabcontent]Using the Microsoft SDK provides a lot of advantages and ease of access, but it is also only applicable to the “Kinect for Windows” hardware and not the Xbox Kinect (as of v1.5).

There are a lot of existing tutorials on the web about setting up your development environment with plenty of sample projects. Below is a list of links to a few of them in no particular order as to avoid reinventing the wheel.

1. http://channel9.msdn.com/Series/KinectQuickstart/Setting-up-your-Development-Environment
2. http://social.msdn.microsoft.com/Forums/el/kinectsdk/thread/7011aca7-defd-445a-bd3c-66837ccc716c
3. http://msdn.microsoft.com/en-us/library/hh855356.aspx
4. Power Point from Stanford

[/wptabcontent]

[/wptabs]

Since there is a plethora of existing tutorials regarding setting up various development environments in C++ and corresponding example projects, this article will show you how to setup the 32-bit OpenNI JAR (OpenNI Java wrapper) in Eclipse IDE. We will then initialize an OpenNI production node to begin accessing Kinect data and to get the RGB stream into OpenCv, which is a popular computer vision library.

Before going on to the following project, make sure that you have all of the dependent libraries installed on your machine. For the instructions on getting the 3rd party libraries and for setting up the development environment check out this post.

Also, I want to clarify that this code is merely one solution that I managed to successfully execute. This said, it may have bugs and/or mayb be done more successfully or more easily using a different solution. If you have any suggestions or find errors, please don’t hesitate to contact us and I will change the post immediately. These posts follow and continue to follow exploration and collaboration.

[/wptabcontent]

[wptabtitle] Using the Kinect’s RGB feed[/wptabtitle] [wptabcontent]

[/wptabcontent]

In this project we will:

1. Make a simple program to capture the RGB feed from the Kinect in Java
2. Get the data into an OpenCV image data structure
3. Display the data on the screen

A high-level overview of the steps we need to take are as follows:

1. Create a new ‘context’ for the Kinect to be started
2. Create and start a ‘generator’ which acts as the mechanism for delivering both data and metadata about its corresponding feed
3. Translate the raw Kinect data into a Java data structure to use in native Java libraries
4. Capture a “frame” and display it on screen

The next tab is the commented code for you to use as you wish.

NOTE: For extremely in-depth and excellent instruction on using JavaCV, the Kinect, along with various other related projects I extremely the book(s) by Andrew Davison from the Imperial College London. A list of his works can be found here http://www.doc.ic.ac.uk/~ajd/publications.html and here http://fivedots.coe.psu.ac.th/~ad/.

[wptabtitle] Sample RGB Project – Part 1[/wptabtitle] [wptabcontent]

[/wptabcontent]

First, let’s import the required libraries:


import org.OpenNI.*;

import com.googlecode.javacv.*;
import static com.googlecode.javacv.cpp.opencv_imgproc.*;
import com.googlecode.javacv.cpp.opencv_core.IplImage;

Then eclipse nicely fills out our class information.


public class sampleRGB {

We define some global variables



static int imWidth, imHeight;

static ImageGenerator imageGen;
static Context contex

Eclipse will also fill out our “main” statement for use by checking the box on the project set up. One addition we will need to make is to surround the following code block with a statement for any exceptions that may be thrown when starting the data feed from the Kinect. Here we are starting a new “context” for the Kinect


public static void main(String[] args) throws GeneralException {
Create a “context”


context = new Context();

[wptabtitle] Sample RGB Project – Part 2[/wptabtitle] [wptabcontent]

[/wptabcontent]

We are manually adding the license information from Primesense. You can also directly reference the xml documents located in the install directory of both the OpenNI and Primesense.


License license = new License("PrimeSense", "0KOIk2JeIBYClPWVnMoRKn5cdY4=");
context.addLicense(license);


Create a “generator” which is the machine that will pump out RGB data



imageGen = ImageGenerator.create(context);


We need to define the resolution of the data coming from the image generator. OpenNI calls this mapmode (imageMaps, depthMaps, etc.). We will use the standard resolution.

First initialize it to null.



MapOutputMode mapMode = null;
mapMode = new MapOutputMode(640, 480, 30);
imageGen.setMapOutputMode(mapMode);


We also need to pick the pixel format to display from the Image Generator. We will use the Red-Green-Blue 8-bit 3 channel or “RGB24”



imageGen.setPixelFormat(PixelFormat.RGB24);

[wptabtitle] Sample RGB Project – Part 3[/wptabtitle] [wptabcontent]

[/wptabcontent]

OpenNI also allows us to easily mirror the image so movement in 3d space is reflected in the image plane



context.setGlobalMirror(true);


Create an Iplimage(opencv image) with the same size and format as the feed from the kinect.



IplImage rgbImage = IplImage.create(imWidth,imHeight, 8, 3);


Next we will use the easy route and utilize JFrame/Javacv optimized canvas to show the image


CanvasFrame canvas = new CanvasFrame("RGB Demo");


Now we will create a never-ending loop to update the data and frames being displayed on the screen. Going line by line we will, update the context every time the image generator gets new data.
Set the opencv image data to the byte buffer created from the imageGen.

NOTE: For some reason the channels coming from the Kinect to the opencv image are ordered differently so we will simply use the opencv convert color to set the “BGR” to “RGB”. We tell the canvas frame that we created to show image.

[wptabtitle] Sample RGB Project – Part 4[/wptabtitle] [wptabcontent]

[/wptabcontent]

Finally, we need to also release the Kinect context or we will get an error the next time we try to start a node because the needed files will be locked


while (true){
context.waitOneUpdateAll(imageGen);
rgbImage.imageData(imageGen.createDataByteBuffer());
cvCvtColor(rgbImage, rgbImage, CV_BGR2RGB);
canvas.showImage(rgbImage);

canvas.setDefaultCloseOperation(CanvasFrame.EXIT_ON_CLOSE);
}

**Note that you can add an argument to the canvas frame to reverse the channels of the image

[/wptabs]

[wptabtitle] RGB Image[/wptabtitle]

[wptabcontent]The color image streaming from the RGB camera is much like that from your average webcam. It has a standard resolution of Height:640 Width:480 at a frame rate of 30 frames per second. You can “force” the Kinect to output a higher resolution image (1280×960) but it will significantly reduce it’s frame rate.

Many things can be done with the RGB data alone such as:

• Image or Video Capture
• Optical Flow Tracking
• Capturing data for textures of models
• Facial Recognition
• Motion Tracking
• And many more….

While it seems silly to purchase a Kinect (about $150) just to use it as a webcam – it is possible. In fact there are ways to hook the camera up to Microsoft’s DirectShow to use it with Skype and other webcam-enabled programs. (Check out this project http://www.e2esoft.cn/kinect/)

• For an Example Project Using the Kinect RGB feed ……

[/wptabcontent]

[wptabtitle] Depth Image[/wptabtitle]

[wptabcontent] The Kinect is suited with two pieces of hardware, which through their combined efforts, give us the “Depth Image”.
It is the Infrared projector with the CMOS IR “camera” that measures the “distance” from the sensor to the corresponding object off of which the Infrared light reflects.

I say “distance” because the depth sensor of the Kinect actually measures the time that the light takes to leave the sensor and to return to the camera. The returning signal to the Kinect can be altered by other factors including:

• The physical distance – the return of this light is dependent on it reflecting off of an object within the range of the Kinect (~1.2–3.5m)
• The surface – like other similar technology (range cameras, laser scanners, etc.) the surface which the IR beam hits affects the returning signal. Most commonly glossy or highly reflective, screens (TV,computer,etc.), and windows pose issues for receiving accurate readings from the sensor.

[/wptabcontent]

[wptabtitle] The IR Projector[/wptabtitle]

[wptabcontent]

The IR projector does not emit uniform beams of light but instead relies on a “speckle pattern” according to the U.S. Patent (located here:http://www.freepatentsonline.com/7433024.pdf )

You can actually see the IRMap, as it’s called, using OpenNI. Here is a picture from Matthew Fisher’s website and another excellent resource on the Kinect (http://graphics.stanford.edu/~mdfisher/Kinect.html)

The algorithm used to compute the depth by the Kinect is derived from the difference between the speckle pattern that is observed and a reference pattern at a known depth.

“The depth computation algorithm is a region-growing stereo method that first finds anchor points using normalized correlation and then grows the solution outward from these anchor points using the assumption that the depth does not change much within a region.”

For a deeper discussion on the IR pattern from the Kinect check out this site : http://www.futurepicture.org/?p=116

[/wptabcontent]

[wptabtitle] Coordinate System[/wptabtitle]

[wptabcontent]

As one might assume the Kinect uses these “anchor points” as references in it’s own internal coordinate system. This origin coordinate system is shown here:

So the (x) is the to left of the sensor, (y) is up, and (z) is going out away from the sensor.

This is shown by the (Kx,Ky,Kz) in the above image, with the translated real world coordinates as (Sx,Sy,Sz).

The image is in meters (to millimeter precision).

[/wptabcontent]

[wptabtitle] Translating Depth & Coordinates[/wptabtitle]

[wptabcontent]

So when you work with the Depth Image and plan on using it to track, identify, or measure objects in real world coordinates you will have to translate the pixel coordinate to 3D space.OpenNI makes this easy by using their function (XnStatus xn::DepthGenerator::ConvertProjectiveToRealWorld) which converts a list of points from projective(internal Kinect) coordinates to real world coordinates.

Of course, you can go the other way too, taking real world coordinates to the projective using (XnStatus xn::DepthGenerator::ConvertRealWorldToProjective)

The depth feed from the sensor is 11-bits, therefore, it is capable of capturing a range of 2,048 values. In order to display this image in more 8-bit image structures you will have to convert the range of values into a 255 monochromatic scale. While it is possible to work with what is called the “raw” depth feed in some computer vision libraries(like OpenCV) most of the examples I’ve seen convert the raw depth feed in the same manner. That is to create a histogram from the raw data and to assign the corresponding depth value (from 0 to 2,048) to one of the 255 “bins” which will be the gray-scale value of black to white(0-255) in an 8-bit monochrome image.

You can look at the samples given by OpenNI to get the code, which can be seen in multiple programming languages by looking at their “Viewer” samples.

[/wptabcontent]

[wptabtitle] Accuracy[/wptabtitle]

[wptabcontent]

Another thing worth noting is the difference in accuracy of the depth image as distance from the Kinect increases. There seems to be a decrease in accuracy as one gets further away from the sensor, which makes sense when looking at the previous image of the pattern that the IR projector emits. The greater the physical distance between the object and the IR projector, the less coverage the speckle pattern has on that object in-between anchor points. Or in other words the dots are spaced further apart(x,y) as distance(z) is increased.

The Kinect comes factory calibrated and according to some sources, it isn’t that far off for most applications.

[/wptabcontent]

[wptabtitle] Links for Recalibrating[/wptabtitle]

[wptabcontent]
Here are some useful links to recalibrating the Kinect if you want to learn more:

• http://www.ros.org/news/2010/12/technical-information-on-kinect-calibration.html
• http://labs.manctl.com/rgbdemo/index.php/Documentation/TutorialProjectorKinectCalibration
• http://openkinect.org/wiki/Calibration
• http://nicolas.burrus.name/index.php/Research/KinectCalibration
• http://sourceforge.net/projects/kinectcalib/ (a MatLab ToolBox)

[/wptabcontent]

[wptabtitle] User Tracking[/wptabtitle]

[wptabcontent]
The Kinect comes with the capability to track users movements and to identify several joints of each user being tracked. The applications that this kind of readily accessible information can be applied to are plentiful. The basic capabilities of this feature, called “skeletal tracking”, have extended further for pose detection, movement prediction, etc.

According to information supplied to retailers, Kinect is capable of simultaneously tracking up to six people, including two active players, for motion analysis with a feature extraction of 20 joints per player. However,PrimeSense has stated that the number of people that the device can “see” (but not process as players) is only limited by how many will fit into the field-of-view of the camera.

Tracking 2 users **Microsoft

An in depth explanation can be found on the patent application for the Kinect here:http://www.engadget.com/photos/microsofts-kinect-patent-application/

[/wptabcontent]

[wptabtitle] Point Cloud Data[/wptabtitle]

[wptabcontent]

The Kinect doesn’t actually capture a ‘point cloud’. Rather you can create one by utilizing the depth image that the IR sensor creates. Using the pixel coordinates and (z) values of this image you can transform the stream of data into a 3D “point cloud”. Using an RGB image feed and a depth map combined, it is possible to project the colored pixels into three dimensions and to create a textured point cloud.

Instead of using 2D graphics to make a depth or range image, we can apply that same data to actually position the “pixels” of the image plane to 3D space. This allows one to view objects from different angles and lighting conditions. One of the advantages of transforming data into a point cloud structure is that it provides for more robust analysis and for more dynamic use than the same data in the form of a 2D graphic.

Connecting this to the geospatial world can be analogous to the practice of extruding Digital Elevation Models (DEM’s) of surface features to three dimensions in order to better understand visibility relationships, slope, environmental dynamics, and distance relationships. While it is certainly possible to determine these things without creating a point cloud, the added ease of interpreting these various relationships from data in a 3D format is self-evident and inherent. Furthermore, the creation of a point cloud allows for an easy transition to creating 3D models that can be applied to various domains from gaming to planning applications.

So with two captured images like this:

Depth Map of the imaged scene, shown left in greyscale.

We can create a 3D point cloud.

[/wptabcontent]

[wptabtitle] Setting Up Your Development Environment[/wptabtitle]

[wptabcontent] We will give you a few examples on how to set up your Development Environment using Kinect API’s.

These are all going to be demonstrated on a Windows 7 64-bit machine using only the 32-bit versions of the downloads covered here.

At the time of this post the versions we will be using are:
OpenNI: v 1.5.2.23
Microsoft SDK: v 1.5
OpenKinect(libfreenect): Not Being Done at this time… Sorry

To use the following posts you need to have installed the above using these directions.

• For Information on setting up Eclipse and the OpenNI/Primesense Java ….

[/wptabcontent]
[/wptabs]

Resources and Learning

1.   www.kinecthacks.com

2.   www.kinect.dashhacks.com

3.   www.kinecteducation.com

4.   www.developkinect.com

5.   www.scratch.saorog.com

6.   www.microsoft.com/education/ww/partners-in-learning/Pages/index.aspx

7.  blogs.msdn.com/b/uk_faculty_connection/archive/2012/04/21/kinect-for-windows-curriculum

8.   dotnet.dzone.com/articles/kinect-sdk-resources

9.   hackaday.com/2012/03/22/kinect-for-windows-resources

10. channel9.msdn.com/coding4fun/kinect

11. www.pcworld.com/article/217283/top_15_kinect_hacks_so_far.html

[/wptabcontent]

[wptabtitle] Links: OpenNI[/wptabtitle] [wptabcontent]

OpenNI

• openni.org – Open Natural Interaction, an industry-led, not-for-profit organization formed to certify and promote the compatibility and interoperability of Natural Interaction (NI) devices, applications and middleware

• github.com/openni – Open source framework for natural interaction devices

• github.com/PrimeSense/Sensor – Open source driver for the PrimeSensor Development Kit

[/wptabcontent]

[wptabtitle] Links: Tech[/wptabtitle] [wptabcontent]

Tech

1. www.ifixit.com/Teardown/Microsoft-Kinect-Teardown/4066 – Hardware teardown. Chip info is here. (via adafruit)

2. kinecthacks.net/kinect-pinout – Pinout info of the Kinect Sensor

3. www.primesense.com/?p=535 – Primesense reference implementation (via adafruit thread)

4. www.sensorland.com/HowPage090.html – How sensors work and the bayer filter

5. www.numenta.com/htm-overview/education/HTM_CorticalLearningAlgorithms.pdf – Suggestions to implement pseudocode near the end

6. http://www.dwheeler.com/essays/floss-license-slide.html – Which licenses are compatible with which

7. http://www.eetimes.com/design/signal-processing-dsp/4211071/Inside-Xbox-360-s-Kinect-controller – Another Hardware Teardown. Note this article incorrectly states that the PS1080 talks to the Marvell chip.

8. http://nvie.com/posts/a-successful-git-branching-model/ – Model for branching within Git

9. http://git.kernel.org/?p=linux/kernel/git/torvalds/linux-2.6.git;a=blob;f=Documentation/SubmittingPatches – Linux contribution procedure

10. http://git.kernel.org/?p=git/git.git;a=blob_plain;f=Documentation/SubmittingPatches;hb=HEAD – Git project contribution procedure

[/wptabcontent]

[wptabtitle] Hardware Options[/wptabtitle]

[wptabcontent]

Well the first option is to build your own, here’s a how-to:

http://www.hackengineer.com/3dcam/

But since not all of us have the time of skills to do that there are other options, like….

1- ASUS Xtion PRO:
Price: $140
Spec’s: http://www.newegg.com/Product/Product.aspx?Item=N82E16826785030

2- Leap Motion:
Price: $70
Spec’s: https://live.leapmotion.com/about.html

Of course there may be more and there is talk of Sony recently filling a patent resembling their own “Kinect-like” device.

[/wptabcontent]

[wptabtitle] Xbox Kinect vs. Kinect for Windows[/wptabtitle]

[wptabcontent]

As you may know there are actually two “Kinect” sensors out on the market today…

both under the Microsoft company but one was the original made for the Xbox 360 Game console, while the other is the recently released “Kinect for Windows”.

Overview

As far as I can tell the two hardware stacks are identical except for the name plate on the front (XBOX or KINECT FOR WINDOWS) and the Windows version has a shorter power cord with a higher price tag due to licensing issues.

There are constant changes being done to the Kinect for Windows to distance itself from it’s Xbox twin, like a firmware update to support “Near Mode” in the Windows SDK….

Microsoft even goes as far as saying the following:

‘The Kinect for Windows SDK has been designed for the Kinect for Windows hardware and application development is only licensed with use of the Kinect for Windows sensor. We do not recommend using Kinect for Xbox 360 to assist in the development of Kinect for Windows applications. Developers should plan to transition to Kinect for Windows hardware for development purposes and should expect that their users will also be using Kinect for Windows hardware as well.’

If you are currently using the Kinect for Xbox you will find that the automatic registration functions found with the Microsoft SDK will not recognize your Kinect and therefore kick out an error every time you try to run one of their samples.

As far as I know, you can however, still manually register the Kinect with the Microsoft SDK and utilize the functions already developed in the API AT THIS POINT with the XBOX version of the sensor. I wouldn’t be surprised if this changes in the near future however.

[/wptabcontent]

[wptabtitle] Published Resources[/wptabtitle] [wptabcontent]

Published Resources

1) Abramov, Alexey et al. “Depth-supported Real-time Video Segmentation with the Kinect.” Proceedings of the 2012 IEEE Workshop on the Applications of Computer Vision. Washington, DC, USA: IEEE Computer Society, 2012. 457–464. Web. 6 July 2012. WACV ’12.

2) Bleiweiss, Amit et al. “Enhanced Interactive Gaming by Blending Full-body Tracking and Gesture Animation.” ACM SIGGRAPH ASIA 2010 Sketches. New York, NY, USA: ACM, 2010. 34:1–34:2. Web. 6 July 2012. SA ’10.

3) Borenstein, Greg. Making Things See: 3D Vision with Kinect, Processing, Arduino, and MakerBot. Make, 2012. Print.

4) Boulos, Maged N Kamel et al. INTERNATIONAL JOURNAL OF HEALTH GEOGRAPHICS EDITORIAL Open Access Web GIS in Practice X: a Microsoft Kinect Natural User Interface for Google Earth Navigation. Print.

5) Burba, Nathan et al. “Unobtrusive Measurement of Subtle Nonverbal Behaviors with the Microsoft Kinect.” Proceedings of the 2012 IEEE Virtual Reality. Washington, DC, USA: IEEE Computer Society, 2012. 1–4. Web. 6 July 2012. VR ’12.

6) Center for History and New Media. “Zotero Quick Start Guide.”

7) Clark, Adrian, and Thammathip Piumsomboon. “A Realistic Augmented Reality Racing Game Using a Depth-sensing Camera.” Proceedings of the 10th International Conference on Virtual Reality Continuum and Its Applications in Industry. New York, NY, USA: ACM, 2011. 499–502. Web. 6 July 2012. VRCAI ’11.

8 ) Cui, Yan, and Didier Stricker. “3D Shape Scanning with a Kinect.” ACM SIGGRAPH 2011 Posters. New York, NY, USA: ACM, 2011. 57:1–57:1. Web. 6 July 2012. SIGGRAPH ’11.

9) Davison, Andrew. Kinect Open Source Programming Secrets: Hacking the Kinect with OpenNI, NITE, and Java. 1st ed. McGraw-Hill/TAB Electronics, 2012. Print.

10) Devereux, D. et al. “Using the Microsoft Kinect to Model the Environment of an Anthropomimetic Robot.” Submitted to the Second IASTED International Conference on Robotics (ROBO 2011). Web. 6 July 2012.

11) Dippon, Andreas, and Gudrun Klinker. “KinectTouch: Accuracy Test for a Very Low-cost 2.5D Multitouch Tracking System.” Proceedings of the ACM International Conference on Interactive Tabletops and Surfaces. New York, NY, USA: ACM, 2011. 49–52. Web. 6 July 2012. ITS ’11.

12) Droeschel, David, and Sven Behnke. “3D Body Pose Estimation Using an Adaptive Person Model for Articulated ICP.” Proceedings of the 4th International Conference on Intelligent Robotics and Applications – Volume Part II. Berlin, Heidelberg: Springer-Verlag, 2011. 157–167. Web. 6 July 2012. ICIRA’11.

13) Dutta, Tilak. “Evaluation of the Kinect Sensor for 3-D Kinematic Measurement in the Workplace.” Applied Ergonomics 43.4 (2012): 645–649. Web. 6 July 2012.

14) Engelharda, N. et al. “Real-time 3D Visual SLAM with a Hand-held RGB-D Camera.” Proc. of the RGB-D Workshop on 3D Perception in Robotics at the European Robotics Forum, Vasteras, Sweden. Vol. 2011. 2011. Web. 6 July 2012.

15) Francese, Rita, Ignazio Passero, and Genoveffa Tortora. “Wiimote and Kinect: Gestural User Interfaces Add a Natural Third Dimension to HCI.” Proceedings of the International Working Conference on Advanced Visual Interfaces. New York, NY, USA: ACM, 2012. 116–123. Web. 6 July 2012. AVI ’12.

16) Giles, J. “Inside the Race to Hack the Kinect.” The New Scientist 208.2789 (2010): 22–23. Print.

17) Gill, T. et al. “A System for Change Detection and Human Recognition in Voxel Space Using the Microsoft Kinect Sensor.” Proceedings of the 2011 IEEE Applied Imagery Pattern Recognition Workshop. Washington, DC, USA: IEEE Computer Society, 2011. 1–8. Web. 6 July 2012. AIPR ’11.

18) Gomez, Juan Diego et al. “Toward 3D Scene Understanding via Audio-description: Kinect-iPad Fusion for the Visually Impaired.” The Proceedings of the 13th International ACM SIGACCESS Conference on Computers and Accessibility. New York, NY, USA: ACM, 2011. 293–294. Web. 6 July 2012. ASSETS ’11.

19) Goth, Gregory. “Brave NUI World.” Commun. ACM 54.12 (2011): 14–16. Web. 6 July 2012.

20) Gottfried, Jens-Malte, Janis Fehr, and Christoph S. Garbe. “Computing Range Flow from Multi-modal Kinect Data.” Proceedings of the 7th International Conference on Advances in Visual Computing – Volume Part I. Berlin, Heidelberg: Springer-Verlag, 2011. 758–767. Web. 6 July 2012. ISVC’11.

21) Henry, P. et al. “RGB-D Mapping: Using Kinect-style Depth Cameras for Dense 3D Modeling of Indoor Environments.” The International Journal of Robotics Research (2012): n. pag. Web. 6 July 2012.

22) Henry, Peter et al. “RGB-D Mapping: Using Kinect-style Depth Cameras for Dense 3D Modeling of Indoor Environments.” Int. J. Rob. Res. 31.5 (2012): 647–663. Web. 6 July 2012.

23) Hilliges, Otmar et al. “HoloDesk: Direct 3d Interactions with a Situated See-through Display.” Proceedings of the 2012 ACM Annual Conference on Human Factors in Computing Systems. New York, NY, USA: ACM, 2012. 2421–2430. Web. 6 July 2012. CHI ’12.

24) Izadi, Shahram et al. “KinectFusion: Real-time 3D Reconstruction and Interaction Using a Moving Depth Camera.” Proceedings of the 24th Annual ACM Symposium on User Interface Software and Technology. New York, NY, USA: ACM, 2011. 559–568. Web. 6 July 2012. UIST ’11.

25) Jean, Jared St. Kinect Hacks: Creative Coding Techniques for Motion and Pattern Detection. O’Reilly Media, 2012. Print.

26) Kean, Sean, Jonathan Hall, and Phoenix Perry. Meet the Kinect: An Introduction to Programming Natural User Interfaces. 1st ed. Berkely, CA, USA: Apress, 2011. Print.

27) Kramer, Jeff et al. Hacking the Kinect. 1st ed. Berkely, CA, USA: Apress, 2012. Print.

28) LaViola, Joseph J., and Daniel F. Keefe. “3D Spatial Interaction: Applications for Art, Design, and Science.” ACM SIGGRAPH 2011 Courses. New York, NY, USA: ACM, 2011. 1:1–1:75. Web. 6 July 2012. SIGGRAPH ’11.

29) Li, Li, Yanhao Xu, and Andreas König. “Robust Depth Camera Based Eye Localization for Human-machine Interactions.” Proceedings of the 15th International Conference on Knowledge-based and Intelligent Information and Engineering Systems – Volume Part I. Berlin, Heidelberg: Springer-Verlag, 2011. 424–435. Web. 6 July 2012. KES’11.

30) Livingston, Mark A. et al. “Performance Measurements for the Microsoft Kinect Skeleton.” Proceedings of the 2012 IEEE Virtual Reality. Washington, DC, USA: IEEE Computer Society, 2012. 119–120. Web. 6 July 2012. VR ’12.

31) Melgar, Enrique Ramos, and Ciriaco Castro Diez. Arduino and Kinect Projects: Design, Build, Blow Their Minds. 1st ed. Berkely, CA, USA: Apress, 2012. Print.

32) Miles, Helen C. et al. “A Review of Virtual Environments for Training in Ball Sports.” Computers & Graphics 36.6 (2012): 714–726. Web. 6 July 2012.

33) Miles, Rob. Start Here! Learn the Kinect API. Microsoft Press, 2012. Print.

34) Mitchell, Grethe, and Andy Clarke. “Capturing and Visualising Playground Games and Performance: a Wii and Kinect Based Motion Capture System.” Proceedings of the 2011 International Conference on Electronic Visualisation and the Arts. Swinton, UK, UK: British Computer Society, 2011. 218–225. Web. 6 July 2012. EVA’11.

35) Molyneaux, David. “KinectFusion Rapid 3D Reconstruction and Interaction with Microsoft Kinect.” Proceedings of the International Conference on the Foundations of Digital Games. New York, NY, USA: ACM, 2012. 3–3. Web. 6 July 2012. FDG ’12.

36) Mutto, Carlo Dal, Pietro Zanuttigh, and Guido M. Cortelazzo. Time-of-Flight Cameras and Microsoft Kinect(TM). Springer Publishing Company, Incorporated, 2012. Print.

37) Panger, Galen. “Kinect in the Kitchen: Testing Depth Camera Interactions in Practical Home Environments.” Proceedings of the 2012 ACM Annual Conference Extended Abstracts on Human Factors in Computing Systems Extended Abstracts. New York, NY, USA: ACM, 2012. 1985–1990. Web. 6 July 2012. CHI EA ’12.

38) Pheatt, Chuck, and Jeremiah McMullen. “Programming for the Xbox Kinect Sensor: Tutorial Presentation.” J. Comput. Sci. Coll. 27.5 (2012): 140–141. Print.

39) Raheja, Jagdish L., Ankit Chaudhary, and Kunal Singal. “Tracking of Fingertips and Centers of Palm Using KINECT.” Proceedings of the 2011 Third International Conference on Computational Intelligence, Modelling & Simulation. Washington, DC, USA: IEEE Computer Society, 2011. 248–252. Web. 6 July 2012. CIMSIM ’11.

40) Riche, Nicolas et al. “3D Saliency for Abnormal Motion Selection: The Role of the Depth Map.” Proceedings of the 8th International Conference on Computer Vision Systems. Berlin, Heidelberg: Springer-Verlag, 2011. 143–152. Web. 6 July 2012. ICVS’11.

41) Rogers, Rick. “Kinect with Linux.” Linux J. 2011.207 (2011): n. pag. Web. 6 July 2012.

42) Shrewsbury, Brandon T. “Providing Haptic Feedback Using the Kinect.” The Proceedings of the 13th International ACM SIGACCESS Conference on Computers and Accessibility. New York, NY, USA: ACM, 2011. 321–322. Web. 6 July 2012. ASSETS ’11.

43) Sidik, Mohd Kufaisal bin Mohd et al. “A Study on Natural Interaction for Human Body Motion Using Depth Image Data.” Proceedings of the 2011 Workshop on Digital Media and Digital Content Management. Washington, DC, USA: IEEE Computer Society, 2011. 97–102. Web. 6 July 2012. DMDCM ’11.

44) Smisek, J., M. Jancosek, and T. Pajdla. “3D with Kinect.” Computer Vision Workshops (ICCV Workshops), 2011 IEEE International Conference On. 2011. 1154 –1160.

45) Solaro, John. “The Kinect Digital Out-of-Box Experience.” Computer 44.6 (2011): 97–99. Web. 6 July 2012.

46) Stone, E. E, and M. Skubic. “Evaluation of an Inexpensive Depth Camera for Passive In-home Fall Risk Assessment.” Pervasive Computing Technologies for Healthcare (PervasiveHealth), 2011 5th International Conference On. 2011. 71–77. Web. 6 July 2012.

47) Sturm, J. et al. “Towards a Benchmark for RGB-D SLAM Evaluation.” Proc. of the RGB-D Workshop on Advanced Reasoning with Depth Cameras at Robotics: Science and Systems Conf.(RSS), Los Angeles, USA. Vol. 2. 2011. 3. Web. 6 July 2012.

48) Sung, J. et al. “Human Activity Detection from RGBD Images.” AAAI Workshop on Pattern, Activity and Intent Recognition (PAIR). 2011. Web. 6 July 2012.

49) Tang, John C., Carolyn Wei, and Reena Kawal. “Social Telepresence Bakeoff: Skype Group Video Calling, Google+ Hangouts, and Microsoft Avatar Kinect.” Proceedings of the ACM 2012 Conference on Computer Supported Cooperative Work Companion. New York, NY, USA: ACM, 2012. 37–40. Web. 6 July 2012. CSCW ’12.

50) “The Kinect Revolution.” The New Scientist 208.2789 (2010): 5. Web. 6 July 2012.

51) Tong, Jing et al. “Scanning 3D Full Human Bodies Using Kinects.” IEEE Transactions on Visualization and Computer Graphics 18.4 (2012): 643–650. Web. 6 July 2012.

52) Villaroman, Norman, Dale Rowe, and Bret Swan. “Teaching Natural User Interaction Using OpenNI and the Microsoft Kinect Sensor.” Proceedings of the 2011 Conference on Information Technology Education. New York, NY, USA: ACM, 2011. 227–232. Web. 6 July 2012. SIGITE ’11.

53) “Virtual Reality from the Keyboard/mouse Couple to Kinect.” Annals of Physical and Rehabilitation Medicine 54, Supplement 1.0 (2011): e239. Web. 6 July 2012.

54) Webb, Jarrett, and James Ashley. Beginning Kinect Programming with the Microsoft Kinect SDK. 1st ed. Apress, 2012. Print.

55) Weise, Thibaut et al. “Kinect-based Facial Animation.” SIGGRAPH Asia 2011 Emerging Technologies. New York, NY, USA: ACM, 2011. 1:1–1:1. Web. 6 July 2012. SA ’11.

56) Wilson, Andrew D. Using a Depth Camera as a Touch Sensor. Print.

57) Xu, Yunfei, and Jongeun Choi. “Spatial Prediction with Mobile Sensor Networks Using Gaussian Processes with Built-in Gaussian Markov Random Fields.” Automatica 0 n. pag. Web. 6 July 2012.

58) Zhang, Zhengyou. “Microsoft Kinect Sensor and Its Effect.” IEEE MultiMedia 19.2 (2012): 4–10. Web. 6 July 2012.

[/wptabcontent]

[/wptabs]

[wptabtitle] Getting Started Fast with the Kinect [/wptabtitle]

[wptabcontent]

Getting Started

There are a variety of software that are being or have been developed that provide you with access to the Kinect sensor through interactive GUI’s. Some use different API’s and libraries (OpenNI,Microsoft SDK, etc.) to do this communication with the Kinect and these may conflict with software that you have already installed. So read the sites before downloading and installing. Make sure that you do clean installs. 

In this post we will provide you with a high level introduction to two excellent software suites provided on a free-to-use basis. 

Instant Access

These provide instant access to the sensor data and help to get your project started whether it involves scanning, recording, viewing, or streaming from the Kinect. 

Follow the links at the end of the pages to see an example workflow with each of the two software packages. 
[/wptabcontent]

[wptabtitle]
RGBDemo
[/wptabtitle]

[wptabcontent]

RGB Demo was initially developed by Nicolas Burrus in the RoboticsLab. He then co-founded the Manctl company that now maintains it, helped by various contributors from the opensource community.

Current features

• Grab kinect images and visualize / replay them
• Support for libfreenect and OpenNI/Nite backends
• Extract skeleton data / hand point position (Nite backend)
• Integration with OpenCV and PCL
• Multiple Kinect support and calibration
• Calibrate the camera to get point clouds in metric space (libfreenect)
• Export to meshlab/blender using .ply files
• Demo of 3D scene reconstruction using a freehand Kinect
• Demo of people detection and localization
• Demo of gesture recognition and skeleton tracking using Nite
• Demo of 3D model estimation of objects lying on a table (based on PCL table top object detector)
• Demo of multiple kinect calibration
• Linux, MacOSX and Windows support

[wptabtitle] Brekel Kinect[/wptabtitle]

[wptabcontent]
In my opinion the Brekel Kinect software is the best that I have seen for easily interfacing with the Kinect.

It uses the OpenNI framework, which we show on the GMV here, but the developer of this software also provides their own packaged OpenNI installer that comes with all the required dependencies. 

Current Features

The Brekel Kinect only offers binaries for the Windows platform. It was developed by  Jasper Brekelmans in his free time. 

It allows you to capture 3D objects and to export them to disk for use in 3D packages. It also allows you to do skeleton tracking which can be streamed into Autodesk’s MotionBuilder in realtime, or exported as BVH files.

The greatest things about the Brekel software are that it requires no programming expertise, it has an easy to use GUI, and it has the ability to export almost all of the Kinect’s capabilities in a variety of formats.

The Brekel website offers a bunch of links to other resources, downloads, tutorials and answers to FAQ’s. Check it out at:

Main Site: http://www.brekel.com

[/wptabcontent]

[/wptabs]