| |
The source code of all examples is included in the KinectJLib distribution.
Example 1: Transferring skeletal data to Java
In many Kinect applications the movement of humans are tracked in order to get information about the skeletal joints for further treatment. In native code the video image is continuously captured by the depth camera and tracked for a valid skeleton. This takes about 40 ms. Whenever a skeleton is successfully detected, the joint coordinates are entered into a circular FIFO queue of 1 kByte size. Java retrieves the skeletal data (3D integer coordinates for the 20 skeletal joints and the skeleton id) by polling the native FIFO queue. Class Kinect's getJoints() method retrieves the eldest skeletal data or waits a maximum amount of time (timeout parameter in getJoints()) for the next skeletal data. To prevent the FIFO queue to overflow, getJoints() must be called fast enough (at least 25 times a second).
In this introductory demonstration example we just write the skeletal data into the console.
import ch.aplu.jaw.*;
import ch.aplu.util.*;
import ch.aplu.kinect.*;
import java.awt.*;
public class KinectEx1 implements KinectCloseListener
{
private final String dllPath = // Path of DLL
Kinect.is64bit()? "KinectHandler64" : "KinectHandler";
private final String title = "Kinect Video Frame";
private final int ulx = 10; // Upper left x of window
private final int uly = 20; // Upper left y of window
private final int width = 640; // Width of window in pixels
private final int height = 480; // Height of window in pixels
public KinectEx1()
{
// May be commented-out to send output to system console
new Console(new Position(700, 10),
new Size(500, 500), new Font("Courier", Font.PLAIN, 12));
Kinect kinect = new Kinect(dllPath, title, ulx, uly, width, height,
NativeHandler.WS_DEFAULT);
kinect.addCloseListener(this);
int nbFrame = 0;
Point3D[] joints = new Point3D[20];
for (int i = 0; i < 20; i++)
joints[i] = new Point3D();
while (true)
{
int skeletonId = kinect.getJoints(joints, 20); // Blocks max 200 ms
if (skeletonId > -1) // Valid
{
nbFrame++;
System.out.println("\nFrame #: " + nbFrame);
System.out.println("Skeleton id: " + skeletonId);
for (int i = 0; i < joints.length; i++)
{
System.out.print("(" + joints[i].x + ", " + joints[i].y
+ ", " + joints[i].z + "), ");
if (i % 4 == 0)
System.out.println();
}
System.out.println();
}
else
System.out.println("Invalid skeleton");
}
}
public void notifyClose()
{
System.exit(0);
}
public static void main(String args[])
{
new KinectEx1();
}
} |
Execute the program locally using WebStart.
System prerequisites:
- Windows 7 or higher (32- or 64-bit)
- Installation of the Kinect SDK from http://kinectforwindows.org
- Kinect for Xbox 360 or Kinect for Windows
(both devices are supported)
(If you install the KinectRuntime and not the full SDK, only the Kinect for Windows is supported.)
(Because WebStart needs a GUI console, we redirect System.out to ch.aplu.util.Console.)
In the next example we extract the position of the right hand and display it. We show a small video window, but we want to maintain the full coordinate resolution of 640 x 480 pixels. Therefore we use a window scale factor of 4.
import ch.aplu.jaw.*;
import ch.aplu.util.*;
import ch.aplu.kinect.*;
import java.awt.*;
public class KinectEx1a
{
private String dllPath =
Kinect.is64bit()? "KinectHandler64" : "KinectHandler";
private final int scaleFactor = 4;
private final String title = "Kinect Video Frame";
private final int ulx = 10;
private final int uly = 10;
private final int width = 640 / scaleFactor;
private final int height = 480 / scaleFactor;
public KinectEx1a()
{
// May be commented-out to send output to system console
new Console(new Position(200, 10),
new Size(500, 500), new Font("Courier", Font.PLAIN, 12));
Kinect kinect = new Kinect(dllPath, title, ulx, uly, width, height,
NativeHandler.WS_BORDER | NativeHandler.WS_VISIBLE);
kinect.setWindowScaleFactor(scaleFactor);
Point3D[] joints = new Point3D[20];
for (int i = 0; i < 20; i++)
joints[i] = new Point3D();
while (true)
{
int skeletonId = kinect.getJoints(joints, 20); // Blocks max 200 ms
if (skeletonId > -1) // Valid
{
int handIndex = SkeletonJoint.HAND_RIGHT.ordinal();
System.out.println("(" + joints[handIndex].x + ", "
+ joints[handIndex].y + ", " + joints[handIndex].z + ")");
}
else
System.out.println("Invalid skeleton");
}
}
public static void main(String args[])
{
new KinectEx1a();
}
} |
Execute the program locally using WebStart.
Finally we want to have some fun and use the right hand's position to draw a painting like a painter. To clear your work, just raise your left hand. We create the graphics window using the convenient class GPanel from the package ch.aplu.util. (Download) To increase the fun we use the z-coordinate (distance from camera) to modify the color.
import ch.aplu.jaw.*;
import ch.aplu.util.*;
import ch.aplu.kinect.*;
import java.awt.*;
public class KinectEx2 implements KinectCloseListener
{
private final String dllPath =
Kinect.is64bit() ? "KinectHandler64" : "KinectHandler";
private final String title = "Kinect Video Frame";
private final int ulx = 10;
private final int uly = 10;
private final int width = 640;
private final int height = 480;
public KinectEx2()
{
GPanel p = new GPanel(0, 640, 480, 0);
p.windowPosition(650, 10);
p.color(Color.red);
Kinect kinect = new Kinect(dllPath, title, ulx, uly, width, height,
NativeHandler.WS_DEFAULT);
kinect.addCloseListener(this);
Point3D[] joints = new Point3D[20];
for (int i = 0; i < 20; i++)
joints[i] = new Point3D();
while (true)
{
int skeletonId = kinect.getJoints(joints, 20); // Blocks max 200 ms
if (skeletonId > -1) // Valid
{
int rightHandIndex = SkeletonJoint.HAND_RIGHT.ordinal();
int leftHandIndex = SkeletonJoint.HAND_LEFT.ordinal();
if (joints[leftHandIndex].y < 100)
p.clear();
else
{
int x = joints[rightHandIndex].x;
int y = joints[rightHandIndex].y;
int z = joints[rightHandIndex].z;
p.title("Right hand at: (" + x + "," + y + "," + z + ")");
p.pos(x, y);
int red = -255 * z / 1000 + 510;
int green = -100 * z / 3000 + 100;
int blue = -50 * z / 3000 + 50;
red = Math.min(Math.max(red, 0), 255);
green = Math.min(Math.max(green, 0), 255);
blue = Math.min(Math.max(blue, 0), 255);
p.color(new Color(red, green, blue));
p.fillCircle(10);
}
}
else
p.title("Invalid skeleton");
}
}
public void notifyClose()
{
System.exit(0);
}
public static void main(String args[])
{
new KinectEx2();
}
} |
Execute the program locally using WebStart.
Enjoy!
(Do it better than me..)
KinectGig: A presentation tool to generate keystrokes
Skeletal positions transferred to Java may be used in many different ways. Kinect may be considered as a new kind of input device to replace keyboard inputs and mouse handling. It is very easy to send keystrokes and mouse press/release/move commands to any other application by the class java.awt.Robot. Not only the Java application but any other application that currently has the focus will get the events. In this way it is easy to develop a Java application that can be used in a slide show with Powerpoint or any other presenting tool that needs few keystrokes to flip the slides, e.g. Page up/down or Curser left/right. To customize the program without the need of recompilation, KinectGig reads an external property file KinectGig.properties that may reside in the current directory or the user home folder, where certain parameters can be adapted.
Download KinectGid (includes the source and a Windows executable).
Execute the program locally using WebStart.
The following gestures are implemented:
- Right hand up: START generating keystrokes
- Left hand up: STOP generating keystrokes
- Left hand moves rapidly to the left: LEFT_KEYSTROKE (here: Cursor left)
- Right hand moves rapidly to the left: RIGHT_KEYSTROKE (here: Cursor right)
System prerequisites:
- Windows 7 or higher (32- or 64-bit)
- Installation of the Kinect SDK from http://kinectforwindows.org
- Kinect for Xbox 360 or Kinect for Windows
(both devices are supported)
(If you install the KinectRuntime and not the full SDK, only the Kinect for Windows is supported.)
| |
