| |
The source code of all examples is included in the KinectJLib distribution.
Example 3: Kinect Sound Direction Finder
The Kinect microphone array uses 4 microphones in a line and enables the user to perform acoustic localization of the sound source in a horizontal plane. The algorithm to deliver the angle to the sound source is by far not trivial. It is implemented in Microsoft's Kinect native library installed with the Kinect SDK. Java gets access to the code via the JNI based framework JAW. JKinectLib shields the Java programmer completely from the complexity of the underlying code: After enabling the sound direction scanning machine with enableSoundDirection(true), the angle to the sound source is polled by calling getSoundDirection(). This method retrieves the last direction together with a confidence level from the native detection machine.
For a more profound discussion of how the Kinect microphone array works, consult this post or this scientific paper.
We demonstrate the Kinect sound direction finder by the following program where the direction to the sound source is displayed as a beam line in a circle centered a the Kinect device. The line is updated only if the confidence level exceeds a given value (here 30%). Of course the sound direction finder can be combined with a native video window and a skeleton detection. For simplicity we use the GPanel class from the ch.aplu.util package to display the graphics.
System prerequisites:
- Windows 7 or higher (32- or 64-bit)
- Installation of the Kinect SDK from http://kinectforwindows.org
- Kinect for Xbox 360 or Kinect for Windows
(both devices are supported)
(If you install the KinectRuntime and not the full SDK, only the Kinect for Windows is supported.)
import ch.aplu.jaw.NativeHandler;
import ch.aplu.util.*;
import ch.aplu.kinect.*;
import java.awt.*;
public class KinectEx3 extends GPanel
{
private final int confidenceLevel = 30; // Minimum confidence level
private final String dllPath =
Kinect.is64bit() ? "KinectHandler64" : "KinectHandler";
private final String title = "Kinect Video Frame";
public KinectEx3()
{
super(-1, 1, 1, -1);
windowPosition(350, 20);
enableRepaint(false);
addStatusBar(50);
bgColor(Color.black);
showCircle();
drawLine(0);
title("Sound Direction Finder Demo");
setStatusText("Initializing...");
Kinect kinect = new Kinect(dllPath, title, 10, 20, 320, 240,
NativeHandler.WS_BORDER | NativeHandler.WS_VISIBLE);
kinect.enableSoundDirection(true);
setStatusText("Audio Direction Finder ready. Please speak now!");
while (true)
{
SoundDirection ad = kinect.getSoundDirection();
if (ad.confidence >= confidenceLevel)
{
setStatusText("New position: " + ad.position
+ " degrees (Confidence: " + ad.confidence + "%)");
clear();
showCircle();
drawLine(ad.position);
repaint();
}
else
setStatusText("Confidence: " + ad.confidence + "% < " +
confidenceLevel + "% (too small)");
delay(100);
}
}
private void drawLine(int angle)
{
color(Color.yellow);
lineWidth(5);
double a = Math.toRadians(angle);
line(0, 0, 0.9 * Math.sin(a), 0.9 * Math.cos(a));
}
private void showCircle()
{
color(Color.white);
lineWidth(1);
circle(0.8);
}
public static void main(String[] args)
{
new KinectEx3();
}
} |
Execute the program locally using WebStart.
| |
