It was bound to happen eventually: someone has managed to get Microsoft's Kinect motion-sensing device to work within the PC version of Bethesda's The Elder Scrolls V: Skyrim. The hack was developed by YouTube user KinectFAAST and uses both motion detection and voice commands to get the job done.
"The gestures are simple and intuitive, yet allow for gameplay depth never before seen with the Kinect," KinectFAAST writes. "Simply swing your right arm forward, up, left, or right, and your character will hack with his sword. Or, if you have a spell in your right arm, simply raise your hand or push it forward to cast. Use your left arm to control your character’s view; if you have something equipped like a spell or shield, you left arm will control that as well."
KinectFAAST explains that the gesture control is handled by the Kinect using a middleware program called FAAST that operates using the OpenNI framework and converts user movements into keystrokes. The OpenNI framework doesn't actually support voice control, so he used a program called VAC (Voice Activated Commands) which allows the user to pair spoken words with keystrokes.
"Trying to avoid any type of controller, wires, or anything that might impede movement, I have used a RazorFish Bluetooth headset to send voice commands to VAC," he said. "This device was relatively inexpensive, and probably accounts for some of the gameplay moments where a command is unrecognized."
A video demonstration can be seen below, but it doesn't showcase a number of voice-related features like using "Left hand" and "Right hand" commands, saying "Journal," saying "Hello" and more.
