Sign in with
Sign up | Sign in

Microsoft Reveals Hand Gesturing System for Tablets

By - Source: PatentBolt | B 16 comments

Here's a patent made public just this month that shows Microsoft working on hand gesturing for tablets and Surface.

PatentBolt has discovered a patent application filed by Microsoft that covers real-time hand-gesturing for tablets, tabletops and more. The thing is, the mysterious patent reportedly doesn't explain a whole lot about the company's invention, but rather serves as a simplified prelude to a more detailed application to be submitted at a later date.

"[The system's] sole purpose is to present some concepts disclosed herein in a simplified form as a prelude to the more detailed description that is presented later," Microsoft states in the application.

According to the Redmond company, object detection and recognition are difficult problems in the field of computer vision. Recognition of hand poses in images are also problematic. However, there is a need to provide simple, accurate, fast and computationally inexpensive methods of both object and gesture recognition for many applications.

So as a solution, Microsoft proposes a system that allows the user to drive an application that is displayed on a tablet or projected on a tabletop simply by using hand gestures. The company is looking into finding ways to accurately determine when the user is hovering a hand above the display, and when the user's hand touches the screen.

"A random decision forest is trained to enable recognition of hand poses and objects and optionally also whether those hand poses are touching or not touching a display surface," Microsoft explains. "The random decision forest uses image features such as appearance, shape and optionally stereo image features. In some cases, the training process is cost aware. The resulting recognition system is operable in real-time."

In a Microsoft patent image, the company shows a tablet with a motion-detection camera mounted within a work surface. The camera is arranged to capture images of the user's hands or other objects positioned between the camera and the display. An image processing system will be incorporated into the tablet and may be set up to classify images captured by the camera. The classification information would then by used by a user interface to control the tablet's display and drive a specific application. This system could also be used to control a display projected onto a surface.

For now it's assumed that the motion detection system will be applied to Windows 8 tablets and Surface tablets, and possibly even the company's pico-like projector system and/or device. Microsoft's patent application was originally filed in December 2011 and made public by the U.S. Patent and Trademark Office just this month.

Discuss
Display all 16 comments.
This thread is closed for comments
  • 1 Hide
    Shin-san , April 18, 2012 3:09 PM
    I'm mixed about relying a lot on hand gestures to use a touch screen
  • 0 Hide
    el33t , April 18, 2012 3:25 PM
    I don't know about the rest of you lot but I'm so sick of these new "physically-interactive" technologies.... I don't think I'll ever get accustomed to them...
  • 2 Hide
    arcainumbro , April 18, 2012 3:37 PM
    I'm personally torn. Think about how many things your grandparents said "I just don't get that, or I just don't like that" about. Well, now it's our turn for those statements. Kids will now grow up with gestures and touch interfaces as the norm, so for them anything that we love and try to stick to will be old fashioned. I have a 7 month old daughter right now, and it may become reality that she will rarely use a computer with a mouse. And to her, having to touch a mouse with buttons will be weird and she won't like it.
  • 1 Hide
    Canidae1111 , April 18, 2012 3:47 PM
    santeanaI got a hand-gesture for ya!


    Lol, beat me to it.
  • 2 Hide
    sporkimus , April 18, 2012 4:08 PM
    /waveshand
    "These are not the tablets you are looking for."
  • -1 Hide
    drwho1 , April 18, 2012 4:13 PM
    I already use a hand gesture system for tablets.
    It simulates a "bird gesture".
    It also works for Apple products.
  • 1 Hide
    Anonymous , April 18, 2012 4:26 PM
    The solution comming to Windows PC's, tablets, phone, and XBox is the Kinect technology. To enable "touch" and gestures without actually touching the screen with greesy fingers is comming soon.
  • 1 Hide
    p05esto , April 18, 2012 4:32 PM
    I like it. It would be cool to scroll down a web page by just waving your fingers instead of touching and dragging. That's probably the most obvious application, but I can see some others. It would have to work good, I see this being difficult and glitchy to start with. Kind of like the SUCK that is Siri on iPhones.
  • -1 Hide
    RipperjackAU , April 18, 2012 4:39 PM
    Just imagine having gesture based passwords. You know, you have to look like a spastic, going through a convulsive fit, just to gesture in your password to log in.
  • -1 Hide
    g00fysmiley , April 18, 2012 5:42 PM
    I can see some uses for this, like reading on a tablet, instead of swipign across the page to ge tto the next page you could gestur at it to not leave any kind of finger print. or if you were watchign a movie a gesture for pause, play, rewinds and such

    though I am reminded of the galactic radion in the hithiker's guide to the galaxy where the radio uses voice comands to change stations then people were to lazy for that so they added where a wave of the hand could change stations, so when you found a station you wanted you had to remain perfectly still or it would change.

    i think it'd be a valid concern as imagine you are reading, set your tablet down and it recognises the fan or somethign as a page back or forward, or registers you walkign away for a min and the cat jumping aorund playing over the tablet and suddenly you are a few pages ahead, it'd be interesting to see how it works
  • 0 Hide
    CaedenV , April 18, 2012 6:36 PM
    arcainumbroI'm personally torn. Think about how many things your grandparents said "I just don't get that, or I just don't like that" about. Well, now it's our turn for those statements. Kids will now grow up with gestures and touch interfaces as the norm, so for them anything that we love and try to stick to will be old fashioned. I have a 7 month old daughter right now, and it may become reality that she will rarely use a computer with a mouse. And to her, having to touch a mouse with buttons will be weird and she won't like it.

    Ya, no kidding! I rebuilt my rig last November and it dawned on me: My son will likely never have a traditional desktop computer. He may have an all-in-one that is tied to his desk, but the idea of having an upgradable box is going to be pretty foreign by the time he is ~8-10 and getting his own PC/Laptop/tablet/computer/thing. Touch will be considered 'normal' and standard equipment. Voice input will likely replace keyboards for many applications (something along the lines of SIRI or Google's voice search), but keyboards will still be around for large text input. 3D screens will still not be standard, but likely more than the niche market that it is today. Quad core mobile devices and 8-16 core/thread 'less mobile' devices will be normal, with 32+ core/thread options available (right now duel core is still normal, with 8-12 thread/core available). SSD or similar tech will be standard equipment. Ram will be a 'non factor' with most systems having 8+GB, but still only using 2-4GB for day-to-day use (Perhaps 'ram' will disappear entirely with things running directly from the storage device and CPU cache). Lastly I think that video walls or multi screen displays will catch on by this point.
    Custom computers will mostly be for servers where you may want more storage space than what your SSD based 'end user' devices have but they will not be big bulky servers like today, and for extreme gaming (3-7 4K displays). Your average 'computer' will be able to do everything from gaming to 4K video editing without any need for additional hardware. Once we can do real time ray tracing, and be able to produce retna display quality gaming for under $500 (which we should be able to do in the next 5-10 years) then you will simply pick your display you want, and it will come with a CPU/GPU combination capable to do whatever you need to do on your display.

    Heck, we are already beginning to see this transition as there is less 'need' for CPU and GPU power. My wife's 4 year old PC has more than enough horsepower for office work, browsing, flash games, simple video editing, advanced audio editing and music scoring, HD movie playback, etc., and with an SSD it is pretty quick at all these tasks. My old computer (which was similar to hers, just 6mo older and with a gaming GPU) had more than enough power for my gaming needs and other tasks, and the only reason I upgraded was because I needed something a bit faster for HD 1080p video editing (HD editing on a 5 year old duel core is a bit painful lol). I went a little overboard with the build because I want it to last a long time, but even $800 all-in-one PCs can do HD editing with no problem these days (though rendering the video may take a little time), and if they can do it now, they will be able to do it much better/cheaper in 10 years. Hell, even the iPad can do simple HD video editing, and it is a gutless wonder! I am not saying it is the 'end of the PC' yet, but you can see it from here, and the end will have come by the time my son is ready for a computer.
  • 0 Hide
    del35 , April 18, 2012 7:18 PM
    Ummm, new technology for Apple to claim down the road.
  • 0 Hide
    alxianthelast , April 18, 2012 11:19 PM
    CaedenV

    Over my dead body (with respect to my spawn). Since some of that can be accomplished with 3rd party hardware and software.. you don't need convenience devices.
    The above would reduce the ability to pick and choose the set-up you want and you ability to troubleshoot and repair it yourself. It was the modularity that help push PCs for users and service people.. I'm hoping that never dies having a foot in each of those shoes.
  • 0 Hide
    arlandi , April 19, 2012 7:34 AM
    And to her, having to touch a mouse with buttons will be weird and she won't like it.


    you know what? that reminds me of Scotty in Star Trek Voyage Home when he talked to the mouse...
  • 0 Hide
    jkflipflop98 , April 19, 2012 8:08 AM
    that's exactly right. Everyone said "you can't sail around the world, you'll fall off the edge, stupid!". Then the "experts" said "You can't go faster than sound, you'll explode, stupid!". Now it's "you can't use a computer without a mouse and keyboard, stupid!".
  • 0 Hide
    Anonymous , April 26, 2012 1:43 AM
    uhh... what about when you're trying to type a paper or using Word to type a homework assignment? I would personally hate to do handmotion for that. Or maybe I'm not getting what this is all about...? lol