Microsoft has been working on touchless input for a long while now, even back in the days of the Lumia patenting a “3D Touch” mechanism that allowed users to interact with Metro UI tiles without touching them. Instead, they would hover the finger above them. Now they’re back at it again, with a new patent for a touch-free and smudge free tablet.
Touchscreens have become a staple of today’s life, but they’re starting to get obsolete as people look for the next best way to control a device. Microsoft decided to implement a system for future Surface devices, that will give them touch-free input. It appears in a recently granted patent published by the USPTO on July 10th. It’s simply called “Touchless Input”.
Just like the Apple TrueDepth camera from the iPhone X maps the face the eliminate the need for regular login, this system relies on a similar camera that captures depth maps of an entire scene, including a human subject. We’ve got a detection system built within the device, which scans every pixel in each map, showing what’s junk and what’s actual input with fingers.
It could even model the whole human being, with a virtual skeleton and joints, in order to accurately predict the motions of the hand. The virtual joints are compared across all captured depth maps to determine a finger gesture based on machine learning. The system is able to track more than one human user and physical object. The device is able to map the physical location of the finger to the on-screen cursor.
The user can move the finger in the air or any direction to provide input and commands. The system tracks multiple fingers, so you can perform complex actions, like clicking and dragging files and folders, zooming, panning, double tap, rotating and more. If the sensors are accurate enough, your fingerprints could be scanned in thin air. A visible light camera similar to the depth ones may be used in the system.
This sounds like a nice way to put the successor of a Kinect to work…
source
