SIXTH SENSE TECHNOLOGY

SixthSense is a wearable gestural interface device that augments the physical world with digital information and lets people use natural hand gestures to interact with that information. It was developed by Pranav Mistry, a PhD student in the Fluid Interfaces Group at the MIT Media Lab.


Components and principles

The SixthSense prototype is composed of a pocket projector, a mirror, and a camera. The hardware components are coupled in a pendant-like mobile wearable device. Both the projector and the camera are connected to the mobile computing device in the user’s pocket, working such as :

* The projector: projects visual information, enabling surfaces, walls and physical objects around the wearer to be used as interfaces;
* The camera and hands: recognizes and tracks the user's hand gestures and physical objects using computer-vision based techniques.
* The software program: processes the video stream data took by the camera and tracks the locations of the colored markers (visual tracking of colored fingers) at the tip of the user’s fingers using simple computer-vision techniques. The movements and arrangements of these fiducials are interpreted into gestures that act as interaction instructions for the projected application interfaces.

The maximum number of tracked fingers is only constrained by the number of unique fiducials, thus SixthSense also supports multi-touch and multi-user interaction.

Interest

The SixthSense prototype implements several applications that demonstrate the usefulness, viability and flexibility of the system. The map application lets the user navigate a map displayed on a nearby surface using hand gestures, similar to gestures supported by multi-touch based systems, letting the user zoom in, zoom out or pan using intuitive hand movements. The drawing application lets the user draw on any surface by tracking the fingertip movements of the user’s index finger. SixthSense also recognizes user’s freehand gestures (postures). For example, it implements a gestural camera that takes photos of the scene the user is looking at by detecting the ‘framing’ gesture. The user can stop by any surface or wall and flick through the photos he/she has taken. SixthSense also lets the user draw icons or symbols in the air using the movement of the index finger and recognizes those symbols as interaction instructions. For example, drawing a magnifying glass symbol takes the user to the map application or drawing an ‘@’ symbol lets the user check his mail. The SixthSense system also augments physical objects the user is interacting with by projecting more information about these objects onto them. For example, a newspaper can show live video news or dynamic information can be provided on a regular piece of paper. The gesture of drawing a circle on the user’s wrist projects an analog watch to effectively tell time.

Cost and license

The current prototype system costs approximately $350 to build, mainly due to the micro-projector. The software may be available for free on the model of open and editable freeware


creator of sixth sense














Pranav Mistry is the inventor of SixthSense, a wearable device that enables new interactions between the real world and the world of data.

PRANAV come from Palanpur, a small town of northern Gujarat, india. PRANAV graduated as Masters of Design from IIT Bombay and Masters of Media Arts and Sciences from MiT Media Lab.

PRANAV completed my bachelors degree in Computer Science and Engineering. Before joining Media Lab, PRANAV worked with Microsoft as a UX Researcher. At present, PRANAV a PhD student and Researcher at MiT Medpranava Lab. Thinking pranavs what PRANAV love the most, and fortunately his work to come up with new innovative ideas