Project Natal
Encyclopedia
Kinect, originally known by the code name
Project Natal, is a motion sensing input device
by Microsoft
for the Xbox 360
video game console
. Based around a webcam
-style add-on peripheral
for the Xbox 360 console, it enables users to control and interact with the Xbox 360 without the need to touch a game controller
, through a natural user interface
using gestures and spoken commands
. The project is aimed at broadening the Xbox 360's audience beyond its typical gamer base. Kinect competes with the Wii Remote Plus and PlayStation Move with PlayStation Eye
motion controller
s for the Wii
and PlayStation 3
home consoles, respectively. A version for Windows
will be released in Early 2012.
Kinect was launched in North America on November 4, 2010, in Europe on November 10, 2010, in Australia, New Zealand and Singapore on November 18, 2010, and in Japan on November 20, 2010. Purchase options for the sensor peripheral include a bundle with the game Kinect Adventures
and console bundles with either a 4 GB or 250 GB Xbox 360 console and Kinect Adventures.
After selling a total of 8 million units in its first 60 days, the Kinect holds the Guinness World Record of being the "fastest selling consumer electronics device". 10 million units of the Kinect sensor have been shipped as of March 9, 2011.
Microsoft released a non-commercial Kinect software development kit for Windows 7 on June 16, 2011, with a commercial version following at a later date. This SDK will allow .NET developers to write Kinecting apps in C++/CLI
, C#, or Visual Basic .NET
.
owned by Microsoft, and on range camera
technology by Israeli
developer PrimeSense
, which developed a system that can interpret specific gestures, making completely hands-free control of electronic devices possible by using an infrared
projector and camera and a special microchip to track the movement of objects and individuals in three dimension. This 3D scanner
system called Light Coding employs a variant of image-based 3D reconstruction.
The Kinect sensor is a horizontal bar connected to a small base with a motorized pivot and is designed to be positioned lengthwise above or below the video display. The device features an "RGB
camera, depth sensor and multi-array microphone
running proprietary software", which provide full-body 3D motion capture
, facial recognition
and voice recognition
capabilities. At launch, voice recognition was only made available in Japan, the United Kingdom, Canada and the United States. Mainland Europe will receive the feature in spring 2011. The Kinect sensor's microphone array enables the Xbox 360 to conduct acoustic source localization
and ambient noise suppression
, allowing for things such as headset-free party chat over Xbox Live
.
The depth sensor consists of an infrared
laser
projector combined with a monochrome CMOS sensor
, which captures video data in 3D under any ambient light
conditions. The sensing range of the depth sensor is adjustable, and the Kinect software is capable of automatically calibrating the sensor based on gameplay and the player's physical environment, accommodating for the presence of furniture or other obstacles.
Described by Microsoft personnel as the primary innovation of Kinect, the software technology enables advanced gesture recognition
, facial recognition and voice recognition. According to information supplied to retailers, Kinect is capable of simultaneously tracking up to six people, including two active players for motion analysis
with a feature extraction
of 20 joints per player. However, PrimeSense has stated that the number of people the device can "see" (but not process as players) is only limited by how many will fit in the field-of-view of the camera.
Reverse engineering has determined that the Kinect sensor outputs video at a frame rate
of 30 Hz
. The RGB video stream uses 8-bit VGA resolution (640 × 480 pixel
s) with a Bayer color filter
, while the monochrome depth sensing video stream is in VGA resolution (640 × 480 pixels) with 11-bit depth
, which provides 2,048 levels of sensitivity. The Kinect sensor has a practical ranging
limit of 1.2–3.5 m (3.9–11.5 ft) distance when used with the Xbox software. The area required to play Kinect is roughly 6m², although the sensor can maintain tracking through an extended range of approximately 0.7–6 m (2.3–19.7 ft). The sensor has an angular field of view
of 57°
horizontally and 43° vertically, while the motorized pivot is capable of tilting
the sensor up to 27° either up or down. The horizontal field of the Kinect sensor at the minimum viewing distance of ~0.8 m (2.6 ft) is therefore ~87 cm (34.3 in), and the vertical field is ~63 cm (24.8 in), resulting in a resolution of just over 1.3 mm (0.0511811023622047 in) per pixel. The microphone array features four microphone capsules and operates with each channel processing 16-bit
audio at a sampling rate
of 16 kHz.
Because the Kinect sensor's motorized tilt mechanism requires more power than the Xbox 360's USB
ports can supply, the device makes use of a proprietary connector combining USB communication with additional power. Redesigned Xbox 360 S models include a special AUX port for accommodating the connector, while older models require a special power supply cable (included with the sensor) that splits the connection into separate USB and power connections; power is supplied from the mains
by way of an AC adapter.
, Alexander Shpunt, Aviad Maizels and Javier Garcia.
Kinect itself was first announced on June 1, 2009 at E3 2009
under the code name "Project Natal". Following in Microsoft's tradition of using cities as code names, "Project Natal" was named after the Brazilian city of Natal
as a tribute to the country by Brazilian-born Microsoft director Alex Kipman, who incubated the project. The name Natal was also chosen because the word natal means "of or relating to birth", reflecting Microsoft's view of the project as "the birth of the next generation of home entertainment".
Three demos were shown to showcase Kinect when it was revealed at Microsoft's E3 2009 Media Briefing: Ricochet, Paint Party and Milo & Kate
. A demo based on Burnout Paradise
was also shown outside of Microsoft's media briefing. The skeletal mapping technology shown at E3 2009 was capable of simultaneously tracking four people, with a feature extraction of 48 skeletal
points on a human body at 30 Hz.
It was rumored that the launch of Project Natal would be accompanied with the release of a new Xbox 360 console (as either a new retail configuration, a significant design revision and/or a modest hardware
upgrade). Microsoft dismissed the reports in public and repeatedly emphasized that Project Natal would be fully compatible with all Xbox 360 consoles. Microsoft indicated that the company considers it to be a significant initiative, as fundamental to the Xbox brand as Xbox Live
, and with a launch akin to that of a new Xbox console platform. Kinect was even referred to as a "new Xbox" by Microsoft CEO
Steve Ballmer
at a speech for the Executives' Club of Chicago. When asked if the introduction will extend the time before the next-generation console platform is launched (historically about 5 years between platforms), Microsoft corporate vice president
Shane Kim reaffirmed that the company believes that the life cycle of the Xbox 360 will last through 2015 (10 years).
During Kinect's development, project team members experimentally adapted numerous games to Kinect-based control schemes to help evaluate usability. Among these games were Beautiful Katamari
and Space Invaders Extreme
, which were demonstrated at the Tokyo Game Show
in September 2009. According to creative director Kudo Tsunoda, adding Kinect-based control to pre-existing games would involve significant code alterations, making it unlikely for Kinect features to be added through software updates.
Although the sensor unit was originally planned to contain a microprocessor that would perform operations such as the system's skeletal mapping, it was revealed in January 2010 that the sensor would no longer feature a dedicated processor. Instead, processing would be handled by one of the processor cores of the Xbox 360's Xenon
CPU. According to Alex Kipman, the Kinect system consumes about 10-15% of the Xbox 360's computing resources. However, in November, Alex Kipman made a statement that "the new motion control tech now only uses a single-digit percentage of the Xbox 360's processing power, down from the previously stated 10 to 15 percent." A number of observers commented that the computational load required for Kinect makes the addition of Kinect functionality to pre-existing games through software updates even less likely, with concepts specific to Kinect more likely to be the focus for developers using the platform.
On March 25, Microsoft sent out a save the date
flier for an event called the "World Premiere 'Project Natal' for the Xbox 360 Experience" at E3 2010
. The event took place on the evening of Sunday, June 13, 2010 at the Galen Center
and featured a performance by Cirque du Soleil
. It was announced that the system would officially be called Kinect, a portmanteau of the words "kinetic" and "connect", which describe key aspects of the initiative. Microsoft also announced that the North American launch date for Kinect will be November 4, 2010. Despite previous statements dismissing speculation of a new Xbox 360 to accompany the launch of the new control system, Microsoft announced at E3 2010 that it was introducing a redesigned Xbox 360, complete with a connector port ready for Kinect. In addition, on July 20, 2010, Microsoft announced a Kinect bundle with a redesigned Xbox 360, to be available with the Kinect launch.
On June 16, 2011, Microsoft announced its official release of its SDK for non-commercial use.
On July 21, 2011, Microsoft announced that the first ever white Kinect sensor would be available as part of the "Xbox 360 Limited Edition Kinect Star Wars Bundle", which also includes custom a Star Wars
-themed console and controller, and copies of Kinect Adventures
and Star Wars Kinect. Previously, all Kinect sensors had been glossy black.
On October 31, 2011, Microsoft announced launching of the commercial version of Kinect for Windows program with release of the SDK to companies. David Dennis, Product Manager at Microsoft, said, "There are hundreds of organizations we are working with to help them determine what's possible with the tech".
On November 22, 2011, Microsoft announced that a new Kinect Hardware, specifically designed to connect with PC, will be coming in early 2012.
bottles, commercials during shows such as Dancing with the Stars
and Glee
as well as print ads in various magazines such as People
and InStyle.
On October 19, Microsoft advertised Kinect on The Oprah Winfrey Show
by giving free Xbox 360 consoles and Kinect sensors to the people in the audience. Two weeks later, Kinect bundles with Xbox 360 consoles were also given away to the audience of Late Night With Jimmy Fallon
. On October 23, Microsoft held a pre-launch party for Kinect in Beverly Hills. The party was hosted by Ashley Tisdale
and was attended by soccer star David Beckham
and his three sons, Cruz, Brooklyn, and Romeo. Guests were treated to sessions with Dance Central
and Kinect Adventures
, followed by Tisdale having a Kinect voice chat with Nick Cannon
. Between November 1 and November 28, Burger King
gave away a free Kinect bundle "every 15 minutes".
A major event was organized on November 3 in Times Square
, where singer Ne-Yo
performed with hundreds of dancers in anticipation of Kinect's midnight launch. During the festivities, Microsoft gave away T-shirts and Kinect games.
or video chat with other Xbox 360 users or users of Windows Live Messenger
. The application can use Kinect's tracking functionality and the Kinect sensor's motorized pivot to keep users in frame even as they move around. Other applications with Kinect support include ESPN
, Zune Marketplace, Netflix
, Hulu Plus and Last.fm
Games that require Kinect have a purple sticker on them with a white silhouette of the Kinect sensor and "Requires Kinect Sensor" underneath in white text. Games that have optional Kinect support (meaning that Kinect is not necessary to play the game or that there are optional Kinect minigames included) feature a standard green Xbox 360 case with a purple bar underneath the header, a silhouette of the Kinect sensor and "Better with Kinect Sensor" next to it in white text.
Kinect launched on November 4, 2010 with 17 titles. Third-party publishers of available and announced Kinect games include, among others, Ubisoft
, Electronic Arts
, LucasArts
, THQ
, Activision
, Konami
, Sega
, Capcom
, Namco Bandai
and MTV Games. Along with retail games, there are also select Xbox Live Arcade
that exclusively use the Kinect peripheral.
The SDK includes Windows 7 compatible PC
drivers for Kinect device. It provides Kinect capabilities to developers to build applications with C++
, C#, or Visual Basic
by using Microsoft Visual Studio 2010 and includes following features:
On November 10, Adafruit announced Héctor Martín
as the winner, who had produced a Linux
driver that allows the use of both the RGB camera and depth sensitivity functions of the device. It was later revealed that Johnny Lee
, a core member of Microsoft's Kinect development team, had secretly approached Adafruit with the idea of a driver development contest and had personally financed it.
In December 2010, PrimeSense
, whose depth sensing reference design Kinect is based on, released their own open source drivers along with motion tracking middleware called NITE. PrimeSense later announced that it had teamed up with Asus
to develop a PC-compatible device similar to Kinect, which will be called WAVI Xtion and is scheduled for release in the second quarter of 2011.
combined Kinect with the iRobot Create
to map a room in 3D and have the robot respond to human gestures, while an MIT Media Lab team is working on a JavaScript extension for Google Chrome
called depthJS that allows users to control the browser with hand gestures. Other programmers, including the Robot Locomotion Group at MIT, are using the drivers to develop a motion-controller user interface similar to the one envisioned in the film Minority Report
. The developers of MRPT
have integrated open source drivers into their libraries and provided examples of live 3D rendering and basic 3D visual SLAM
. Another team has shown an application that allows Kinect users to play a virtual piano by tapping their fingers on an empty desk. Oliver Kreylos, a researcher at University of California, Davis
, adopted the technology to improve live 3-dimensional videoconferencing, which NASA
has shown interest in.
Alexandre Alahi from EPFL
presented a video surveillance system that combines multiple Kinect devices to track groups of people even in complete darkness. Companies So touch and Evoluce have developed presentation software for Kinect that can be controlled by hand gestures; among its features is a multi-touch zoom mode. In December 2010, the free public beta of HTPC
software KinEmote was launched; it allows navigation of Boxee
and XBMC menus using a Kinect sensor. Soroush Falahati wrote an application that can be used to create stereoscopic
3D images with a Kinect sensor.
For a limited time in May 2011, a Topshop
store in Moscow set up a Kinect kiosk that could overlay a collection of dresses onto the live video feed of customers. Through automatic tracking, position and rotation of the virtual dress were updated even as customers turned around to see the back of the outfit.
Kinect also shows compelling potential for use in medicine. Researchers at the University of Minnesota
have used Kinect to measure a range of disorder symptoms in children, creating new ways of objective evaluation to detect such conditions as autism, attention-deficit disorder and obsessive-compulsive disorder. At the Institute of Forensic Medicine Virtopsy Project at the University of Bern in Switzerland
, researchers have devised a way for surgeons to manipulate imaging techniques (such as MRI) to guide surgery, using a Kinect to capture their hand motions to direct the imaging, freeing their hands from having to use computer keyboards, thus also reducing the chance of contamination. This technique is already working at Sunnybrook Health Sciences Centre
in Toronto
, where doctors use it to guide imaging during cancer surgery.
gave the device 7.5 out of 10, saying that "Kinect can be a tremendous amount of fun for casual players, and the creative, controller-free concept is undeniably appealing", though adding that for "$149.99, a motion-tracking camera add-on for the Xbox 360 is a tough sell, especially considering that the entry level variation of the Xbox 360 itself is only $199.99". Game Informer
rated Kinect 8 out of 10, praising the technology but noting that the experience takes a while to get used to and that the spatial requirement may pose a barrier. Computer and Video Games
called the device a technological gem and applauded the gesture and voice controls, while criticizing the launch lineup and Kinect Hub.
CNET
's review pointed out how Kinect keeps players active with its full-body motion sensing but criticized the learning curve, the additional power supply needed for older Xbox 360 consoles and the space requirements. Engadget
, too, listed the large space requirements as a negative, along with Kinect's launch lineup and the slowness of the hand gesture UI. The review praised the system's powerful technology and the potential of its yoga and dance games. Kotaku
considered the device revolutionary upon first use but noted that games were sometimes unable to recognize gestures or had slow responses, concluding that Kinect is "not must-own yet, more like must-eventually own." TechRadar
praised the voice control and saw a great deal of potential in the device whose lag and space requirements were identified as issues. Gizmodo
also noted Kinect's potential and expressed curiosity in how more mainstream titles would utilize the technology. Ars Technica
's review expressed concern that the core feature of Kinect, its lack of a controller, would hamper development of games beyond those that have either stationary players or control the player's movement automatically.
The mainstream press also reviewed Kinect. USA Today
compared it to the futuristic control scheme seen in Minority Report
, stating that "playing games feels great" and giving the device 3.5 out of 4 stars. David Pogue from The New York Times
predicted you will feel a "crazy, magical, omigosh rush the first time you try the Kinect." Despite calling the motion tracking less precise than Wii
's implementation, Pogue concluded that "Kinect’s astonishing technology creates a completely new activity that’s social, age-spanning and even athletic." The Globe and Mail
titled Kinect as setting a "new standard for motion control." The slight input lag between making a physical movement and Kinect registering it was not considered a major issue with most games, and the review called Kinect "a good and innovative product," rating it 3.5 out of 4 stars.
Code name
A code name or cryptonym is a word or name used clandestinely to refer to another name or word. Code names are often used for military purposes, or in espionage...
Project Natal, is a motion sensing input device
Input device
In computing, an input device is any peripheral used to provide data and control signals to an information processing system such as a computer or other information appliance...
by Microsoft
Microsoft
Microsoft Corporation is an American public multinational corporation headquartered in Redmond, Washington, USA that develops, manufactures, licenses, and supports a wide range of products and services predominantly related to computing through its various product divisions...
for the Xbox 360
Xbox 360
The Xbox 360 is the second video game console produced by Microsoft and the successor to the Xbox. The Xbox 360 competes with Sony's PlayStation 3 and Nintendo's Wii as part of the seventh generation of video game consoles...
video game console
Video game console
A video game console is an interactive entertainment computer or customized computer system that produces a video display signal which can be used with a display device to display a video game...
. Based around a webcam
Webcam
A webcam is a video camera that feeds its images in real time to a computer or computer network, often via USB, ethernet, or Wi-Fi.Their most popular use is the establishment of video links, permitting computers to act as videophones or videoconference stations. This common use as a video camera...
-style add-on peripheral
Peripheral
A peripheral is a device attached to a host computer, but not part of it, and is more or less dependent on the host. It expands the host's capabilities, but does not form part of the core computer architecture....
for the Xbox 360 console, it enables users to control and interact with the Xbox 360 without the need to touch a game controller
Game controller
A game controller is a device used with games or entertainment systems used to control a playable character or object, or otherwise provide input in a computer game. A controller is typically connected to a game console or computer by means of a wire, cord or nowadays, by means of wireless connection...
, through a natural user interface
Natural user interface
In computing, a natural user interface, or NUI, is the common parlance used by designers and developers of computer interfaces to refer to a user interface that is effectively invisible, or becomes invisible with successive learned interactions, to its users. The word natural is used because most...
using gestures and spoken commands
Speech recognition
Speech recognition converts spoken words to text. The term "voice recognition" is sometimes used to refer to recognition systems that must be trained to a particular speaker—as is the case for most desktop recognition software...
. The project is aimed at broadening the Xbox 360's audience beyond its typical gamer base. Kinect competes with the Wii Remote Plus and PlayStation Move with PlayStation Eye
PlayStation Eye
The PlayStation Eye is a digital camera device, similar to a webcam, for the PlayStation 3. The technology uses computer vision and gesture recognition to process images taken by the camera. This allows players to interact with games using motion and color detection as well as sound through its...
motion controller
Motion Controller
A motion controller controls the motion of some object. Frequently motion controllers are implemented using digital computers, but motion controllers can also be implemented with only analog components as well.-Implementation:...
s for the Wii
Wii
The Wii is a home video game console released by Nintendo on November 19, 2006. As a seventh-generation console, the Wii primarily competes with Microsoft's Xbox 360 and Sony's PlayStation 3. Nintendo states that its console targets a broader demographic than that of the two others...
and PlayStation 3
PlayStation 3
The is the third home video game console produced by Sony Computer Entertainment and the successor to the PlayStation 2 as part of the PlayStation series. The PlayStation 3 competes with Microsoft's Xbox 360 and Nintendo's Wii as part of the seventh generation of video game consoles...
home consoles, respectively. A version for Windows
Microsoft Windows
Microsoft Windows is a series of operating systems produced by Microsoft.Microsoft introduced an operating environment named Windows on November 20, 1985 as an add-on to MS-DOS in response to the growing interest in graphical user interfaces . Microsoft Windows came to dominate the world's personal...
will be released in Early 2012.
Kinect was launched in North America on November 4, 2010, in Europe on November 10, 2010, in Australia, New Zealand and Singapore on November 18, 2010, and in Japan on November 20, 2010. Purchase options for the sensor peripheral include a bundle with the game Kinect Adventures
Kinect Adventures
Kinect Adventures is a 2010 sports video game for the Xbox 360, which utilizes the Kinect motion camera and is included as a pack-in game with the device. It was officially unveiled at the 2010 Electronic Entertainment Expo in Los Angeles...
and console bundles with either a 4 GB or 250 GB Xbox 360 console and Kinect Adventures.
After selling a total of 8 million units in its first 60 days, the Kinect holds the Guinness World Record of being the "fastest selling consumer electronics device". 10 million units of the Kinect sensor have been shipped as of March 9, 2011.
Microsoft released a non-commercial Kinect software development kit for Windows 7 on June 16, 2011, with a commercial version following at a later date. This SDK will allow .NET developers to write Kinecting apps in C++/CLI
C++/CLI
C++/CLI is Microsoft's language specification intended to supersede Managed Extensions for C++. It is a complete revision that aims to simplify the older Managed C++ syntax . C++/CLI is standardized by Ecma as ECMA-372...
, C#, or Visual Basic .NET
Visual Basic .NET
Visual Basic .NET , is an object-oriented computer programming language that can be viewed as an evolution of the classic Visual Basic , which is implemented on the .NET Framework...
.
Technology
Kinect builds on software technology developed internally by Rare, a subsidiary of Microsoft Game StudiosMicrosoft Game Studios
Microsoft Studios is the video game production wing for Microsoft, responsible for the development and publishing of games for the Xbox, Xbox 360, Games for Windows and Windows Phone platforms. They were established in 2002 as Microsoft Game Studios to coincide with the release of the Xbox, before...
owned by Microsoft, and on range camera
Range imaging
Range imaging is the name for a collection of techniques which are used to produce a 2D image showing the distance to points in a scene from a specific point, normally associated with some type of sensor device....
technology by Israeli
Israel
The State of Israel is a parliamentary republic located in the Middle East, along the eastern shore of the Mediterranean Sea...
developer PrimeSense
PrimeSense
PrimeSense is a Israeli company founded in 2005 by Aviad Maizels, Alexander Shpunt, Ophir Sharon, Tamir Berliner and Dima Rais. The company was selected by MIT technological review as one of world's 50 most innovative companies for 2011....
, which developed a system that can interpret specific gestures, making completely hands-free control of electronic devices possible by using an infrared
Infrared
Infrared light is electromagnetic radiation with a wavelength longer than that of visible light, measured from the nominal edge of visible red light at 0.74 micrometres , and extending conventionally to 300 µm...
projector and camera and a special microchip to track the movement of objects and individuals in three dimension. This 3D scanner
3D scanner
A 3D scanner is a device that analyzes a real-world object or environment to collect data on its shape and possibly its appearance . The collected data can then be used to construct digital, three dimensional models....
system called Light Coding employs a variant of image-based 3D reconstruction.
The Kinect sensor is a horizontal bar connected to a small base with a motorized pivot and is designed to be positioned lengthwise above or below the video display. The device features an "RGB
RGB color space
An RGB color space is any additive color space based on the RGB color model. A particular RGB color space is defined by the three chromaticities of the red, green, and blue additive primaries, and can produce any chromaticity that is the triangle defined by those primary colors...
camera, depth sensor and multi-array microphone
Microphone array
A microphone array is any number of microphones operating in tandem. There are many applications:*Systems for extracting voice input from ambient noise *Surround sound and related technologies...
running proprietary software", which provide full-body 3D motion capture
Motion capture
Motion capture, motion tracking, or mocap are terms used to describe the process of recording movement and translating that movement on to a digital model. It is used in military, entertainment, sports, and medical applications, and for validation of computer vision and robotics...
, facial recognition
Three-dimensional face recognition
Three-dimensional face recognition is a modality of facial recognition methods in which the three-dimensional geometry of the human face is used...
and voice recognition
Speaker recognition
Speaker recognition is the computing task of validating a user's claimed identity using characteristics extracted from their voices .There is a difference between speaker recognition and speech recognition . These two terms are frequently confused, as is voice recognition...
capabilities. At launch, voice recognition was only made available in Japan, the United Kingdom, Canada and the United States. Mainland Europe will receive the feature in spring 2011. The Kinect sensor's microphone array enables the Xbox 360 to conduct acoustic source localization
Acoustic source localization
Acoustic source localization is the task of locating a sound source given measurements of the sound field. The sound field can be described using physical quantities like sound pressure and particle velocity. By measuring these properties it is possible to obtain a source...
and ambient noise suppression
Active noise control
Active noise control is a method for reducing unwanted sound.- Explanation :...
, allowing for things such as headset-free party chat over Xbox Live
Xbox Live
Xbox Live is an online multiplayer gaming and digital media delivery service created and operated by Microsoft Corporation. It is currently the only online gaming service on consoles that charges users a fee to play multiplayer gaming. It was first made available to the Xbox system in 2002...
.
The depth sensor consists of an infrared
Infrared
Infrared light is electromagnetic radiation with a wavelength longer than that of visible light, measured from the nominal edge of visible red light at 0.74 micrometres , and extending conventionally to 300 µm...
laser
Laser
A laser is a device that emits light through a process of optical amplification based on the stimulated emission of photons. The term "laser" originated as an acronym for Light Amplification by Stimulated Emission of Radiation...
projector combined with a monochrome CMOS sensor
Active pixel sensor
An active-pixel sensor is an image sensor consisting of an integrated circuit containing an array of pixel sensors, each pixel containing a photodetector and an active amplifier. There are many types of active pixel sensors including the CMOS APS used most commonly in cell phone cameras, web...
, which captures video data in 3D under any ambient light
Available light
In photography and cinematography, available light or ambient light refers to any source of light that is not explicitly supplied by the photographer for the purpose of taking photos. The term usually refers to sources of light that are already available naturally or artificial light already being...
conditions. The sensing range of the depth sensor is adjustable, and the Kinect software is capable of automatically calibrating the sensor based on gameplay and the player's physical environment, accommodating for the presence of furniture or other obstacles.
Described by Microsoft personnel as the primary innovation of Kinect, the software technology enables advanced gesture recognition
Gesture recognition
Gesture recognition is a topic in computer science and language technology with the goal of interpreting human gestures via mathematical algorithms. Gestures can originate from any bodily motion or state but commonly originate from the face or hand. Current focuses in the field include emotion...
, facial recognition and voice recognition. According to information supplied to retailers, Kinect is capable of simultaneously tracking up to six people, including two active players for motion analysis
Motion analysis
Motion analysis is a topic in computer vision, image processing, and machine vision that studies methods and applications in which two or more consecutive images from an image sequences, e.g., produced by a video camera, are processed to produce information based on the apparent motion in the images...
with a feature extraction
Feature extraction
In pattern recognition and in image processing, feature extraction is a special form of dimensionality reduction.When the input data to an algorithm is too large to be processed and it is suspected to be notoriously redundant then the input data will be transformed into a reduced representation...
of 20 joints per player. However, PrimeSense has stated that the number of people the device can "see" (but not process as players) is only limited by how many will fit in the field-of-view of the camera.
Reverse engineering has determined that the Kinect sensor outputs video at a frame rate
Frame rate
Frame rate is the frequency at which an imaging device produces unique consecutive images called frames. The term applies equally well to computer graphics, video cameras, film cameras, and motion capture systems...
of 30 Hz
Hertz
The hertz is the SI unit of frequency defined as the number of cycles per second of a periodic phenomenon. One of its most common uses is the description of the sine wave, particularly those used in radio and audio applications....
. The RGB video stream uses 8-bit VGA resolution (640 × 480 pixel
Pixel
In digital imaging, a pixel, or pel, is a single point in a raster image, or the smallest addressable screen element in a display device; it is the smallest unit of picture that can be represented or controlled....
s) with a Bayer color filter
Bayer filter
A Bayer filter mosaic is a color filter array for arranging RGB color filters on a square grid of photosensors. Its particular arrangement of color filters is used in most single-chip digital image sensors used in digital cameras, camcorders, and scanners to create a color image...
, while the monochrome depth sensing video stream is in VGA resolution (640 × 480 pixels) with 11-bit depth
Color depth
In computer graphics, color depth or bit depth is the number of bits used to represent the color of a single pixel in a bitmapped image or video frame buffer. This concept is also known as bits per pixel , particularly when specified along with the number of bits used...
, which provides 2,048 levels of sensitivity. The Kinect sensor has a practical ranging
Ranging
Ranging is a process or method to determine the distance from one location or position to another location or position. Another term for this method is lateration, see unilateration...
limit of 1.2–3.5 m (3.9–11.5 ft) distance when used with the Xbox software. The area required to play Kinect is roughly 6m², although the sensor can maintain tracking through an extended range of approximately 0.7–6 m (2.3–19.7 ft). The sensor has an angular field of view
Angle of view
In photography, angle of view describes the angular extent of a given scene that is imaged by a camera. It is used interchangeably with the more general term field of view....
of 57°
Degree (angle)
A degree , usually denoted by ° , is a measurement of plane angle, representing 1⁄360 of a full rotation; one degree is equivalent to π/180 radians...
horizontally and 43° vertically, while the motorized pivot is capable of tilting
Tilt (camera)
Tilting is a cinematographic technique in which the camera is stationary and rotates in a vertical plane . A rotation in a horizontal plane is known as panning...
the sensor up to 27° either up or down. The horizontal field of the Kinect sensor at the minimum viewing distance of ~0.8 m (2.6 ft) is therefore ~87 cm (34.3 in), and the vertical field is ~63 cm (24.8 in), resulting in a resolution of just over 1.3 mm (0.0511811023622047 in) per pixel. The microphone array features four microphone capsules and operates with each channel processing 16-bit
Audio bit depth
In digital audio, bit depth describes the number of bits of information recorded for each sample. Bit depth directly corresponds to the resolution of each sample in a set of digital audio data...
audio at a sampling rate
Sampling rate
The sampling rate, sample rate, or sampling frequency defines the number of samples per unit of time taken from a continuous signal to make a discrete signal. For time-domain signals, the unit for sampling rate is hertz , sometimes noted as Sa/s...
of 16 kHz.
Because the Kinect sensor's motorized tilt mechanism requires more power than the Xbox 360's USB
Universal Serial Bus
USB is an industry standard developed in the mid-1990s that defines the cables, connectors and protocols used in a bus for connection, communication and power supply between computers and electronic devices....
ports can supply, the device makes use of a proprietary connector combining USB communication with additional power. Redesigned Xbox 360 S models include a special AUX port for accommodating the connector, while older models require a special power supply cable (included with the sensor) that splits the connection into separate USB and power connections; power is supplied from the mains
Mains electricity
Mains is the general-purpose alternating current electric power supply. In the US, electric power is referred to by several names including household power, household electricity, powerline, domestic power, wall power, line power, AC power, city power, street power, and grid power...
by way of an AC adapter.
History
The technology behind Kinect was invented in 2005 by Zeev ZalevskyZeev Zalevsky
Zeev Zalevsky ; was born in the Russian SFSR in the Soviet Union in 1971 and came with his family to Israel. Zalevsky is an Israeli physicist specializing in Optoelectronics. He is a Professor of Electrical engineering and Nanophotonics at Bar Ilan University in Ramat Gan, Israel...
, Alexander Shpunt, Aviad Maizels and Javier Garcia.
Kinect itself was first announced on June 1, 2009 at E3 2009
Electronic Entertainment Expo 2009
The Electronic Entertainment Expo 2009, commonly known as the E3 2009, was the 15th Electronic Entertainment Expo held. The event took place on June 2, 3, and 4 at the Los Angeles Convention Center. E3 is an annual trade show for the computer and video games industry presented by the Entertainment...
under the code name "Project Natal". Following in Microsoft's tradition of using cities as code names, "Project Natal" was named after the Brazilian city of Natal
Natal, Rio Grande do Norte
-History:The northeastern tip of South America, Cabo São Roque, to the north of Natal and the closest point to Europe from Latin America, was first visited by European navigators in 1501, in the 1501–1502 Portuguese expedition led by Amerigo Vespucci, who named the spot after the saint of the day...
as a tribute to the country by Brazilian-born Microsoft director Alex Kipman, who incubated the project. The name Natal was also chosen because the word natal means "of or relating to birth", reflecting Microsoft's view of the project as "the birth of the next generation of home entertainment".
Three demos were shown to showcase Kinect when it was revealed at Microsoft's E3 2009 Media Briefing: Ricochet, Paint Party and Milo & Kate
Milo and Kate
Project Milo was a project in development by Lionhead Studios for the Xbox 360 video game console...
. A demo based on Burnout Paradise
Burnout Paradise
Burnout Paradise is the seventh game in the Burnout racing video game series. It was developed by Criterion Games and published by Electronic Arts. It was released in January 2008 for PlayStation 3 and Xbox 360 and on February 5, 2009 for Microsoft Windows...
was also shown outside of Microsoft's media briefing. The skeletal mapping technology shown at E3 2009 was capable of simultaneously tracking four people, with a feature extraction of 48 skeletal
Skeletal animation
Skeletal animation is a technique in computer animation in which a character is represented in two parts: a surface representation used to draw the character and a hierarchical set of interconnected bones used to animate the mesh...
points on a human body at 30 Hz.
It was rumored that the launch of Project Natal would be accompanied with the release of a new Xbox 360 console (as either a new retail configuration, a significant design revision and/or a modest hardware
Xbox 360 hardware
The hardware of the Xbox 360 includes several parts, both in the console itself and as separate pieces of kit, or accessories.- Central processing unit :...
upgrade). Microsoft dismissed the reports in public and repeatedly emphasized that Project Natal would be fully compatible with all Xbox 360 consoles. Microsoft indicated that the company considers it to be a significant initiative, as fundamental to the Xbox brand as Xbox Live
Xbox Live
Xbox Live is an online multiplayer gaming and digital media delivery service created and operated by Microsoft Corporation. It is currently the only online gaming service on consoles that charges users a fee to play multiplayer gaming. It was first made available to the Xbox system in 2002...
, and with a launch akin to that of a new Xbox console platform. Kinect was even referred to as a "new Xbox" by Microsoft CEO
Chief executive officer
A chief executive officer , managing director , Executive Director for non-profit organizations, or chief executive is the highest-ranking corporate officer or administrator in charge of total management of an organization...
Steve Ballmer
Steve Ballmer
Steven Anthony "Steve" Ballmer is an American business magnate. He is the chief executive officer of Microsoft, having held that post since January 2000. , his personal wealth is estimated at US$13.9 billion, ranking number 19 on the Forbes 400.-Early life:Ballmer was born in Detroit, Michigan to...
at a speech for the Executives' Club of Chicago. When asked if the introduction will extend the time before the next-generation console platform is launched (historically about 5 years between platforms), Microsoft corporate vice president
Vice president
A vice president is an officer in government or business who is below a president in rank. The name comes from the Latin vice meaning 'in place of'. In some countries, the vice president is called the deputy president...
Shane Kim reaffirmed that the company believes that the life cycle of the Xbox 360 will last through 2015 (10 years).
During Kinect's development, project team members experimentally adapted numerous games to Kinect-based control schemes to help evaluate usability. Among these games were Beautiful Katamari
Beautiful Katamari
Beautiful Katamari, released in Japan as , is a video game by Namco Bandai for the Xbox 360. Beautiful Katamari is the fourth game in the Katamari series of games following Katamari Damacy, We Love Katamari and Me & My Katamari....
and Space Invaders Extreme
Space Invaders Extreme
is a re-vamped incarnation of the classic arcade game Space Invaders. The DS and PSP versions were released to mark the 30th anniversary of Space Invaders which saw its original arcade release in...
, which were demonstrated at the Tokyo Game Show
Tokyo Game Show
The , commonly known as TGS, is a video game expo / convention held annually in the Makuhari Messe, in Chiba, Japan. It is presented by the Computer Entertainment Supplier's Association and the Nikkei Business Publications, Inc...
in September 2009. According to creative director Kudo Tsunoda, adding Kinect-based control to pre-existing games would involve significant code alterations, making it unlikely for Kinect features to be added through software updates.
Although the sensor unit was originally planned to contain a microprocessor that would perform operations such as the system's skeletal mapping, it was revealed in January 2010 that the sensor would no longer feature a dedicated processor. Instead, processing would be handled by one of the processor cores of the Xbox 360's Xenon
Xenon (processor)
Xenon is a CPU that is used in the Xbox 360 game console. The processor, internally codenamed "Waternoose", which was named after Henry J. Waternoose III in Monsters Inc. by IBM and XCPU by Microsoft, is based on IBM's PowerPC instruction set architecture, consisting of three independent processor...
CPU. According to Alex Kipman, the Kinect system consumes about 10-15% of the Xbox 360's computing resources. However, in November, Alex Kipman made a statement that "the new motion control tech now only uses a single-digit percentage of the Xbox 360's processing power, down from the previously stated 10 to 15 percent." A number of observers commented that the computational load required for Kinect makes the addition of Kinect functionality to pre-existing games through software updates even less likely, with concepts specific to Kinect more likely to be the focus for developers using the platform.
On March 25, Microsoft sent out a save the date
Save the date
A save the date is a notice stating the date of a significant event such as a wedding. It is typically sent to those likely to be interested or affected by the event and states the planned date of the occasion so that recipients who wish to attend know to keep that date free.For weddings, save the...
flier for an event called the "World Premiere 'Project Natal' for the Xbox 360 Experience" at E3 2010
Electronic Entertainment Expo 2010
The Electronic Entertainment Expo 2010, commonly known as E3 2010, was the 16th Electronic Entertainment Expo held. The event took place on June 14–17 at the Los Angeles Convention Center. E3 is an annual trade show for the computer and video games industry presented by the Entertainment Software...
. The event took place on the evening of Sunday, June 13, 2010 at the Galen Center
Galen Center
The Galen Center is a multipurpose indoor arena and athletic facility owned and operated by the University of Southern California. Located at the southeast corner of Jefferson Boulevard and Figueroa Street in the Exposition Park area of Los Angeles, it is right across the street from the campus and...
and featured a performance by Cirque du Soleil
Cirque du Soleil
Cirque du Soleil , is a Canadian entertainment company, self-described as a "dramatic mix of circus arts and street entertainment." Based in Montreal, Quebec, Canada, and located in the inner-city area of Saint-Michel, it was founded in Baie-Saint-Paul in 1984 by two former street performers, Guy...
. It was announced that the system would officially be called Kinect, a portmanteau of the words "kinetic" and "connect", which describe key aspects of the initiative. Microsoft also announced that the North American launch date for Kinect will be November 4, 2010. Despite previous statements dismissing speculation of a new Xbox 360 to accompany the launch of the new control system, Microsoft announced at E3 2010 that it was introducing a redesigned Xbox 360, complete with a connector port ready for Kinect. In addition, on July 20, 2010, Microsoft announced a Kinect bundle with a redesigned Xbox 360, to be available with the Kinect launch.
On June 16, 2011, Microsoft announced its official release of its SDK for non-commercial use.
On July 21, 2011, Microsoft announced that the first ever white Kinect sensor would be available as part of the "Xbox 360 Limited Edition Kinect Star Wars Bundle", which also includes custom a Star Wars
Star Wars
Star Wars is an American epic space opera film series created by George Lucas. The first film in the series was originally released on May 25, 1977, under the title Star Wars, by 20th Century Fox, and became a worldwide pop culture phenomenon, followed by two sequels, released at three-year...
-themed console and controller, and copies of Kinect Adventures
Kinect Adventures
Kinect Adventures is a 2010 sports video game for the Xbox 360, which utilizes the Kinect motion camera and is included as a pack-in game with the device. It was officially unveiled at the 2010 Electronic Entertainment Expo in Los Angeles...
and Star Wars Kinect. Previously, all Kinect sensors had been glossy black.
On October 31, 2011, Microsoft announced launching of the commercial version of Kinect for Windows program with release of the SDK to companies. David Dennis, Product Manager at Microsoft, said, "There are hundreds of organizations we are working with to help them determine what's possible with the tech".
On November 22, 2011, Microsoft announced that a new Kinect Hardware, specifically designed to connect with PC, will be coming in early 2012.
Launch
Microsoft had an advertising budget of US$500 million for the launch of Kinect, a larger sum than the investment at launch of the Xbox console. The marketing campaign You Are the Controller, aiming to reach new audiences, included advertisements on Kellogg's cereal boxes and PepsiPepsi
Pepsi is a carbonated soft drink that is produced and manufactured by PepsiCo...
bottles, commercials during shows such as Dancing with the Stars
Dancing with the Stars
Dancing with the Stars is the name of several international television series based on the format of the British TV series Strictly Come Dancing, which is distributed by BBC Worldwide – the commercial arm of the BBC. Currently the format has been licensed to over 35 countries...
and Glee
Glee (TV series)
Glee is an American musical comedy-drama television series that airs on Fox in the United States, and on GlobalTV in Canada. It focuses on the high school glee club New Directions competing on the show choir competition circuit, while its members deal with relationships, sexuality and social issues...
as well as print ads in various magazines such as People
People (magazine)
In 1998, the magazine introduced a version targeted at teens called Teen People. However, on July 27, 2006, the company announced it would shut down publication of Teen People immediately. The last issue to be released was scheduled for September 2006. Subscribers to this magazine received...
and InStyle.
On October 19, Microsoft advertised Kinect on The Oprah Winfrey Show
The Oprah Winfrey Show
The Oprah Winfrey Show is an American syndicated talk show hosted and produced by its namesake Oprah Winfrey. It ran nationally for 25 seasons beginning in 1986, before concluding in 2011. It is the highest-rated talk show in American television history....
by giving free Xbox 360 consoles and Kinect sensors to the people in the audience. Two weeks later, Kinect bundles with Xbox 360 consoles were also given away to the audience of Late Night With Jimmy Fallon
Late Night with Jimmy Fallon
Late Night with Jimmy Fallon is an American late-night talk show hosted by Jimmy Fallon on NBC. The show premiered on March 2, 2009, as the third incarnation of the Late Night franchise originated by David Letterman....
. On October 23, Microsoft held a pre-launch party for Kinect in Beverly Hills. The party was hosted by Ashley Tisdale
Ashley Tisdale
Ashley Michelle Tisdale is an American actress and singer who rose to prominence portraying the candy-counter girl Maddie Fitzpatrick in Disney Channel's The Suite Life of Zack & Cody and the female antagonist Sharpay Evans in the High School Musical film series...
and was attended by soccer star David Beckham
David Beckham
David Robert Joseph Beckham, OBE is an English footballer who plays midfield for Los Angeles Galaxy in Major League Soccer, having previously played for Manchester United, Preston North End, Real Madrid, and A.C...
and his three sons, Cruz, Brooklyn, and Romeo. Guests were treated to sessions with Dance Central
Dance Central
Dance Central is a music video game for the Xbox 360 that utilizes the Kinect motion peripheral. The game was developed by Harmonix, creators of the Guitar Hero and Rock Band franchises, and is a launch title for Kinect...
and Kinect Adventures
Kinect Adventures
Kinect Adventures is a 2010 sports video game for the Xbox 360, which utilizes the Kinect motion camera and is included as a pack-in game with the device. It was officially unveiled at the 2010 Electronic Entertainment Expo in Los Angeles...
, followed by Tisdale having a Kinect voice chat with Nick Cannon
Nick Cannon
Nicholas Scott "Nick" Cannon is an American actor, comedian, rapper, entrepreneur, record producer, radio, and television personality. On television, Cannon began as a teenage sketch comedian on All That before going on to host The Nick Cannon Show, Wild 'N Out, and America's Got Talent...
. Between November 1 and November 28, Burger King
Burger King
Burger King, often abbreviated as BK, is a global chain of hamburger fast food restaurants headquartered in unincorporated Miami-Dade County, Florida, United States. The company began in 1953 as Insta-Burger King, a Jacksonville, Florida-based restaurant chain...
gave away a free Kinect bundle "every 15 minutes".
A major event was organized on November 3 in Times Square
Times Square
Times Square is a major commercial intersection in the borough of Manhattan in New York City, at the junction of Broadway and Seventh Avenue and stretching from West 42nd to West 47th Streets...
, where singer Ne-Yo
Ne-Yo
Shaffer Chimere Smith, Jr. , better known by his stage name Ne-Yo, is an American pop and R&B singer-songwriter, record producer, dancer and actor. Beginning his career as a songwriter, Ne-Yo penned the hit "Let Me Love You" for singer Mario...
performed with hundreds of dancers in anticipation of Kinect's midnight launch. During the festivities, Microsoft gave away T-shirts and Kinect games.
Software
Requiring at least 190 MB of available storage space, the Kinect system software allows users to operate the Xbox 360 Dashboard console user interface through voice commands and hand gestures. Techniques such as voice recognition and facial recognition are employed to automatically identify users. Among the applications for Kinect is Video Kinect, which enables voice chatVoice chat
Voice chat is a modern form of communication used on the Internet. The means of communicating with voice chat is through any of the messengers, mainly Skype, Yahoo! Messenger, AOL Instant Messenger, inSpeak Communicator or Windows Live Messenger...
or video chat with other Xbox 360 users or users of Windows Live Messenger
Windows Live Messenger
Windows Live Messenger is an instant messaging client created by Microsoft that is currently designed to work with Windows XP , Windows Vista, Windows 7, Windows Mobile, Windows CE, Xbox 360, Blackberry OS, iOS, Java ME, S60 on Symbian OS 9.x and Zune HD...
. The application can use Kinect's tracking functionality and the Kinect sensor's motorized pivot to keep users in frame even as they move around. Other applications with Kinect support include ESPN
ESPN
Entertainment and Sports Programming Network, commonly known as ESPN, is an American global cable television network focusing on sports-related programming including live and pre-taped event telecasts, sports talk shows, and other original programming....
, Zune Marketplace, Netflix
Netflix
Netflix, Inc., is an American provider of on-demand internet streaming media in the United States, Canada, and Latin America and flat rate DVD-by-mail in the United States. The company was established in 1997 and is headquartered in Los Gatos, California...
, Hulu Plus and Last.fm
Last.fm
Last.fm is a music website, founded in the United Kingdom in 2002. It has claimed 30 million active users in March 2009. On 30 May 2007, CBS Interactive acquired Last.fm for UK£140m ....
Games that require Kinect have a purple sticker on them with a white silhouette of the Kinect sensor and "Requires Kinect Sensor" underneath in white text. Games that have optional Kinect support (meaning that Kinect is not necessary to play the game or that there are optional Kinect minigames included) feature a standard green Xbox 360 case with a purple bar underneath the header, a silhouette of the Kinect sensor and "Better with Kinect Sensor" next to it in white text.
Kinect launched on November 4, 2010 with 17 titles. Third-party publishers of available and announced Kinect games include, among others, Ubisoft
Ubisoft
Ubisoft Entertainment S.A. is a major French video game publisher and developer, with headquarters in Montreuil, France. The company has a worldwide presence with 25 studios in 17 countries and subsidiaries in 26 countries....
, Electronic Arts
Electronic Arts
Electronic Arts, Inc. is a major American developer, marketer, publisher and distributor of video games. Founded and incorporated on May 28, 1982 by Trip Hawkins, the company was a pioneer of the early home computer games industry and was notable for promoting the designers and programmers...
, LucasArts
LucasArts
LucasArts Entertainment Company, LLC is an American video game developer and publisher. The company was once famous for its innovative line of graphic adventure games, the critical and commercial success of which peaked in the mid 1990s...
, THQ
THQ
THQ Inc. is an American developer and publisher of video games. Founded in 1989 in the United States, the company develops products for video game consoles, handheld game systems, as well as for personal computers and wireless devices...
, Activision
Activision
Activision is an American publisher, majority owned by French conglomerate Vivendi SA. Its current CEO is Robert Kotick. It was founded on October 1, 1979 and was the world's first independent developer and distributor of video games for gaming consoles...
, Konami
Konami
is a Japanese leading developer and publisher of numerous popular and strong-selling toys, trading cards, anime, tokusatsu, slot machines, arcade cabinets and video games...
, Sega
Sega
, usually styled as SEGA, is a multinational video game software developer and an arcade software and hardware development company headquartered in Ōta, Tokyo, Japan, with various offices around the world...
, Capcom
Capcom
is a Japanese developer and publisher of video games, known for creating multi-million-selling franchises such as Devil May Cry, Chaos Legion, Street Fighter, Mega Man and Resident Evil. Capcom developed and published Bionic Commando, Lost Planet and Dark Void too, but they are less known. Its...
, Namco Bandai
Namco Bandai
, also known as the Bandai Namco Group, is a Japanese holding company formed from the merger of Namco and Bandai. It has interests in toys, video games and arcades, anime, and amusement parks. The new entity was founded on September 29, 2005...
and MTV Games. Along with retail games, there are also select Xbox Live Arcade
Xbox Live Arcade
Xbox Live Arcade is a type of video game download distribution available primarily in a section of the Xbox Live Marketplace, Microsoft's digital distribution network for the Xbox 360, that focuses on smaller downloadable games from both major publishers and independent game developers...
that exclusively use the Kinect peripheral.
Kinect Fun Labs
At E3 2011, Microsoft announced Kinect Fun Labs: a collection of various gadgets and minigames that are accessible from the Xbox 360 Dashboard. These gadgets includes Build A Buddy, Air Band, Kinect Googly Eyes, Kinect Me, Bobblehead, Kinect Sparkler and Avatar Kinect.Kinect for Windows SDK
On February 21, 2011 Microsoft announced that it would release a non-commercial Kinect software development kit (SDK) for Windows in spring 2011, which was released for Windows 7 on June 16, 2011; a commercial version is planned for a later release date.The SDK includes Windows 7 compatible PC
Personal computer
A personal computer is any general-purpose computer whose size, capabilities, and original sales price make it useful for individuals, and which is intended to be operated directly by an end-user with no intervening computer operator...
drivers for Kinect device. It provides Kinect capabilities to developers to build applications with C++
C++
C++ is a statically typed, free-form, multi-paradigm, compiled, general-purpose programming language. It is regarded as an intermediate-level language, as it comprises a combination of both high-level and low-level language features. It was developed by Bjarne Stroustrup starting in 1979 at Bell...
, C#, or Visual Basic
Visual Basic
Visual Basic is the third-generation event-driven programming language and integrated development environment from Microsoft for its COM programming model...
by using Microsoft Visual Studio 2010 and includes following features:
- Raw sensor streams: Access to low-level streams from the depth sensor, color camera sensor, and four-element microphone array.
- Skeletal tracking: The capability to track the skeleton image of one or two people moving within the Kinect field of view for gesture-driven applications.
- Advanced audio capabilities: Audio processing capabilities include sophisticated acoustic noise suppression and echo cancellationEcho cancellation'The term echo cancellation is used in telephony to describe the process of removing echo from a voice communication in order to improve voice quality on a telephone call...
, beam formation to identify the current sound source, and integration with the Windows speech recognitionWindows Speech RecognitionWindows Speech Recognition is a speech recognition application included in Windows Vista and more recently, Windows 7.-Features:Windows Speech Recognition allows the user to control the computer by giving specific voice commands...
API. - Sample code and Documentation.
Open source drivers
In November 2010, Adafruit Industries offered a bounty for an open-source driver for Kinect. Microsoft initially voiced its disapproval of the bounty, stating that it "does not condone the modification of its products" and that it had "built in numerous hardware and software safeguards designed to reduce the chances of product tampering". This reaction, however, was caused by a misunderstanding within Microsoft, and the company later clarified its position, claiming that while it does not condone hacking of either the physical device or the console, the USB connection was left open by design.On November 10, Adafruit announced Héctor Martín
Héctor Martín Cantero
Héctor Martín Cantero , alias marcan is known for working with other people to hack the PlayStation 3's security system , work on Wii homebrew, work on iPhone support for Linux and publishing the first open source driver for the Kinect.- References :...
as the winner, who had produced a Linux
Linux
Linux is a Unix-like computer operating system assembled under the model of free and open source software development and distribution. The defining component of any Linux system is the Linux kernel, an operating system kernel first released October 5, 1991 by Linus Torvalds...
driver that allows the use of both the RGB camera and depth sensitivity functions of the device. It was later revealed that Johnny Lee
Johnny Lee (computer scientist)
Johnny Chung Lee is a Human-Computer Interaction researcher currently working at Google. Lee is best known for his work on Kinect development, extending the functionality of the Wii Remote controller of the Wii video game console, most notably by taking advantage of its high resolution IR...
, a core member of Microsoft's Kinect development team, had secretly approached Adafruit with the idea of a driver development contest and had personally financed it.
In December 2010, PrimeSense
PrimeSense
PrimeSense is a Israeli company founded in 2005 by Aviad Maizels, Alexander Shpunt, Ophir Sharon, Tamir Berliner and Dima Rais. The company was selected by MIT technological review as one of world's 50 most innovative companies for 2011....
, whose depth sensing reference design Kinect is based on, released their own open source drivers along with motion tracking middleware called NITE. PrimeSense later announced that it had teamed up with Asus
ASUS
ASUSTeK Computer Incorporated is a multinational computer technology and consumer electronics product manufacturer headquartered in Taipei, Taiwan. Its product range includes motherboards, desktops, laptops, monitors, tablet PCs, servers and mobile phones...
to develop a PC-compatible device similar to Kinect, which will be called WAVI Xtion and is scheduled for release in the second quarter of 2011.
Third party development
Numerous developers are researching possible applications of Kinect that go beyond the system's intended purpose of playing games. For example, Philipp Robbel of MITMassachusetts Institute of Technology
The Massachusetts Institute of Technology is a private research university located in Cambridge, Massachusetts. MIT has five schools and one college, containing a total of 32 academic departments, with a strong emphasis on scientific and technological education and research.Founded in 1861 in...
combined Kinect with the iRobot Create
IRobot Create
iRobot Create is a hobbyist robot manufactured by iRobot that is based on the Roomba platform and was introduced in 2007. However, iRobot Create is explicitly designed for robotics development, rather than simply hacking the Roomba...
to map a room in 3D and have the robot respond to human gestures, while an MIT Media Lab team is working on a JavaScript extension for Google Chrome
Google Chrome
Google Chrome is a web browser developed by Google that uses the WebKit layout engine. It was first released as a beta version for Microsoft Windows on September 2, 2008, and the public stable release was on December 11, 2008. The name is derived from the graphical user interface frame, or...
called depthJS that allows users to control the browser with hand gestures. Other programmers, including the Robot Locomotion Group at MIT, are using the drivers to develop a motion-controller user interface similar to the one envisioned in the film Minority Report
Minority Report (film)
Minority Report is a 2002 American neo-noir science fiction film directed by Steven Spielberg and loosely based on the short story "The Minority Report" by Philip K. Dick. It is set primarily in Washington, D.C...
. The developers of MRPT
Mobile Robot Programming Toolkit
The Mobile Robot Programming Toolkit is a cross-platform and open source C++ library aimed to help robotics researchers to design and implement algorithms related to Simultaneous Localization and Mapping , computer vision and motion planning...
have integrated open source drivers into their libraries and provided examples of live 3D rendering and basic 3D visual SLAM
Simultaneous localization and mapping
Simultaneous localization and mapping is a technique used by robots and autonomous vehicles to build up a map within an unknown environment , or to update a map within a known environment , while at the same time keeping track of their current location.- Operational definition :Maps are used...
. Another team has shown an application that allows Kinect users to play a virtual piano by tapping their fingers on an empty desk. Oliver Kreylos, a researcher at University of California, Davis
University of California, Davis
The University of California, Davis is a public teaching and research university established in 1905 and located in Davis, California, USA. Spanning over , the campus is the largest within the University of California system and third largest by enrollment...
, adopted the technology to improve live 3-dimensional videoconferencing, which NASA
NASA
The National Aeronautics and Space Administration is the agency of the United States government that is responsible for the nation's civilian space program and for aeronautics and aerospace research...
has shown interest in.
Alexandre Alahi from EPFL
École polytechnique fédérale de Lausanne
The École polytechnique fédérale de Lausanne is one of the two Swiss Federal Institutes of Technology and is located in Lausanne, Switzerland.The school was founded by the Swiss Federal Government with the stated mission to:...
presented a video surveillance system that combines multiple Kinect devices to track groups of people even in complete darkness. Companies So touch and Evoluce have developed presentation software for Kinect that can be controlled by hand gestures; among its features is a multi-touch zoom mode. In December 2010, the free public beta of HTPC
Home theater PC
A Home Theater PC or Media Center appliance is a convergence device that combines some or all the capabilities of a personal computer with a software application that supports video, photo, music playback, and sometimes video recording functionality...
software KinEmote was launched; it allows navigation of Boxee
BOXEE
Boxee is a cross-platform freeware HTPC software application with a 10-foot user interface and social networking features designed for the living-room TV...
and XBMC menus using a Kinect sensor. Soroush Falahati wrote an application that can be used to create stereoscopic
Stereoscopy
Stereoscopy refers to a technique for creating or enhancing the illusion of depth in an image by presenting two offset images separately to the left and right eye of the viewer. Both of these 2-D offset images are then combined in the brain to give the perception of 3-D depth...
3D images with a Kinect sensor.
For a limited time in May 2011, a Topshop
Topshop
Topshop is a British clothes retailer with shops in over 20 countries and online operations in a number of its markets. Its sales come primarily from women's clothing and fashion accessories...
store in Moscow set up a Kinect kiosk that could overlay a collection of dresses onto the live video feed of customers. Through automatic tracking, position and rotation of the virtual dress were updated even as customers turned around to see the back of the outfit.
Kinect also shows compelling potential for use in medicine. Researchers at the University of Minnesota
University of Minnesota
The University of Minnesota, Twin Cities is a public research university located in Minneapolis and St. Paul, Minnesota, United States. It is the oldest and largest part of the University of Minnesota system and has the fourth-largest main campus student body in the United States, with 52,557...
have used Kinect to measure a range of disorder symptoms in children, creating new ways of objective evaluation to detect such conditions as autism, attention-deficit disorder and obsessive-compulsive disorder. At the Institute of Forensic Medicine Virtopsy Project at the University of Bern in Switzerland
Switzerland
Switzerland name of one of the Swiss cantons. ; ; ; or ), in its full name the Swiss Confederation , is a federal republic consisting of 26 cantons, with Bern as the seat of the federal authorities. The country is situated in Western Europe,Or Central Europe depending on the definition....
, researchers have devised a way for surgeons to manipulate imaging techniques (such as MRI) to guide surgery, using a Kinect to capture their hand motions to direct the imaging, freeing their hands from having to use computer keyboards, thus also reducing the chance of contamination. This technique is already working at Sunnybrook Health Sciences Centre
Sunnybrook Health Sciences Centre
Sunnybrook Health Sciences Centre, abbreviated SHSC and known simply as Sunnybrook, is an academic health sciences centre located in Toronto, Ontario....
in Toronto
Toronto
Toronto is the provincial capital of Ontario and the largest city in Canada. It is located in Southern Ontario on the northwestern shore of Lake Ontario. A relatively modern city, Toronto's history dates back to the late-18th century, when its land was first purchased by the British monarchy from...
, where doctors use it to guide imaging during cancer surgery.
Reception
IGNIGN
IGN is an entertainment website that focuses on video games, films, music and other media. IGN's main website comprises several specialty sites or "channels", each occupying a subdomain and covering a specific area of entertainment...
gave the device 7.5 out of 10, saying that "Kinect can be a tremendous amount of fun for casual players, and the creative, controller-free concept is undeniably appealing", though adding that for "$149.99, a motion-tracking camera add-on for the Xbox 360 is a tough sell, especially considering that the entry level variation of the Xbox 360 itself is only $199.99". Game Informer
Game Informer
Game Informer is an American-based monthly magazine featuring articles, news, strategy, and reviews of popular video games and associated consoles. It was formed in August 1991, when FuncoLand started publishing a six-page magazine, free in all its retail locations...
rated Kinect 8 out of 10, praising the technology but noting that the experience takes a while to get used to and that the spatial requirement may pose a barrier. Computer and Video Games
Computer and video games
A video game is an electronic game that involves human interaction with a user interface to generate visual feedback on a video device. The word video in video game traditionally referred to a raster display device, but following popularization of the term "video game", it now implies any type of...
called the device a technological gem and applauded the gesture and voice controls, while criticizing the launch lineup and Kinect Hub.
CNET
CNET.com
CNET is a tech media website that publishes news articles, blogs, and podcasts on technology and consumer electronics. Originally founded in 1994 by Halsey Minor and Shelby Bonnie, it was the flagship brand of CNET Networks and became a brand of CBS Interactive through CNET Networks' acquisition...
's review pointed out how Kinect keeps players active with its full-body motion sensing but criticized the learning curve, the additional power supply needed for older Xbox 360 consoles and the space requirements. Engadget
Engadget
Engadget is a multilingual technology blog network with daily coverage of gadgets and consumer electronics. Though on appearance Engadget functions much like a blog and may be defined as such, much of its editorial content takes the form of an online magazine...
, too, listed the large space requirements as a negative, along with Kinect's launch lineup and the slowness of the hand gesture UI. The review praised the system's powerful technology and the potential of its yoga and dance games. Kotaku
Kotaku
Kotaku is a video games-focused blog. It is part of Gawker Media's "Gawker" network of sites, which also includes Gizmodo, Deadspin, Lifehacker, io9 and Jezebel. Named to CNET News' Blog 100, Kotaku is consistently listed in the top 40 of Technorati's Top 100...
considered the device revolutionary upon first use but noted that games were sometimes unable to recognize gestures or had slow responses, concluding that Kinect is "not must-own yet, more like must-eventually own." TechRadar
TechRadar
TechRadar is an online publication focused on technology company which looks and reviews the most available tech today. TechRadar is owned by the Future plc company, the sixth-largest in the United Kingdom. It has a great number of reviews. Such as: Tablet Computer, Smartphone, PC, Macintosh,...
praised the voice control and saw a great deal of potential in the device whose lag and space requirements were identified as issues. Gizmodo
Gizmodo
Gizmodo is a technology weblog about consumer electronics. It is part of the Gawker Media network run by Nick Denton and is known for its up-to-date coverage of the technology industry, along with topics as broad as design; architecture; space and science....
also noted Kinect's potential and expressed curiosity in how more mainstream titles would utilize the technology. Ars Technica
Ars Technica
Ars Technica is a technology news and information website created by Ken Fisher and Jon Stokes in 1998. It publishes news, reviews and guides on issues such as computer hardware and software, science, technology policy, and video games. Ars Technica is known for its features, long articles that go...
's review expressed concern that the core feature of Kinect, its lack of a controller, would hamper development of games beyond those that have either stationary players or control the player's movement automatically.
The mainstream press also reviewed Kinect. USA Today
USA Today
USA Today is a national American daily newspaper published by the Gannett Company. It was founded by Al Neuharth. The newspaper vies with The Wall Street Journal for the position of having the widest circulation of any newspaper in the United States, something it previously held since 2003...
compared it to the futuristic control scheme seen in Minority Report
Minority Report (film)
Minority Report is a 2002 American neo-noir science fiction film directed by Steven Spielberg and loosely based on the short story "The Minority Report" by Philip K. Dick. It is set primarily in Washington, D.C...
, stating that "playing games feels great" and giving the device 3.5 out of 4 stars. David Pogue from The New York Times
The New York Times
The New York Times is an American daily newspaper founded and continuously published in New York City since 1851. The New York Times has won 106 Pulitzer Prizes, the most of any news organization...
predicted you will feel a "crazy, magical, omigosh rush the first time you try the Kinect." Despite calling the motion tracking less precise than Wii
Wii
The Wii is a home video game console released by Nintendo on November 19, 2006. As a seventh-generation console, the Wii primarily competes with Microsoft's Xbox 360 and Sony's PlayStation 3. Nintendo states that its console targets a broader demographic than that of the two others...
's implementation, Pogue concluded that "Kinect’s astonishing technology creates a completely new activity that’s social, age-spanning and even athletic." The Globe and Mail
The Globe and Mail
The Globe and Mail is a nationally distributed Canadian newspaper, based in Toronto and printed in six cities across the country. With a weekly readership of approximately 1 million, it is Canada's largest-circulation national newspaper and second-largest daily newspaper after the Toronto Star...
titled Kinect as setting a "new standard for motion control." The slight input lag between making a physical movement and Kinect registering it was not considered a major issue with most games, and the review called Kinect "a good and innovative product," rating it 3.5 out of 4 stars.
Sales
10 million units of the Kinect sensor had been shipped by March 9, 2011. Having sold 8 million units in its first 60 days on the market, Kinect has claimed the Guinness World Record of being the "fastest selling consumer electronics device". According to Wedbush analyst Michael Pachter, Kinect bundles accounted for about half of all Xbox 360 console sales in December 2010 and for more than two-thirds in February 2011. More than 750,000 Kinect units were sold during the week of Black Friday 2011.Awards
- The machine learningMachine learningMachine learning, a branch of artificial intelligence, is a scientific discipline concerned with the design and development of algorithms that allow computers to evolve behaviors based on empirical data, such as from sensor data or databases...
work on human motion capture within Kinect won the 2011 MacRobert AwardMacRobert AwardThe MacRobert Award has been presented every year since 1969 by the Royal Academy of Engineering.The award seeks to recognise innovative ideas in engineering.-History:The award is named in honour of Lady Rachel Workman MacRobert .-Winners:...
for engineering innovation. - Kinect Won T3's "Gadget of the Year" award for 2011. It also won the "Gaming Gadget of the Year" prize.
- 'Microsoft Kinect for Windows Software Development Kit' was ranked second in "The 10 Most Innovative Tech Products of 2011" at Popular MechanicsPopular MechanicsPopular Mechanics is an American magazine first published January 11, 1902 by H. H. Windsor, and has been owned since 1958 by the Hearst Corporation...
Breakthrough Awards ceremony in New York City.
See also
- DreameyeDreameyeThe Dreameye is a digital camera released for the Dreamcast home video console by Sega. It was designed to be used as a webcam and a digital still camera, and there were plans for games to involve the Dreameye. The Dreameye was only released in Japan, and Dreameye functionality was absent in...
- EyeToyEyeToyThe EyeToy is a color digital camera device, similar to a webcam, for the PlayStation 2. The technology uses computer vision and Gesture recognition to process images taken by the camera...
- List of Kinect games
- Motion captureMotion captureMotion capture, motion tracking, or mocap are terms used to describe the process of recording movement and translating that movement on to a digital model. It is used in military, entertainment, sports, and medical applications, and for validation of computer vision and robotics...
- PlayStation EyePlayStation EyeThe PlayStation Eye is a digital camera device, similar to a webcam, for the PlayStation 3. The technology uses computer vision and gesture recognition to process images taken by the camera. This allows players to interact with games using motion and color detection as well as sound through its...
- PlayStation Move
- Sega Activator
- Structured-light 3D scanner
- Wii Balance BoardWii Balance BoardThe Wii Balance Board is a balance board accessory for the Nintendo Wii video game console. Along with Wii Fit, it was introduced on July 11, 2007 at the Electronic Entertainment Expo...
- Wii RemoteWii RemoteThe , also known as the Wiimote, is the primary controller for Nintendo's Wii console. A main feature of the Wii Remote is its motion sensing capability, which allows the user to interact with and manipulate items on screen via gesture recognition and pointing through the use of accelerometer and...
- Xbox Live Vision
External links
- Kinect Playlist at the official YouTube Xbox channel
- Kinect on Microsoft Wave
- KinectShare KinectShare.com
- iFixit teardown of Kinect
- Patent application relating to Kinect
- Kinect for Windows SDK from Microsoft Research