Articulated body pose estimation
Encyclopedia
Articulated body pose estimation, in computer vision
, is the study of algorithms and systems that recover the pose of an articulated body, which consists of joints and rigid parts using image-based observations. It is one of longest-lasting problems in computer vision because of the complexity of the models that relate observation with pose, and because of the variety of situations in which it would be useful.
-less, vision-based articulated body pose estimation systems to recover the pose of bodies such as the human body, a hand, or non-human creatures. Such a system have several foreseeable applications, including
One of the major difficulties in recovering pose from images is the high number of degrees-of-freedom (DOF) in the body's movement that has to be recovered. Any rigid object requires six DOF to fully describe its pose, and each additional rigid object connected to it adds at least one DOF. A human body contains no less than 10 large body parts, equating to more than 20 DOF. This difficulty is compounded by the problem of self-occlusion, where body parts occlude each other depending on the configuration of the parts. Other challenges involve dealing with varying lighting, which affect appearance; varying subject attire or body type; required camera configuration; and required computation time.
The typical articulated body pose estimation system involves a model-based approach, in which an observation is made and provided as input to the model to generate pose estimates. Different kinds of sensors have been explored for use in making the observation, including
These sensors produce intermediate representations that is directly used by the model; the representations include
technique is optical motion capture
. This approach involves placing markers on the individual at strategic locations to capture the 6 degrees-of-freedom of each body part.
Computer vision
Computer vision is a field that includes methods for acquiring, processing, analysing, and understanding images and, in general, high-dimensional data from the real world in order to produce numerical or symbolic information, e.g., in the forms of decisions...
, is the study of algorithms and systems that recover the pose of an articulated body, which consists of joints and rigid parts using image-based observations. It is one of longest-lasting problems in computer vision because of the complexity of the models that relate observation with pose, and because of the variety of situations in which it would be useful.
Description
There is a need to develop accurate tetherTether
A tether is a cord, fixture, or signal that anchors something movable to a reference point which may be fixed or moving. There are a number of applications for tethers: balloons, kites, tethered wind-energy conversion systems, anchors, tethered water-flow energy conversion systems, towing, animal...
-less, vision-based articulated body pose estimation systems to recover the pose of bodies such as the human body, a hand, or non-human creatures. Such a system have several foreseeable applications, including
- Marker-less motion captureMotion captureMotion capture, motion tracking, or mocap are terms used to describe the process of recording movement and translating that movement on to a digital model. It is used in military, entertainment, sports, and medical applications, and for validation of computer vision and robotics...
for human-computer interfaces, - Physiotherapy,
- 3D animation,
- ErgonomicsErgonomicsErgonomics is the study of designing equipment and devices that fit the human body, its movements, and its cognitive abilities.The International Ergonomics Association defines ergonomics as follows:...
studies, - RobotRobotA robot is a mechanical or virtual intelligent agent that can perform tasks automatically or with guidance, typically by remote control. In practice a robot is usually an electro-mechanical machine that is guided by computer and electronic programming. Robots can be autonomous, semi-autonomous or...
control, and - Visual surveillance.
One of the major difficulties in recovering pose from images is the high number of degrees-of-freedom (DOF) in the body's movement that has to be recovered. Any rigid object requires six DOF to fully describe its pose, and each additional rigid object connected to it adds at least one DOF. A human body contains no less than 10 large body parts, equating to more than 20 DOF. This difficulty is compounded by the problem of self-occlusion, where body parts occlude each other depending on the configuration of the parts. Other challenges involve dealing with varying lighting, which affect appearance; varying subject attire or body type; required camera configuration; and required computation time.
The typical articulated body pose estimation system involves a model-based approach, in which an observation is made and provided as input to the model to generate pose estimates. Different kinds of sensors have been explored for use in making the observation, including
- Visible wavelength imagery,
- Long-wave thermal infraredInfraredInfrared light is electromagnetic radiation with a wavelength longer than that of visible light, measured from the nominal edge of visible red light at 0.74 micrometres , and extending conventionally to 300 µm...
imagery, - Time-of-flight imagery, and
- Laser range scanner imagery.
These sensors produce intermediate representations that is directly used by the model; the representations include
- Image appearance,
- Voxel (volume element) reconstruction,
- 3D surface point clouds, and
- 3D surface meshes.
Related technology
A commercially successful but specialized computer vision-based articulated body pose estimation3D Pose Estimation
3D pose estimation is the problem of determining the transformation of an object in a 2D image which gives the 3D object. The need for 3D pose estimation arises from the limitations of feature based pose estimation. There exist environments where it is difficult to extract corners or edges from...
technique is optical motion capture
Motion capture
Motion capture, motion tracking, or mocap are terms used to describe the process of recording movement and translating that movement on to a digital model. It is used in military, entertainment, sports, and medical applications, and for validation of computer vision and robotics...
. This approach involves placing markers on the individual at strategic locations to capture the 6 degrees-of-freedom of each body part.
Active Research Groups
A number of groups are actively pursuing this topic, including groups in Brown University; Carnegie Mellon University; MPI Saarbruecken; Stanford University; the University of California, San Diego; the Univeresity of Toronto; and the Ecole Centrale de Paris.External links
- Michael J. Black, Professor at Brown University
- Research Project Page of German Cheung at Carnegie Mellon University
- Homepage of Dr.-Ing at MPI Saarbruecken
- Markerless Motion Capture Project at Stanford
- Computer Vision and Robotics Research Laboratory at the University of California, San Diego
- Research Projects of David J. Fleet at the Univeresity of Toronto
- Ronald Poppe at the University of Twente.
- Professor Nikos Paragios at the Ecole Centrale de Paris