H.264/MPEG-4 AVC

H.264/MPEG-4 AVC

Discussion
Ask a question about 'H.264/MPEG-4 AVC'
Start a new discussion about 'H.264/MPEG-4 AVC'
Answer questions from other users
Full Discussion Forum
 
Encyclopedia
H.264/MPEG-4 Part 10 or AVC (Advanced Video Coding) is a standard for video compression, and is currently one of the most commonly used formats for the recording, compression, and distribution of high definition video. The final drafting work on the first version of the standard was completed in May 2003.

H.264/MPEG-4 AVC is a block-oriented motion-compensation
Motion compensation
Motion compensation is an algorithmic technique employed in the encoding of video data for video compression, for example in the generation of MPEG-2 files. Motion compensation describes a picture in terms of the transformation of a reference picture to the current picture. The reference picture...

-based codec standard developed by the ITU-T
ITU-T
The ITU Telecommunication Standardization Sector is one of the three sectors of the International Telecommunication Union ; it coordinates standards for telecommunications....

 Video Coding Experts Group (VCEG) together with the International Organization for Standardization
International Organization for Standardization
The International Organization for Standardization , widely known as ISO, is an international standard-setting body composed of representatives from various national standards organizations. Founded on February 23, 1947, the organization promulgates worldwide proprietary, industrial and commercial...

 (ISO)/International Electrotechnical Commission
International Electrotechnical Commission
The International Electrotechnical Commission is a non-profit, non-governmental international standards organization that prepares and publishes International Standards for all electrical, electronic and related technologies – collectively known as "electrotechnology"...

 (IEC) Moving Picture Experts Group
Moving Picture Experts Group
The Moving Picture Experts Group is a working group of experts that was formed by ISO and IEC to set standards for audio and video compression and transmission. It was established in 1988 by the initiative of Hiroshi Yasuda and Leonardo Chiariglione, who has been from the beginning the Chairman...

 (MPEG). It was the product of a partnership effort known as the Joint Video Team (JVT). The ITU-T H.264 standard and the ISO/IEC MPEG-4
MPEG-4
MPEG-4 is a method of defining compression of audio and visual digital data. It was introduced in late 1998 and designated a standard for a group of audio and video coding formats and related technology agreed upon by the ISO/IEC Moving Picture Experts Group under the formal standard ISO/IEC...

 AVC
standard (formally, ISO/IEC 14496-10 – MPEG-4
MPEG-4
MPEG-4 is a method of defining compression of audio and visual digital data. It was introduced in late 1998 and designated a standard for a group of audio and video coding formats and related technology agreed upon by the ISO/IEC Moving Picture Experts Group under the formal standard ISO/IEC...

 Part 10, Advanced Video Coding) are jointly maintained so that they have identical technical content.

H.264 is perhaps best known as being one of the codec
Codec
A codec is a device or computer program capable of encoding or decoding a digital data stream or signal. The word codec is a portmanteau of "compressor-decompressor" or, more commonly, "coder-decoder"...

 standards for Blu-ray Disc
Blu-ray Disc
Blu-ray Disc is an optical disc storage medium designed to supersede the DVD format. The plastic disc is 120 mm in diameter and 1.2 mm thick, the same size as DVDs and CDs. Blu-ray Discs contain 25 GB per layer, with dual layer discs being the norm for feature-length video discs...

s; all Blu-ray Disc players must be able to decode H.264. It is also widely used by streaming internet sources, such as videos from Vimeo
Vimeo
Vimeo is a video-sharing website on which users can upload, share, and view videos. It was founded by Zach Klein and Jake Lodwick in November 2004...

, YouTube
YouTube
YouTube is a video-sharing website, created by three former PayPal employees in February 2005, on which users can upload, view and share videos....

, and the iTunes Store
ITunes Store
The iTunes Store is a software-based online digital media store operated by Apple. Opening as the iTunes Music Store on April 28, 2003, with over 200,000 items to purchase, it is, as of April 2008, the number-one music vendor in the United States...

, web software such as the Adobe Flash Player
Adobe Flash Player
The Adobe Flash Player is software for viewing multimedia, Rich Internet Applications and streaming video and audio, on a computer web browser or on supported mobile devices. Flash Player runs SWF files that can be created by the Adobe Flash authoring tool, by Adobe Flex or by a number of other...

 and Microsoft Silverlight
Microsoft Silverlight
Microsoft Silverlight is an application framework for writing and running rich Internet applications, with features and purposes similar to those of Adobe Flash. The run-time environment for Silverlight is available as a plug-in for web browsers running under Microsoft Windows and Mac OS X...

, broadcast services for DVB and SBTVD
SBTVD
ISDB-T International or SBTVD, short for Sistema Brasileiro de Televisão Digital is a technical standard for digital television broadcast used in Brazil, Peru, Argentina, Chile, Venezuela, Ecuador, Costa Rica, Paraguay, Philippines, Bolivia, Nicaragua and Uruguay, based on the Japanese ISDB-T...

, direct-broadcast satellite television services, cable television services, and real-time videoconferencing.

Overview


The intent of the H.264/AVC project was to create a standard capable of providing good video quality at substantially lower bit rates than previous standards (i.e., half or less the bit rate of MPEG-2
MPEG-2
MPEG-2 is a standard for "the generic coding of moving pictures and associated audio information". It describes a combination of lossy video compression and lossy audio data compression methods which permit storage and transmission of movies using currently available storage media and transmission...

, H.263
H.263
H.263 is a video compression standard originally designed as a low-bitrate compressed format for videoconferencing. It was developed by the ITU-T Video Coding Experts Group in a project ending in 1995/1996 as one member of the H.26x family of video coding standards in the domain of the ITU-T.H.263...

, or MPEG-4 Part 2
MPEG-4 Part 2
MPEG-4 Part 2, MPEG-4 Visual is a video compression technology developed by MPEG. It belongs to the MPEG-4 ISO/IEC standards. It is a discrete cosine transform compression standard, similar to previous standards such as MPEG-1 and MPEG-2...

), without increasing the complexity of design so much that it would be impractical or excessively expensive to implement. An additional goal was to provide enough flexibility to allow the standard to be applied to a wide variety of applications on a wide variety of networks and systems, including low and high bit rates, low and high resolution video, broadcast
Broadcasting
Broadcasting is the distribution of audio and video content to a dispersed audience via any audio visual medium. Receiving parties may include the general public or a relatively large subset of thereof...

, DVD
DVD
A DVD is an optical disc storage media format, invented and developed by Philips, Sony, Toshiba, and Panasonic in 1995. DVDs offer higher storage capacity than Compact Discs while having the same dimensions....

 storage, RTP
Real-time Transport Protocol
The Real-time Transport Protocol defines a standardized packet format for delivering audio and video over IP networks. RTP is used extensively in communication and entertainment systems that involve streaming media, such as telephony, video teleconference applications, television services and...

/IP
Internet Protocol
The Internet Protocol is the principal communications protocol used for relaying datagrams across an internetwork using the Internet Protocol Suite...

 packet networks, and ITU-T
ITU-T
The ITU Telecommunication Standardization Sector is one of the three sectors of the International Telecommunication Union ; it coordinates standards for telecommunications....

 multimedia telephony
Telephony
In telecommunications, telephony encompasses the general use of equipment to provide communication over distances, specifically by connecting telephones to each other....

 systems.

The H.264 standard can be viewed as a "family of standards", the members of which are the profiles described below. A specific decoder decodes at least one, but not necessarily all profiles. The decoder specification describes which of the profiles can be decoded.

The H.264 name follows the ITU-T
ITU-T
The ITU Telecommunication Standardization Sector is one of the three sectors of the International Telecommunication Union ; it coordinates standards for telecommunications....

 naming convention, where the standard is a member of the H.26x line of VCEG video coding standards; the MPEG-4 AVC name relates to the naming convention in ISO
International Organization for Standardization
The International Organization for Standardization , widely known as ISO, is an international standard-setting body composed of representatives from various national standards organizations. Founded on February 23, 1947, the organization promulgates worldwide proprietary, industrial and commercial...

/IEC
International Electrotechnical Commission
The International Electrotechnical Commission is a non-profit, non-governmental international standards organization that prepares and publishes International Standards for all electrical, electronic and related technologies – collectively known as "electrotechnology"...

 MPEG, where the standard is part 10 of ISO/IEC 14496, which is the suite of standards known as MPEG-4. The standard was developed jointly in a partnership of VCEG and MPEG, after earlier development work in the ITU-T as a VCEG project called H.26L. It is thus common to refer to the standard with names such as H.264/AVC, AVC/H.264, H.264/MPEG-4 AVC, or MPEG-4/H.264 AVC, to emphasize the common heritage. Occasionally, it is also referred to as "the JVT codec", in reference to the Joint Video Team (JVT) organization that developed it. (Such partnership and multiple naming is not uncommon. For example, the video codec standard known as MPEG-2
MPEG-2
MPEG-2 is a standard for "the generic coding of moving pictures and associated audio information". It describes a combination of lossy video compression and lossy audio data compression methods which permit storage and transmission of movies using currently available storage media and transmission...

 also arose from the partnership between MPEG and the ITU-T
ITU-T
The ITU Telecommunication Standardization Sector is one of the three sectors of the International Telecommunication Union ; it coordinates standards for telecommunications....

, where MPEG-2 video is known to the ITU-T community as H.262.) Some software programs (such as VLC media player
VLC media player
VLC media player is a free and open source media player and multimedia framework written by the VideoLAN project.VLC is a portable multimedia player, encoder, and streamer supporting many audio and video codecs and file formats as well as DVDs, VCDs, and various streaming protocols. It is able to...

) internally identify this standard as AVC1.

The standardization of the first version of H.264/AVC was completed in May 2003. In the first project to extend the original standard, the JVT then developed what was called the Fidelity Range Extensions (FRExt). These extensions enabled higher quality video coding by supporting increased sample bit depth precision and higher-resolution color information, including sampling structures known as Y'CbCr 4:2:2 (=YUV 4:2:2) and Y'CbCr 4:4:4. Several other features were also included in the Fidelity Range Extensions project, such as adaptive switching between 4×4 and 8×8 integer transforms, encoder-specified perceptual-based quantization weighting matrices, efficient inter-picture lossless coding, and support of additional color spaces. The design work on the Fidelity Range Extensions was completed in July 2004, and the drafting work on them was completed in September 2004.

Further recent extensions of the standard then included adding five other new profiles intended primarily for professional applications, adding extended-gamut color space support, defining additional aspect ratio indicators, defining two additional types of "supplemental enhancement information" (post-filter hint and tone mapping), and deprecating one of the prior FRExt profiles that industry feedback indicated should have been designed differently.

The next major feature added to the standard was Scalable Video Coding
Scalable Video Coding
Scalable Video Coding is the name for the Annex G extension of the H.264/MPEG-4 AVC video compression standard. SVC standardizes the encoding of a high-quality video bitstream that also contains one or more subset bitstreams. A subset video bitstream is derived by dropping packets from the...

 (SVC). Specified in Annex G of H.264/AVC, SVC allows the construction of bitstreams that contain sub-bitstreams that also conform to the standard, including one such bitstream known as the "base layer" that can be decoded by an H.264/AVC that does not support SVC. For temporal bitstream scalability, i.e., the presence of a sub-bitstream with a smaller temporal sampling rate than the bitstream, complete access units are removed from the bitstream when deriving the sub-bitstream. In this case, high-level syntax and inter prediction reference pictures in the bitstream are constructed accordingly. For spatial and quality bitstream scalability, i.e. the presence of a sub-bitstream with lower spatial resolution or quality than the bitstream, NAL (Network Abstraction Layer
Network Abstraction Layer
The Network Abstraction Layer is a part of the H.264/AVC Video Coding Standard. The main goal of the H.264/AVC NAL is the provision of a "network-friendly" video representation addressing "conversational" and "non conversational" applications...

) removed from the bitstream when deriving the sub-bitstream. In this case, inter-layer prediction, i.e., the prediction of the higher spatial resolution or quality signal by data of the lower spatial resolution or quality signal, is typically used for efficient coding. The Scalable Video Coding
Scalable Video Coding
Scalable Video Coding is the name for the Annex G extension of the H.264/MPEG-4 AVC video compression standard. SVC standardizes the encoding of a high-quality video bitstream that also contains one or more subset bitstreams. A subset video bitstream is derived by dropping packets from the...

 extensions were completed in November 2007.

The next major feature added to the standard was Multiview Video Coding
Multiview Video Coding
Multiview Video Coding is an amendment to H.264/MPEG-4 AVC video compression standard developed with joint efforts by MPEG/VCEG that enables efficient encoding of sequences captured simultaneously from multiple cameras using a single video stream....

 (MVC). Specified in Annex H of H.264/AVC, MVC enables the construction of bitstreams that represent more than one view of a video scene. An important example of this functionality is stereoscopic 3D
Stereoscopy
Stereoscopy refers to a technique for creating or enhancing the illusion of depth in an image by presenting two offset images separately to the left and right eye of the viewer. Both of these 2-D offset images are then combined in the brain to give the perception of 3-D depth...

 video coding. Two profiles were developed in the MVC work: Multiview High Profile supports an arbitrary number of views, and Stereo High Profile is designed specifically for two-view stereoscopic video. The Multiview Video Coding extensions were completed in November 2009.

Standardization committee and history


In early 1998, the Video Coding Experts Group (VCEG – ITU-T SG16 Q.6) issued a call for proposals on a project called H.26L, with the target to double the coding efficiency (which means halving the bit rate necessary for a given level of fidelity) in comparison to any other existing video coding standards for a broad variety of applications. VCEG
VCEG
The Video Coding Experts Group or Visual Coding Experts Group is the informal name of Question 6 of Working Party 3 of Study Group 16 of the ITU-T. Its abbreviated title is ITU-T Q.6/SG 16...

 was chaired by Gary Sullivan
Gary Sullivan (engineer)
Gary J. Sullivan is an American electrical engineer who led the development of the H.264/AVC video coding standard and created the DirectX Video Acceleration API/DDI video decoding feature of the Microsoft Windows operating system platform...

 (Microsoft
Microsoft
Microsoft Corporation is an American public multinational corporation headquartered in Redmond, Washington, USA that develops, manufactures, licenses, and supports a wide range of products and services predominantly related to computing through its various product divisions...

, formerly PictureTel, USA). The first draft design for that new standard was adopted in August 1999. In 2000, Thomas Wiegand
Thomas Wiegand
Thomas Wiegand is a German electrical engineer who actively participated in the creation of the H.264/AVC video coding standard. He was one of the chairmen of the Joint Video Team standardization committee that created the standard and was the editor of the standard itself...

 (Heinrich Hertz Institute, Germany) became VCEG co-chair.
In December 2001, VCEG and the Moving Picture Experts Group (MPEG – ISO/IEC JTC 1/SC 29/WG 11) formed a Joint Video Team (JVT), with the charter to finalize the video coding standard. Formal approval of the specification came in March 2003. The JVT was (is) chaired by Gary Sullivan
Gary Sullivan (engineer)
Gary J. Sullivan is an American electrical engineer who led the development of the H.264/AVC video coding standard and created the DirectX Video Acceleration API/DDI video decoding feature of the Microsoft Windows operating system platform...

, Thomas Wiegand
Thomas Wiegand
Thomas Wiegand is a German electrical engineer who actively participated in the creation of the H.264/AVC video coding standard. He was one of the chairmen of the Joint Video Team standardization committee that created the standard and was the editor of the standard itself...

, and Ajay Luthra (Motorola
Motorola
Motorola, Inc. was an American multinational telecommunications company based in Schaumburg, Illinois, which was eventually divided into two independent public companies, Motorola Mobility and Motorola Solutions on January 4, 2011, after losing $4.3 billion from 2007 to 2009...

, USA). In June 2004, the Fidelity range extensions (FRExt) project was finalized. From January 2005 to November 2007, the JVT was working on an extension of H.264/AVC towards scalability by an Annex (G) called Scalable Video Coding
Scalable Video Coding
Scalable Video Coding is the name for the Annex G extension of the H.264/MPEG-4 AVC video compression standard. SVC standardizes the encoding of a high-quality video bitstream that also contains one or more subset bitstreams. A subset video bitstream is derived by dropping packets from the...

 (SVC). The JVT management team was extended by Jens-Rainer Ohm (Aachen University, Germany). From July 2006 to November 2009, the JVT worked on Multiview Video Coding
Multiview Video Coding
Multiview Video Coding is an amendment to H.264/MPEG-4 AVC video compression standard developed with joint efforts by MPEG/VCEG that enables efficient encoding of sequences captured simultaneously from multiple cameras using a single video stream....

 (MVC), an extension of H.264/AVC towards free viewpoint television
Free viewpoint television
Free viewpoint television is a system for viewing natural video, allowing the user to interactively control the viewpoint and generate new views of a dynamic scene from any 3D position....

 and 3D television
3D television
A 3D television is a television set that employs techniques of 3D presentation, such as stereoscopic capture, multi-view capture, or 2D-plus-depth, and a 3D display – a special viewing device to project a television program into a realistic three-dimensional field.- History :In the late-1890's,...

. That work included the development of two new profiles of the standard: the Multiview High Profile and the Stereo High Profile.

Applications


The H.264 video format has a very broad application range that covers all forms of digital compressed video from low bit-rate Internet streaming applications to HDTV broadcast and Digital Cinema applications with nearly lossless coding. With the use of H.264, bit rate savings of 50% or more are reported. For example, H.264 has been reported to give the same Digital Satellite TV quality as current MPEG-2 implementations with less than half the bitrate, with current MPEG-2 implementations working at around 3.5 Mbit/s and H.264 at only 1.5 Mbit/s. To ensure compatibility and problem-free adoption of H.264/AVC, many standards bodies have amended or added to their video-related standards so that users of these standards can employ H.264/AVC.

Both the Blu-ray Disc
Blu-ray Disc
Blu-ray Disc is an optical disc storage medium designed to supersede the DVD format. The plastic disc is 120 mm in diameter and 1.2 mm thick, the same size as DVDs and CDs. Blu-ray Discs contain 25 GB per layer, with dual layer discs being the norm for feature-length video discs...

 format and the now-discontinued HD DVD
HD DVD
HD DVD is a discontinued high-density optical disc format for storing data and high-definition video.Supported principally by Toshiba, HD DVD was envisioned to be the successor to the standard DVD format...

 format include the H.264/AVC High Profile as one of 3 mandatory video compression formats.

The Digital Video Broadcast project (DVB) approved the use of H.264/AVC for broadcast television in late 2004.

The Advanced Television Systems Committee
Advanced Television Systems Committee
The Advanced Television Systems Committee is the group, established in 1982, that developed the eponymous ATSC Standards for digital television in the United States, also adopted by Canada, Mexico, South Korea, and recently Honduras and is being considered by other countries.-See also:*ATSC...

 (ATSC) standards body in the United States approved the use of H.264/AVC for broadcast television in July 2008, although the standard is not yet used for fixed ATSC broadcasts within the United States. It has also been approved for use with the more recent ATSC-M/H
ATSC-M/H
ATSC-M/H is a standard in the USA for mobile digital TV, that allows TV broadcasts to be received by mobile devices.....

 (Mobile/Handheld) standard, using the AVC and SVC portions of H.264.

AVCHD
AVCHD
AVCHD is a file-based format for the digital recording and playback of high-definition video....

 is a high-definition recording format designed by Sony
Sony
, commonly referred to as Sony, is a Japanese multinational conglomerate corporation headquartered in Minato, Tokyo, Japan and the world's fifth largest media conglomerate measured by revenues....

 and Panasonic
Panasonic
Panasonic is an international brand name for Japanese electric products manufacturer Panasonic Corporation, which was formerly known as Matsushita Electric Industrial Co., Ltd...

 that uses H.264 (conforming to H.264 while adding additional application-specific features and constraints).

AVC-Intra
AVC-Intra
AVC-Intra is a type of video coding developed by Panasonic that is fully compliant with the H.264/MPEG-4 AVC standard and additionally follows the SMPTE RP 2027-2007 recommended practice specification....

 is an intraframe-only compression format, developed by Panasonic
Panasonic
Panasonic is an international brand name for Japanese electric products manufacturer Panasonic Corporation, which was formerly known as Matsushita Electric Industrial Co., Ltd...

.

The CCTV (Closed Circuit TV) and Video Surveillance markets have included the technology in many products.

Canon DSLRs use the H.264 QuickTime MOV as the native recording.

Patent licensing


In countries where patents on software algorithms
Software patent
Software patent does not have a universally accepted definition. One definition suggested by the Foundation for a Free Information Infrastructure is that a software patent is a "patent on any performance of a computer realised by means of a computer program".In 2005, the European Patent Office...

 are upheld, vendors and commercial users of products that use H.264/AVC are expected to pay patent licensing royalties for the patented technology that their products use. This applies to the Baseline Profile as well.
A private organization known as MPEG LA
MPEG LA
MPEG LA, LLC, is a Denver-based firm that licenses patent pools covering essential patents requiredfor use of the MPEG-2, MPEG-4 Visual , IEEE 1394, VC-1, ATSC and AVC/H.264 standards...

, which is not affiliated in any way with the MPEG standardization organization, administers the licenses for patents applying to this standard, as well as the patent pool
Patent pool
In patent law, a patent pool is a consortium of at least two companies agreeing to cross-license patents relating to a particular technology. The creation of a patent pool can save patentees and licensees time and money, and, in case of blocking patents, it may also be the only reasonable method...

s for MPEG-2 Part 1 Systems, MPEG-2 Part 2 Video, MPEG-4 Part 2 Video, and other technologies. The MPEG-LA patents in the US last at least until 2027.

On August 26, 2010 MPEG LA announced that H.264 encoded internet video that is free to end users will never be charged for royalties. All other royalties will remain in place such as the royalties for products that decode and encode H.264 video. The license terms are updated in 5-year blocks.

In 2005, Qualcomm, which was the assignee of and , sued Broadcom in US District Court, alleging that Broadcom infringed the two patents by making products that were compliant with the H.264 video compression standard. In 2007, the District Court found that the patents were unenforceable because Qualcomm had failed to disclose them to the JVT prior to the release of the H.264 standard in May 2003. In December 2008, the US Court of Appeals for the Federal Circuit affirmed the District Court's order that the patents be unenforceable but remanded to the District Court with instructions to limit the scope of unenforceability to H.264 compliant products.

Controversies


Controversies surrounding the H.264 video compression standard stem primarily from its use within the HTML5 Internet standard. HTML5 adds two new tags to the HTML standard:
HTML5 video
HTML5 video is an element introduced in the HTML5 draft specification for the purpose of playing videos or movies, partially replacing the object element...

 and