Video Acceleration API
Encyclopedia
Video Acceleration API (VA API) is an open source
software library ("libVA") and API
specification. It enables and provides access to graphics hardware (GPU
) acceleration for video processing. VA API is targeted at the X Window System
on Unix
-based operating system
s (including Linux
, FreeBSD
, and Solaris). Accelerated processing includes video decoding
, video encoding, subpicture blending and rendering. The specification was originally designed by Intel for its GMA (Graphics Media Accelerator)
series of GPU
hardware. However, the API is not limited to GPUs or Intel specific hardware, as other hardware and manufacturers can also freely use this API for hardware accelerated video decoding.
VA API was designed with the purpose of some day fully replacing XvMC
, which used to be the Unix equivalent of the Microsoft Windows DirectX Video Acceleration (DxVA) API, and more.
, IDCT, Motion Compensation
, deblocking
) for the prevailing coding standards today (MPEG-2
, MPEG-4 ASP
/H.263
, MPEG-4 AVC/H.264
, and VC-1/WMV3
). Extending XvMC was considered, but due to its original design for MPEG-2 MotionComp
only, it made more sense to design an interface from scratch that can fully expose the video decode capabilities in today's GPU
s.
As of October 3, 2011 VA API is natively supported by :
VDPAU (Video Decode and Presentation API for Unix
), a competing API designed by NVIDIA
, can potentially be also used as a backend for the VA API. If this is supported, any software that supports VA API then also indirectly supports a subset of VDPAU. Additionally, in November 2009, VA-API gained a new proprietary
backend named "xvba-video" which allows VA-API powered applications to take advantage of AMD Radeon's UVD
2 chipsets via the XvBA library (X-Video Bitstream Acceleration API designed by AMD).
. In a nutshell it is basically a scheme to pass various types of data buffers from the application to the GPU
for decoding
a compressed bit-stream
. Feedback on this API
is greatly welcomed, as this is intended to be a community collaborative effort.
http://www.freedesktop.org/wiki/Software/vaapi
"A new video acceleration API is being developed, in an effort led by Intel. This new API supports more complete offload (VLD) as well as iDCT+MC, and can support acceleration of MPEG4, H.264, VC-1, as well as MPEG-2."
http://www.mythtv.org/wiki/XvMC
The "VA API" and "libVA" can also be read about here:
"The end user impact is improved performance of H.264, VC-1, MPEG-2 and MPEG-4 video playback with media players that use the new API compared to playback using a pure software implementation."
https://wiki.ubuntu.com/mobile-hw-decode
Some softwares may gain VA API support in the future : Lightspark
(Flash / SWF
player), GStreamer
(via "gstreamer-vaapi" VA-API plugins for GStreamer) and Xine
(via "xine-lib-vaapi" library).
Open source
The term open source describes practices in production and development that promote access to the end product's source materials. Some consider open source a philosophy, others consider it a pragmatic methodology...
software library ("libVA") and API
Application programming interface
An application programming interface is a source code based specification intended to be used as an interface by software components to communicate with each other...
specification. It enables and provides access to graphics hardware (GPU
Graphics processing unit
A graphics processing unit or GPU is a specialized circuit designed to rapidly manipulate and alter memory in such a way so as to accelerate the building of images in a frame buffer intended for output to a display...
) acceleration for video processing. VA API is targeted at the X Window System
X Window System
The X window system is a computer software system and network protocol that provides a basis for graphical user interfaces and rich input device capability for networked computers...
on Unix
Unix
Unix is a multitasking, multi-user computer operating system originally developed in 1969 by a group of AT&T employees at Bell Labs, including Ken Thompson, Dennis Ritchie, Brian Kernighan, Douglas McIlroy, and Joe Ossanna...
-based operating system
Operating system
An operating system is a set of programs that manage computer hardware resources and provide common services for application software. The operating system is the most important type of system software in a computer system...
s (including Linux
Linux
Linux is a Unix-like computer operating system assembled under the model of free and open source software development and distribution. The defining component of any Linux system is the Linux kernel, an operating system kernel first released October 5, 1991 by Linus Torvalds...
, FreeBSD
FreeBSD
FreeBSD is a free Unix-like operating system descended from AT&T UNIX via BSD UNIX. Although for legal reasons FreeBSD cannot be called “UNIX”, as the direct descendant of BSD UNIX , FreeBSD’s internals and system APIs are UNIX-compliant...
, and Solaris). Accelerated processing includes video decoding
Video codec
A video codec is a device or software that enables video compression and/or decompression for digital video. The compression usually employs lossy data compression. Historically, video was stored as an analog signal on magnetic tape...
, video encoding, subpicture blending and rendering. The specification was originally designed by Intel for its GMA (Graphics Media Accelerator)
Intel GMA
The Intel Graphics Media Accelerator, or GMA, is a series of Intel integrated graphics processors built into various motherboard chipsets....
series of GPU
Graphics processing unit
A graphics processing unit or GPU is a specialized circuit designed to rapidly manipulate and alter memory in such a way so as to accelerate the building of images in a frame buffer intended for output to a display...
hardware. However, the API is not limited to GPUs or Intel specific hardware, as other hardware and manufacturers can also freely use this API for hardware accelerated video decoding.
VA API was designed with the purpose of some day fully replacing XvMC
XvMC
X-Video Motion Compensation , is an extension of the X video extension for the X Window System. The XvMC API allows video programs to offload portions of the video decoding process to the GPU video-hardware. In theory this process should also reduce bus bandwidth requirements...
, which used to be the Unix equivalent of the Microsoft Windows DirectX Video Acceleration (DxVA) API, and more.
Overview
The main motivation for VA API is to enable hardware accelerated video decode at various entry-points (VLDVariable-length code
In coding theory a variable-length code is a code which maps source symbols to a variable number of bits.Variable-length codes can allow sources to be compressed and decompressed with zero error and still be read back symbol by symbol...
, IDCT, Motion Compensation
Motion compensation
Motion compensation is an algorithmic technique employed in the encoding of video data for video compression, for example in the generation of MPEG-2 files. Motion compensation describes a picture in terms of the transformation of a reference picture to the current picture. The reference picture...
, deblocking
Deblocking filter (video)
A deblocking filter is a video filter applied to blocks in decoded video to improve visual quality and prediction performance by smoothing the sharp edges which can form between macroblocks when block coding techniques are used...
) for the prevailing coding standards today (MPEG-2
H.262/MPEG-2 Part 2
H.262 or MPEG-2 Part 2 is a digital video compression and encoding standard developed and maintained jointly by ITU-T Video Coding Experts Group and ISO/IEC Moving Picture Experts Group . It is the second part of the ISO/IEC MPEG-2 standard...
, MPEG-4 ASP
MPEG-4 Part 2
MPEG-4 Part 2, MPEG-4 Visual is a video compression technology developed by MPEG. It belongs to the MPEG-4 ISO/IEC standards. It is a discrete cosine transform compression standard, similar to previous standards such as MPEG-1 and MPEG-2...
/H.263
H.263
H.263 is a video compression standard originally designed as a low-bitrate compressed format for videoconferencing. It was developed by the ITU-T Video Coding Experts Group in a project ending in 1995/1996 as one member of the H.26x family of video coding standards in the domain of the ITU-T.H.263...
, MPEG-4 AVC/H.264
H.264/MPEG-4 AVC
H.264/MPEG-4 Part 10 or AVC is a standard for video compression, and is currently one of the most commonly used formats for the recording, compression, and distribution of high definition video...
, and VC-1/WMV3
VC-1
VC-1 is the informal name of the SMPTE 421M video codec standard, which was initially developed as a proprietary video format by Microsoft before it was released as a formal SMPTE standard video format on April 3, 2006...
). Extending XvMC was considered, but due to its original design for MPEG-2 MotionComp
Motion compensation
Motion compensation is an algorithmic technique employed in the encoding of video data for video compression, for example in the generation of MPEG-2 files. Motion compensation describes a picture in terms of the transformation of a reference picture to the current picture. The reference picture...
only, it made more sense to design an interface from scratch that can fully expose the video decode capabilities in today's GPU
Graphics processing unit
A graphics processing unit or GPU is a specialized circuit designed to rapidly manipulate and alter memory in such a way so as to accelerate the building of images in a frame buffer intended for output to a display...
s.
As of October 3, 2011 VA API is natively supported by :
- the non-free drivers for Intel's Poulsbo ChipsetPoulsbo (chipset)System Controller Hub is a family of Intel microchips employed in chipsets for low-power Atom-based platforms. Its architecture is consistent with the Intel Hub Architecture but combines the traditional northbridge and southbridge functions into a single microchip.- Poulsbo :Poulsbo is the...
with GMA 500 integrated graphics - the free drivers of the Intel G45 chipset (with Intel GMA X4500HD integrated graphics) and later
- SoCSystem-on-a-chipA system on a chip or system on chip is an integrated circuit that integrates all components of a computer or other electronic system into a single chip. It may contain digital, analog, mixed-signal, and often radio-frequency functions—all on a single chip substrate...
Atom E6xx are also supported, but a non-free software from Intel, Media Infrastructure Accelerator (MI-X), may be mandatory - S3 GraphicsS3 GraphicsS3 Graphics, Ltd is an American company specializing in graphics chipsets. Although they do not have the large market share that they once had, they still produce graphics accelerators for home computers under the "S3 Chrome" brand name.-History:...
's Chrome 400S3 Graphics Chrome 400-History:VIA planned the production of 2 new graphics chips, Chrome 440 and 430, by the end of 2007. The Chrome 460 will support DirectX 10 and adopts a 90 nm process, while the Chrome 430 will support DirectX 10.1 and adopts a 65nm process. The Chrome 400 series was codenamed 'Destination'...
and later series are also supported.
VDPAU (Video Decode and Presentation API for Unix
VDPAU
VDPAU is an open source library and API originally designed by Nvidia for its GeForce 8 series and later GPU hardware, targeted at the X Window System on Unix-based operating systems...
), a competing API designed by NVIDIA
NVIDIA
Nvidia is an American global technology company based in Santa Clara, California. Nvidia is best known for its graphics processors . Nvidia and chief rival AMD Graphics Techonologies have dominated the high performance GPU market, pushing other manufacturers to smaller, niche roles...
, can potentially be also used as a backend for the VA API. If this is supported, any software that supports VA API then also indirectly supports a subset of VDPAU. Additionally, in November 2009, VA-API gained a new proprietary
Proprietary software
Proprietary software is computer software licensed under exclusive legal right of the copyright holder. The licensee is given the right to use the software under certain conditions, while restricted from other uses, such as modification, further distribution, or reverse engineering.Complementary...
backend named "xvba-video" which allows VA-API powered applications to take advantage of AMD Radeon's UVD
Unified Video Decoder
The Unified Video Decoder, previously called Universal Video Decoder, or UVD in short, is the video decoding unit from ATI Technologies to support hardware decode of H.264 and VC-1 video codec standards, and being a part of ATI Avivo HD technology....
2 chipsets via the XvBA library (X-Video Bitstream Acceleration API designed by AMD).
Processes that can be accelerated with VA API
Processes that can be accelerated if both the device drivers and GPU hardware supports them:- Motion compensationMotion compensationMotion compensation is an algorithmic technique employed in the encoding of video data for video compression, for example in the generation of MPEG-2 files. Motion compensation describes a picture in terms of the transformation of a reference picture to the current picture. The reference picture...
(mocomp) - Inverse discrete cosine transform (iDCT)
- In-loop deblocking filterDeblocking filter (video)A deblocking filter is a video filter applied to blocks in decoded video to improve visual quality and prediction performance by smoothing the sharp edges which can form between macroblocks when block coding techniques are used...
- Intra-frame prediction
- Variable-Length DecodingHuffman codingIn computer science and information theory, Huffman coding is an entropy encoding algorithm used for lossless data compression. The term refers to the use of a variable-length code table for encoding a source symbol where the variable-length code table has been derived in a particular way based on...
(VLD), more commonly known as slice-level acceleration - Bitstream processing (CAVLCContext-adaptive variable-length codingContext-adaptive variable-length coding is a form of entropy coding used in H.264/MPEG-4 AVC video encoding. It is an inherently lossless compression technique, like almost all entropy-coders. In H.264/MPEG-4 AVC, it is used to encode residual, zig-zag order, blocks of transform coefficients. It...
/CABACContext-adaptive binary arithmetic codingContext-adaptive binary arithmetic coding is a form of entropy encoding used in H.264/MPEG-4 AVC video encoding. It is a lossless compression technique. It is notable for providing much better compression than most other encoding algorithms used in video encoding, and is one of the primary...
)
Architecture
The current interface is focused on video decode only and is window system independent, so that potentially it can be used with graphics sub-systems other than the X Window SystemX Window System
The X window system is a computer software system and network protocol that provides a basis for graphical user interfaces and rich input device capability for networked computers...
. In a nutshell it is basically a scheme to pass various types of data buffers from the application to the GPU
Graphics processing unit
A graphics processing unit or GPU is a specialized circuit designed to rapidly manipulate and alter memory in such a way so as to accelerate the building of images in a frame buffer intended for output to a display...
for decoding
Video codec
A video codec is a device or software that enables video compression and/or decompression for digital video. The compression usually employs lossy data compression. Historically, video was stored as an analog signal on magnetic tape...
a compressed bit-stream
Elementary stream
An elementary stream as defined by MPEG communication protocol is usually the output of an audio or video encoder. ES contains only one kind of data, e.g. audio, video or closed caption. An elementary stream is often referred to as "elementary", "data", "audio", or "video" bitstreams or streams...
. Feedback on this API
Application programming interface
An application programming interface is a source code based specification intended to be used as an interface by software components to communicate with each other...
is greatly welcomed, as this is intended to be a community collaborative effort.
Sources
"This API is intended to provide an interface between a video decode application (client) and a hardware decode accelerator (server), to off-load video decode operations from the host to the hardware accelerator at various entry-points."http://www.freedesktop.org/wiki/Software/vaapi
"A new video acceleration API is being developed, in an effort led by Intel. This new API supports more complete offload (VLD) as well as iDCT+MC, and can support acceleration of MPEG4, H.264, VC-1, as well as MPEG-2."
http://www.mythtv.org/wiki/XvMC
The "VA API" and "libVA" can also be read about here:
"The end user impact is improved performance of H.264, VC-1, MPEG-2 and MPEG-4 video playback with media players that use the new API compared to playback using a pure software implementation."
https://wiki.ubuntu.com/mobile-hw-decode
Software supporting VA API
- Helix media playerHelix (project)Helix DNA is a project to produce software that can play audio and video media in various formats, aid in producing such media, and serve them over a network. It is intended as a largely free and open source digital media framework that runs on numerous operating systems and processors and was...
(LinuxLinuxLinux is a Unix-like computer operating system assembled under the model of free and open source software development and distribution. The defining component of any Linux system is the Linux kernel, an operating system kernel first released October 5, 1991 by Linus Torvalds...
) - Clutter (toolkit)
- GnashGnashGnash is a media player for playing SWF files. Gnash is available both as a standalone player for desktop computers and embedded device, as well as a plugin for several browsers. It is part of the GNU Project and is a Free and open-source alternative to Adobe Flash Player...
Flash / SWFSWFSWF is an Adobe Flash file format used for multimedia, vector graphics and ActionScript. Originating with FutureWave Software, then transferred to Macromedia, and then coming under the control of Adobe, SWF files can contain animations or applets of varying degrees of interactivity and function.,...
player
- ffmpeg, a command line tool from the FFmpegFFmpegFFmpeg is a free software project that produces libraries and programs for handling multimedia data. The most notable parts of FFmpeg are libavcodec, an audio/video codec library used by several other projects, libavformat, an audio/video container mux and demux library, and the ffmpeg command line...
project, does not support VA-API, but libavcodecLibavcodeclibavcodec is a free software/open source LGPL-licensed library of codecs for encoding and decoding video and audio data. Same name but incompatible libraries are provided from both FFmpeg project and Libav project....
(also part of the FFmpeg project) does contain code that other applications use to support hardware accelerated decoding, including VA API. - FluendoFluendoFluendo DVD Player is one of the first products that can be bought that allows legal/licensed DVD playback for Unix/Linux users...
- MythTVMythTVMythTV is a free and open source home entertainment application with a simplified "10-foot user interface" design for the living-room TV, and turns a computer with the necessary hardware into a network streaming digital video recorder, a digital multimedia home entertainment system, or home theater...
- XBMC Media Center (LinuxLinuxLinux is a Unix-like computer operating system assembled under the model of free and open source software development and distribution. The defining component of any Linux system is the Linux kernel, an operating system kernel first released October 5, 1991 by Linus Torvalds...
) - XBMC Live (Linux Live CD/USB operating-system)
- VLC media playerVLC media playerVLC media player is a free and open source media player and multimedia framework written by the VideoLAN project.VLC is a portable multimedia player, encoder, and streamer supporting many audio and video codecs and file formats as well as DVDs, VCDs, and various streaming protocols. It is able to...
(starting from release 1.1.0)
Some softwares may gain VA API support in the future : Lightspark
Lightspark
Lightspark is a free and open source SWF player released under the terms of the GNU Lesser General Public License version 3. It supports OpenGL-based rendering and LLVM-based ActionScript execution. It supports most of ActionScript 3.0 and has a Mozilla-compatible plug-in. It uses OpenGL shaders ....
(Flash / SWF
SWF
SWF is an Adobe Flash file format used for multimedia, vector graphics and ActionScript. Originating with FutureWave Software, then transferred to Macromedia, and then coming under the control of Adobe, SWF files can contain animations or applets of varying degrees of interactivity and function.,...
player), GStreamer
GStreamer
GStreamer is a pipeline-based multimedia framework written in the C programming language with the type system based on GObject.GStreamer allows a programmer to create a variety of media-handling components, including simple audio playback, audio and video playback, recording, streaming and editing...
(via "gstreamer-vaapi" VA-API plugins for GStreamer) and Xine
Xine
xine is a multimedia playback engine for Unix-like operating systems released under the GNU General Public License. xine is built around a shared library that supports different frontend player applications. Another important feature of xine is the ability to manually correct the synchronization...
(via "xine-lib-vaapi" library).
See also
- VDPAU (Video Decode and Presentation API for Unix)VDPAUVDPAU is an open source library and API originally designed by Nvidia for its GeForce 8 series and later GPU hardware, targeted at the X Window System on Unix-based operating systems...
- X-Video Bitstream Acceleration (XvBA)X-Video Bitstream AccelerationX-Video Bitstream Acceleration , designed by AMD for its Radeon GPU and Fusion APU, is an arbitrary extension of the X video extension for the X Window System on Linux operating-systems. XvBA API allows video programs to offload portions of the video decoding process to the GPU video-hardware...
- X-Video Motion Compensation (XvMC)
- DirectX Video Acceleration (DxVA) API - Microsoft WindowsMicrosoft WindowsMicrosoft Windows is a series of operating systems produced by Microsoft.Microsoft introduced an operating environment named Windows on November 20, 1985 as an add-on to MS-DOS in response to the growing interest in graphical user interfaces . Microsoft Windows came to dominate the world's personal...
analogue - Video Decode Acceleration Framework is Apple Inc.s API for hardware-accelerated decoding of H.264 on Mac OS XMac OS XMac OS X is a series of Unix-based operating systems and graphical user interfaces developed, marketed, and sold by Apple Inc. Since 2002, has been included with all new Macintosh computer systems...
- VideoToolBox is an undocumented API from Apple Inc. for hardware-accelerated decoding on Apple TV and Mac OS XMac OS XMac OS X is a series of Unix-based operating systems and graphical user interfaces developed, marketed, and sold by Apple Inc. Since 2002, has been included with all new Macintosh computer systems...
10.5 or later. - OpenVideo Decode (OVD) – an new open cross-platform video acceleration API from AMD.
- OpenMAX IL (Open Media Acceleration Integration Layer)OpenMAXOpenMAX is a royalty-free, cross-platform set of C-language programming interfaces that provides abstractions for routines especially useful for audio, video, and still images...
- a royalty-free cross-platform media abstraction API from the Khronos GroupKhronos GroupThe Khronos Group is a not-for-profit member-funded industry consortium based in Beaverton, Oregon, focused on the creation of open standard, royalty-free APIs to enable the authoring and accelerated playback of dynamic media on a wide variety of platforms and devices... - Nvidia PureVideoNvidia PureVideoNvidia PureVideo is a hardware feature designed to offload video decoding processes and video post-processing from a computer's CPU hardware to Nvidia's GPU hardware series GeForce 6 and later, GeForce M series ; and Nvidia Quadro series...
- the bit-stream technologyStream processingStream processing is a computer programming paradigm, related to SIMD , that allows some applications to more easily exploit a limited form of parallel processing...
from NVIDIANVIDIANvidia is an American global technology company based in Santa Clara, California. Nvidia is best known for its graphics processors . Nvidia and chief rival AMD Graphics Techonologies have dominated the high performance GPU market, pushing other manufacturers to smaller, niche roles...
used in their graphics chips to accelerate video decoding on hardware GPUGraphics processing unitA graphics processing unit or GPU is a specialized circuit designed to rapidly manipulate and alter memory in such a way so as to accelerate the building of images in a frame buffer intended for output to a display...
. - UVD (Unified Video Decoder)Unified Video DecoderThe Unified Video Decoder, previously called Universal Video Decoder, or UVD in short, is the video decoding unit from ATI Technologies to support hardware decode of H.264 and VC-1 video codec standards, and being a part of ATI Avivo HD technology....
- the bit-stream technologyStream processingStream processing is a computer programming paradigm, related to SIMD , that allows some applications to more easily exploit a limited form of parallel processing...
from ATI TechnologiesATI TechnologiesATI Technologies Inc. was a semiconductor technology corporation based in Markham, Ontario, Canada, that specialized in the development of graphics processing units and chipsets. Founded in 1985 as Array Technologies Inc., the company was listed publicly in 1993 and was acquired by Advanced Micro...
used in their graphics chips to accelerate video decoding on hardware GPUGraphics processing unitA graphics processing unit or GPU is a specialized circuit designed to rapidly manipulate and alter memory in such a way so as to accelerate the building of images in a frame buffer intended for output to a display...
.
External links
- http://www.freedesktop.org/wiki/Software/vaapi
- http://intellinuxgraphics.org/vaapi.html
- http://wiki.ubuntu.com/mobile-hw-decode
- Splitted-Desktop Systems patches to add VA API support to FFmpeg and MPlayer
- MPlayer, FFmpeg Gain VA-API Support