PEVQ
Encyclopedia
PEVQ is a standardized end-to-end (E2E) measurement algorithm to score the picture quality of a video presentation by means of a 5-point mean opinion score
(MOS). The measurement algorithm can be applied to analyze visible artefacts caused by a digital video encoding/decoding (or transcoding) process, RF- or IP-based transmission networks and end-user devices. Application scenarios address next generation networking
and mobile services and include IPTV
(Standard-definition television
and HDTV), streaming video, Mobile TV
, video telephony, video conferencing and video messaging.
Because the setup is exactly reflecting the situation how human viewers would evaluate the video quality based on subjective comparison, it addresses Quality-of-Experience
(QoE) testing. PEVQ is based on modelling the behaviour of the human visual tract and besides an overall quality MOS score (as a figure of merit) abnormalities in the video signal are quantified by a variety of KPIs, including PSNR, distortion indicators and lip-sync delay.
PEVQ is full-reference algorithm and analyzes the picture pixel-by-pixel after a temporal alignment (also referred to as 'temporal registration') of corresponding frames of reference and test signal. PEVQ MOS results range from 1 (bad) to 5 (excellent).
Rec. BT.500. Extensions to take into account low picture resolutions (VGA, CIF and QCIF), e.g. for mobile and multimedia applications are referred to in ITU-T
Rec. P.910. Advanced setups for typical artefacts of high resolution (HDTV), e.g. in next generation networks incl. IPTV are also under development within the Video Quality Experts Group (VQEG).
Mean Opinion Score
The Mean Opinion Score test has been used for decades in telephony networks to obtain the human user's view of the quality of the network. In multimedia especially when codecs are used to compress the bandwidth requirement , the mean opinion score ...
(MOS). The measurement algorithm can be applied to analyze visible artefacts caused by a digital video encoding/decoding (or transcoding) process, RF- or IP-based transmission networks and end-user devices. Application scenarios address next generation networking
Next Generation Networking
Next-generation network is a broad term used to describe key architectural evolutions in telecommunication core and access networks. The general idea behind the NGN is that one network transports all information and services by encapsulating these into packets, similar to those used on the...
and mobile services and include IPTV
IPTV
Internet Protocol television is a system through which television services are delivered using the Internet protocol suite over a packet-switched network such as the Internet, instead of being delivered through traditional terrestrial, satellite signal, and cable television formats.IPTV services...
(Standard-definition television
Standard-definition television
Sorete-definition television is a television system that uses a resolution that is not considered to be either enhanced-definition television or high-definition television . The term is usually used in reference to digital television, in particular when broadcasting at the same resolution as...
and HDTV), streaming video, Mobile TV
Mobile TV
Mobile television usually means television watched on a small handheld device. It may be a pay TV service broadcast on mobile phone networks or received free-to-air via terrestrial television stations from either regular broadcast or a special mobile TV transmission format...
, video telephony, video conferencing and video messaging.
Measurement scope
The development for picture quality analysis algorithms available today started with still image models which were later enhanced to also cover motion pictures. The measurement paradigm is to assess degradations of a decoded video sequence output from the network (for example as received by a TV set top box) in comparison to the original reference picture (broadcast from the studio). Consequently, the setup is referred to as end-to-end (E2E) quality testing.Because the setup is exactly reflecting the situation how human viewers would evaluate the video quality based on subjective comparison, it addresses Quality-of-Experience
Quality of experience
Quality of experience , some times also known as quality of user experience, is a subjective measure of a customer's experiences with a service...
(QoE) testing. PEVQ is based on modelling the behaviour of the human visual tract and besides an overall quality MOS score (as a figure of merit) abnormalities in the video signal are quantified by a variety of KPIs, including PSNR, distortion indicators and lip-sync delay.
Testing typology
Depending on the information that is made available to the algorithm, video quality test algorithms can be divided into three categories:- A “Full Reference” (FR) algorithm has access to and makes use of the original reference sequence for a comparison (i.e. a difference analysis). It can compare each pixel of the reference sequence to each corresponding pixel of the degraded sequence. FR measurements deliver the highest accuracy and repeatability but tend to be processing intensive.
- A “Reduced Reference” (RR) algorithm uses a reduced side channel between the sender and the receiver which is not capable of transmitting the full reference signal. Instead, parameters are extracted at the sending side which help predicting the quality at the receiving side. RR measurements may offer reduced accuracy and represent a working compromise if bandwidth for the reference signal is limited.
- A “No Reference” (NR) algorithm only uses the degraded signal for the quality estimation and has no information of the original reference sequence. NR algorithms are low accuracy estimates, only, as the originating quality of the source reference is completely unknown. A common variant of NR algorithms don't even analyze the decoded video on a pixel level but work on an analysis of the digital bit stream on an IP packet level, only. The measurement is consequently limited to a transport stream analysis.
PEVQ is full-reference algorithm and analyzes the picture pixel-by-pixel after a temporal alignment (also referred to as 'temporal registration') of corresponding frames of reference and test signal. PEVQ MOS results range from 1 (bad) to 5 (excellent).
Verification by subjective testing
The accuracy of perceptual objective test methods can be verified by comparison with subjective video quality tests. However, subjective testing can be both time-consuming and costly. In order to achieve statistically relevant results a huge test population must be evaluated. Procedures for subjective video quality testing have been standardized, e.g. in ITU-RITU-R
The ITU Radiocommunication Sector is one of the three sectors of the International Telecommunication Union and is responsible for radio communication....
Rec. BT.500. Extensions to take into account low picture resolutions (VGA, CIF and QCIF), e.g. for mobile and multimedia applications are referred to in ITU-T
ITU-T
The ITU Telecommunication Standardization Sector is one of the three sectors of the International Telecommunication Union ; it coordinates standards for telecommunications....
Rec. P.910. Advanced setups for typical artefacts of high resolution (HDTV), e.g. in next generation networks incl. IPTV are also under development within the Video Quality Experts Group (VQEG).
Independent validation and international standardization
PEVQ was benchmarked by the Video Quality Experts Group (VQEG) in the course of the Multimedia Test Phase 2007-2008. Based on the performance results PEVQ became part of the new International Standard ITU-T Rec. J. 247 (2008).See also
- Video qualityVideo qualityVideo quality is a characteristic of a video passed through a video transmission/processing system, a formal or informal measure of perceived video degradation...
- Subjective video qualitySubjective video qualitySubjective video quality is a subjective characteristic of video quality. It is concerned with how video is perceived by a viewer and designates his or her opinion on a particular video sequence...
- Video codecs
- Mean Opinion ScoreMean Opinion ScoreThe Mean Opinion Score test has been used for decades in telephony networks to obtain the human user's view of the quality of the network. In multimedia especially when codecs are used to compress the bandwidth requirement , the mean opinion score ...
- PSNRPeak signal-to-noise ratioThe phrase peak signal-to-noise ratio, often abbreviated PSNR, is an engineering term for the ratio between the maximum possible power of a signal and the power of corrupting noise that affects the fidelity of its representation...
- Perceptual Evaluation of Speech Quality (PESQ)PESQPESQ, Perceptual Evaluation of Speech Quality, is a family of standards comprising a test methodology for automated assessment of the speech quality as experienced by a user of a telephony system. It is standardised as ITU-T recommendation P.862...
- Perceptual Evaluation of Audio Quality (PEAQ)PEAQPEAQ is a standardized algorithm for objectively measuring perceived audio quality, developed in 1994-1998 by a joint venture of experts within Task Group 6Q of the International Telecommunication Union . It was originally released as ITU-R Recommendation BS.1387 in 1998 and last updated in 2001...
- Perceptual Speech Quality Measure (PSQM)PSQMPSQM is a computational and modeling algorithm defined in ITU Recommendation ITU-T P.861 that objectively evaluates and quantifies voice quality of voice-band speech codecs....
Further reading
- PEVQ Paper MMSP 2007, IEEE 9th Workshop on Multimedia Signal Processing
- VQEG Multimedia Test Plan and Final Report
- Testing MPEG based IP video QoE/QoS