PEVQ

PEVQ (Perceptual Evaluation of Video Quality) is a standardized end-to-end (E2E) measurement algorithm to score the picture quality of a video presentation by means of a 5-point mean opinion score (MOS). The measurement algorithm can be applied to analyze visible artefacts caused by a digital video encoding/decoding (or transcoding) process, RF- or IP-based transmission networks and end-user devices. Application scenarios address next generation networking and mobile services and include IPTV (Standard-definition television and HDTV), streaming video, Mobile TV, video telephony, video conferencing and video messaging.

Measurement Scope

The development for picture quality analysis algorithms available today started with still image models which were later enhanced to also cover motion pictures. The measurement paradigm is to assess degradations of a decoded video sequence output from the network (for example as received by a TV set top box) in comparison to the original reference picture (broadcasted from the studio). Consequently, the setup is referred to as end-to-end (E2E) quality testing.

Because the setup is exactly reflecting the situation how human viewers would evaluate the video quality based on subjective comparison, it addresses Quality-of-Experience (QoE) testing. PEVQ is based on modelling the behaviour of the human visual tract and besides an overall quality MOS score (as a figure of merit) abnormalities in the video signal are quantified by a variety of KPIs, including PSNR, distortion indicators and lip-sync delay.

Testing Typology

Depending on the information that is made available to the algorithm, video quality test algorithms can be divided into three categories:

(1) A “Full Reference” (FR) algorithm has access to and makes use of the original reference sequence for a comparison (i.e. a difference analysis). It can compare each pixel of the reference sequence to each corresponding pixel of the degraded sequence. FR measurements deliver the highest accuracy and repeatability but tend to be processing intensive.

(2) A “Reduced Reference” (RR) algorithm uses a reduced side channel between the sender and the receiver which is not capable of transmitting the full reference signal. Instead, parameters are extracted at the sending side which help predicting the quality at the receiving side. RR measurements may offer reduced accuracy and represent a working compromise if bandwidth for the reference signal is limited.

(3) A “No Reference” (NR) algorithm only uses the degraded signal for the quality estimation and has no information of the original reference sequence. NR algorithms are low accuracy estimates, only, as the originating quality of the source reference is completely unknown. A common variant of NR algorithms don't even analyze the decoded video on a pixel level but work on an analysis of the digital bit stream on an IP packet level, only. The measurement is consequently limited to a transport stream analysis.

PEVQ is full-reference algorithm according to (1) and analyzes the picture pixel-by-pixel after a temporal alignment (also referred to as 'temporal registration') of corresponding frames of reference and test signal. PEVQ MOS results range from 1 (bad) to 5 (excellent).

Verification by Subjective Testing

The accuracy of perceptual objective test methods can be verified by comparison with subjective video quality tests. However, subjective testing can be both time-consuming and costly. In order to achieve statistically relevant results a huge test population must be evaluated. Procedures for subjective video quality testing have been standardized, e.g. in ITU-R Rec. BT.500. Extensions to take into account low picture resolutions (VGA, CIF and QCIF), e.g. for mobile and multimedia applications are referred to in ITU-T Rec. P.910. Advanced setups for typical artefacts of high resolution (HDTV), e.g. in next generation networks incl. IPTV are also under development within the Video Quality Experts Group (VQEG).

Independent Validation and International Standardization

PEVQ was benchmarked by the Video Quality Experts Group (VQEG) in the course of the Multimedia Test Phase 2007-2008. Based on the performance results PEVQ became part of the new International Standard ITU-T Rec. J. 247 (2008).

References

* [http://www.itu.int/rec/T-REC-J.247/en ITU-T Rec. J.247 (08/08) Objective perceptual multimedia video quality measurement in the presence of a full reference]
* [http://www.itu.int/rec/T-REC-P.910/en ITU-T Rec. P.910 Subjective video quality assessment methods for multimedia applications]
* [http://stefan.winkler.net/book.html Digital Video Quality] , [http://www.genista.com/management.htm Stefan Winkler] , Wiley, March 2005, ISBN 0-470-02404-6

ee also

* Video quality
* Subjective video quality
* Video codecs
* Mean Opinion Score
* PSNR
* Perceptual Evaluation of Speech Quality (PESQ)
* Perceptual Evaluation of Audio Quality (PEAQ)
* Perceptual Speech Quality Measure (PSQM)

Further reading

* [http://www.ieeexplore.ieee.org/xpl/freeabs_all.jsp?isnumber=441276&arnumber=4412851&count=122&index=54 PEVQ Paper MMSP 2007, IEEE 9th Workshop on Multimedia Signal Processing ]
* [http://www.its.bldrdoc.gov/vqeg/projects/multimedia/ VQEG Multimedia Test Plan and Final Report]
* [http://www.shenick.com/pdfs/Testing_MPEG_IPTV_VOD_QOE.pdf Testing MPEG based IP video QoE/QoS]

External links

* [http://www.pevq.org PEVQ Market Overview]
* [http://www.pevq.org/video-quality-measurement.html PEVQ Technology Overview]
* [http://www.vqeg.org Video Quality Experts Group]