Subjective video quality

Subjective video quality is a subjective characteristic of video quality. It is concerned with how video is perceived by a viewer and designates his or her opinion on a particular video sequence. Subjective video quality tests are quite expensive in terms of time (preparation and running) and human resources.

Measurement

The main idea of measuring subjective video quality is the same as in the Mean Opinion Score for audio.To evaluate the subjective video quality of a video processing system:

*Choose video sequences for testing (they are often named SRC).
*Choose settings of system that you want to evaluate (often named HRC or Hyperthetical Reference Circuit).
*Choose a test method (how sequences are presented to experts and how their opinion is collected: DSCQS, SSCQE, DSCS).
*Invite sufficient number of experts (18 or more is recommended).
*Carry out testing.
*Calculate average marks for each HRC based on experts' opinion.

Many parameters of the viewing conditions can influence the results, such as room illumination, display type, brightness, contrast, resolution, viewing distance, and the age and educational level of experts.

Testing methods

There are an enormous number of ways of showing video sequences to experts and to record their opinions. A few of them have been standardized. They are thoroughly described in ITU-R recommendation BT.500. One of the standardized methods is DSIS - Double Stimulus Impairment Scale: the expert is presented with an unimpaired reference video, then with the same video impaired, and after that he is asked to vote on the second video using an impairment scale (from "impairments are imperceptible" to "impairments are very annoying").

Further refined testing methods to take into account low picture resolutions (VGA, CIF and QCIF), e.g. for mobile and multimedia applications are referred to in ITU-T Rec. P.910. Advanced setups for typical artefacts of high resolution (HDTV), e.g. in next generation networks incl. IPTV are also under development within the Video Quality Experts Group (VQEG).

Analysis of results

Opinions of experts can be averaged; average mark is usually given with confidence interval. Additional procedures can be used for averaging, for example experts who give unstable results can be rejected (for instance, if their correlation with average opinion is small)...

ee also

* Video quality
* Quality
* Mean Opinion Score
* PSNR
* Perceptual Evaluation of Video Quality (PEVQ)
* MDI
* SSIM

Further reading

* [http://bellerofonte.dii.unisi.it/dottorato/corsi/corsi2003/Perceptual%20Image%20Modeling/Further_readings/Part4/ITU-R_BT.500-11.pdf ITU-R BT.500-11 Recommendation: Methodology for the subjective assessment of the quality of television pictures]
* [http://www.its.bldrdoc.gov/pub/n3/video/spie03subj.pdf Comparing subjective video quality testing methodologies]
* [http://www.shenick.com/pdfs/Testing_MPEG_IPTV_VOD_QOE.pdf IPTV MPEG and Quality of Experience Testing]

External links

* [http://www.itu.int/rec/recommendation.asp?type=products&lang=e&parent=R-REC-BT ITU-T page with recommendations on subjective video quality assessments]
* [http://www.its.bldrdoc.gov/vqeg/ Video Quality Experts Group]