fbpx
Wikipedia

Video Multimethod Assessment Fusion

January 01, 1970

Video Multimethod Assessment Fusion (VMAF) is an objective full-reference video quality metric developed by Netflix in cooperation with the University of Southern California, The IPI/LS2N lab Nantes Université, and the Laboratory for Image and Video Engineering (LIVE) at The University of Texas at Austin. It predicts subjective video quality based on a reference and distorted video sequence. The metric can be used to evaluate the quality of different video codecs, encoders, encoding settings, or transmission variants.

History edit

The metric is based on initial work from the group of Professor C.-C. Jay Kuo at the University of Southern California.[1][2][3] Here, the applicability of fusion of different video quality metrics using support vector machines (SVM) has been investigated, leading to a "FVQA (Fusion-based Video Quality Assessment) Index" that has been shown to outperform existing image quality metrics on a subjective video quality database.

The method has been further developed in cooperation with Netflix, using different subjective video datasets, including a Netflix-owned dataset ("NFLX"). Subsequently renamed "Video Multimethod Assessment Fusion", it was announced on the Netflix TechBlog in June 2016[4] and version 0.3.1 of the reference implementation was made available under a permissive open-source license.[5]

In 2017, the metric was updated to support a custom model that includes an adaptation for cellular phone screen viewing, generating higher quality scores for the same input material. In 2018, a model that predicts the quality of up to 4K resolution content was released. The datasets on which these models were trained have not been made available to the public.

In 2021, a Technology and Engineering Emmy Award was awarded to Beamr, Netflix, University of Southern California, University of Nantes, The University of Texas at Austin, SSIMWAVE, Disney, Google, Brightcove and ATEME for the Development of Open Perceptual Metrics for Video Encoding Optimization. It was the second time in 20 years that universities got an Emmy Award. It was also the first time a French University got one.[6][7]

Components edit

VMAF uses existing image quality metrics and other features to predict video quality:

  • Visual Information Fidelity (VIF): considers information fidelity loss at four different spatial scales
  • Detail Loss Metric (DLM):[8] measures loss of details, and impairments which distract viewer attention
  • Mean Co-Located Pixel Difference (MCPD): measures temporal difference between frames on the luminance component

The above features are fused using a SVM-based regression to provide a single output score in the range of 0–100 per video frame, with 100 being quality identical to the reference video. These scores are then temporally pooled over the entire video sequence using the arithmetic mean to provide an overall differential mean opinion score (DMOS).

Due to the public availability of the training source code ("VMAF Development Kit", VDK), the fusion method can be re-trained and evaluated based on different video datasets and features.

Anti-noise signal-to-noise ratio (AN-SNR) was used in earlier versions of VMAF as a quality metric but was subsequently abandoned.[9]

Performance edit

An early version of VMAF has been shown to outperform other image and video quality metrics such as SSIM, PSNR-HVS and VQM-VFD on three of four datasets in terms of prediction accuracy, when compared to subjective ratings.[4] Its performance has also been analyzed in another paper, which found that VMAF did not perform better than SSIM and MS-SSIM on a video dataset.[10] In 2017, engineers from RealNetworks reported good reproducibility of Netflix' performance findings.[11] In MSU video quality metrics benchmark, where its various versions (including VMAF NEG) were tested, VMAF outperformed all other metrics on all compression standards (H.265, VP9, AV1, VVC).

VMAF scores can be artificially increased without improving perceived quality by applying various operations before or after distorting the video, sometimes without impacting the popular PSNR metric.[12][13]

Software edit

A reference implementation written in C and Python ("VMAF Development Kit, VDK") is published as free software under the terms of BSD+Patent license.[14] Its source code and additional material are available on GitHub.[5]

See also edit

References edit

  1. ^ Liu, Tsung-Jung; Lin, Joe Yuchieh; Lin, Weisi; Kuo, C.-C. Jay (2013). "Visual quality assessment: recent developments, coding applications and future trends". APSIPA Transactions on Signal and Information Processing. 2. doi:10.1017/atsip.2013.5. hdl:10356/106287. ISSN 2048-7703.
  2. ^ Lin, Joe Yuchieh; Liu, T. J.; Wu, E. C. H.; Kuo, C. C. J. (December 2014). "A fusion-based video quality assessment (Fvqa) index". Signal and Information Processing Association Annual Summit and Conference (APSIPA), 2014 Asia-Pacific. pp. 1–5. doi:10.1109/apsipa.2014.7041705. ISBN 978-6-1636-1823-8. S2CID 7742774.
  3. ^ Lin, Joe Yuchieh; Wu, Chi-Hao; Ioannis, Katsavounidis; Li, Zhi; Aaron, Anne; Kuo, C.-C. Jay (June 2015). "EVQA: An ensemble-learning-based video quality assessment index". 2015 IEEE International Conference on Multimedia & Expo Workshops (ICMEW). pp. 1–5. doi:10.1109/ICMEW.2015.7169760. ISBN 978-1-4799-7079-7. S2CID 6996075.
  4. ^ a b Blog, Netflix Technology (2016-06-06). "Toward A Practical Perceptual Video Quality Metric". Netflix TechBlog. Retrieved 2017-07-15.
  5. ^ a b vmaf: Perceptual video quality assessment based on multi-method fusion, Netflix, Inc., 2017-07-14, retrieved 2017-07-15
  6. ^ "72nd Annual Technology & Engineering Emmy® Awards Recipients – The Emmys". theemmys.tv. Retrieved 2021-02-08.
  7. ^ PATRON, Julien. "Technologie : l'Université de Nantes récompensée d'un Emmy Award !". UNNEWS (in French). Retrieved 2021-02-08.
  8. ^ Li, S.; Zhang, F.; Ma, L.; Ngan, K. N. (October 2011). "Image Quality Assessment by Separately Evaluating Detail Losses and Additive Impairments". IEEE Transactions on Multimedia. 13 (5): 935–949. doi:10.1109/tmm.2011.2152382. ISSN 1520-9210. S2CID 8618041.
  9. ^ Zhili, Henry. "Removal of AN-SNR". Github.
  10. ^ Bampis, Christos G.; Bovik, Alan C. (2017-03-02). "Learning to Predict Streaming Video QoE: Distortions, Rebuffering and Memory". arXiv:1703.00633 [cs.MM].
  11. ^ Rassool, Reza (2017). "VMAF reproducibility: Validating a perceptual practical video quality metric" (PDF). 2017 IEEE International Symposium on Broadband Multimedia Systems and Broadcasting (BMSB). pp. 1–2. doi:10.1109/BMSB.2017.7986143. ISBN 978-1-5090-4937-0. S2CID 5449498. Retrieved 2017-11-30.
  12. ^ Zvezdakova, Anastasia; Zvezdakov, Sergey; Kulikov, Dmitriy; Vatolin, Dmitriy (2019-04-29). "Hacking VMAF with Video Color and Contrast Distortion". arXiv:2107.04510 [cs.MM].
  13. ^ Siniukov, Maksim; Antsiferova, Anastasia; Kulikov, Dmitriy; Vatolin, Dmitriy (2021). "Hacking VMAF and VMAF NEG: vulnerability to different preprocessing methods". arXiv:2107.04510 [cs.MM].
  14. ^ "BSD+Patent | Open Source Initiative". 18 May 2017.

External links edit

  • Reference implementation

January 01, 1970
video, multimethod, assessment, fusion, vmaf, objective, full, reference, video, quality, metric, developed, netflix, cooperation, with, university, southern, california, ls2n, nantes, université, laboratory, image, video, engineering, live, university, texas,. Video Multimethod Assessment Fusion VMAF is an objective full reference video quality metric developed by Netflix in cooperation with the University of Southern California The IPI LS2N lab Nantes Universite and the Laboratory for Image and Video Engineering LIVE at The University of Texas at Austin It predicts subjective video quality based on a reference and distorted video sequence The metric can be used to evaluate the quality of different video codecs encoders encoding settings or transmission variants Contents 1 History 2 Components 3 Performance 4 Software 5 See also 6 References 7 External linksHistory editThe metric is based on initial work from the group of Professor C C Jay Kuo at the University of Southern California 1 2 3 Here the applicability of fusion of different video quality metrics using support vector machines SVM has been investigated leading to a FVQA Fusion based Video Quality Assessment Index that has been shown to outperform existing image quality metrics on a subjective video quality database The method has been further developed in cooperation with Netflix using different subjective video datasets including a Netflix owned dataset NFLX Subsequently renamed Video Multimethod Assessment Fusion it was announced on the Netflix TechBlog in June 2016 4 and version 0 3 1 of the reference implementation was made available under a permissive open source license 5 In 2017 the metric was updated to support a custom model that includes an adaptation for cellular phone screen viewing generating higher quality scores for the same input material In 2018 a model that predicts the quality of up to 4K resolution content was released The datasets on which these models were trained have not been made available to the public In 2021 a Technology and Engineering Emmy Award was awarded to Beamr Netflix University of Southern California University of Nantes The University of Texas at Austin SSIMWAVE Disney Google Brightcove and ATEME for the Development of Open Perceptual Metrics for Video Encoding Optimization It was the second time in 20 years that universities got an Emmy Award It was also the first time a French University got one 6 7 Components editVMAF uses existing image quality metrics and other features to predict video quality Visual Information Fidelity VIF considers information fidelity loss at four different spatial scales Detail Loss Metric DLM 8 measures loss of details and impairments which distract viewer attention Mean Co Located Pixel Difference MCPD measures temporal difference between frames on the luminance component The above features are fused using a SVM based regression to provide a single output score in the range of 0 100 per video frame with 100 being quality identical to the reference video These scores are then temporally pooled over the entire video sequence using the arithmetic mean to provide an overall differential mean opinion score DMOS Due to the public availability of the training source code VMAF Development Kit VDK the fusion method can be re trained and evaluated based on different video datasets and features Anti noise signal to noise ratio AN SNR was used in earlier versions of VMAF as a quality metric but was subsequently abandoned 9 Performance editAn early version of VMAF has been shown to outperform other image and video quality metrics such as SSIM PSNR HVS and VQM VFD on three of four datasets in terms of prediction accuracy when compared to subjective ratings 4 Its performance has also been analyzed in another paper which found that VMAF did not perform better than SSIM and MS SSIM on a video dataset 10 In 2017 engineers from RealNetworks reported good reproducibility of Netflix performance findings 11 In MSU video quality metrics benchmark where its various versions including VMAF NEG were tested VMAF outperformed all other metrics on all compression standards H 265 VP9 AV1 VVC VMAF scores can be artificially increased without improving perceived quality by applying various operations before or after distorting the video sometimes without impacting the popular PSNR metric 12 13 Software editA reference implementation written in C and Python VMAF Development Kit VDK is published as free software under the terms of BSD Patent license 14 Its source code and additional material are available on GitHub 5 See also editPerceptual Evaluation of Video Quality PEVQ VQuad HDReferences edit Liu Tsung Jung Lin Joe Yuchieh Lin Weisi Kuo C C Jay 2013 Visual quality assessment recent developments coding applications and future trends APSIPA Transactions on Signal and Information Processing 2 doi 10 1017 atsip 2013 5 hdl 10356 106287 ISSN 2048 7703 Lin Joe Yuchieh Liu T J Wu E C H Kuo C C J December 2014 A fusion based video quality assessment Fvqa index Signal and Information Processing Association Annual Summit and Conference APSIPA 2014 Asia Pacific pp 1 5 doi 10 1109 apsipa 2014 7041705 ISBN 978 6 1636 1823 8 S2CID 7742774 Lin Joe Yuchieh Wu Chi Hao Ioannis Katsavounidis Li Zhi Aaron Anne Kuo C C Jay June 2015 EVQA An ensemble learning based video quality assessment index 2015 IEEE International Conference on Multimedia amp Expo Workshops ICMEW pp 1 5 doi 10 1109 ICMEW 2015 7169760 ISBN 978 1 4799 7079 7 S2CID 6996075 a b Blog Netflix Technology 2016 06 06 Toward A Practical Perceptual Video Quality Metric Netflix TechBlog Retrieved 2017 07 15 a b vmaf Perceptual video quality assessment based on multi method fusion Netflix Inc 2017 07 14 retrieved 2017 07 15 72nd Annual Technology amp Engineering Emmy Awards Recipients The Emmys theemmys tv Retrieved 2021 02 08 PATRON Julien Technologie l Universite de Nantes recompensee d un Emmy Award UNNEWS in French Retrieved 2021 02 08 Li S Zhang F Ma L Ngan K N October 2011 Image Quality Assessment by Separately Evaluating Detail Losses and Additive Impairments IEEE Transactions on Multimedia 13 5 935 949 doi 10 1109 tmm 2011 2152382 ISSN 1520 9210 S2CID 8618041 Zhili Henry Removal of AN SNR Github Bampis Christos G Bovik Alan C 2017 03 02 Learning to Predict Streaming Video QoE Distortions Rebuffering and Memory arXiv 1703 00633 cs MM Rassool Reza 2017 VMAF reproducibility Validating a perceptual practical video quality metric PDF 2017 IEEE International Symposium on Broadband Multimedia Systems and Broadcasting BMSB pp 1 2 doi 10 1109 BMSB 2017 7986143 ISBN 978 1 5090 4937 0 S2CID 5449498 Retrieved 2017 11 30 Zvezdakova Anastasia Zvezdakov Sergey Kulikov Dmitriy Vatolin Dmitriy 2019 04 29 Hacking VMAF with Video Color and Contrast Distortion arXiv 2107 04510 cs MM Siniukov Maksim Antsiferova Anastasia Kulikov Dmitriy Vatolin Dmitriy 2021 Hacking VMAF and VMAF NEG vulnerability to different preprocessing methods arXiv 2107 04510 cs MM BSD Patent Open Source Initiative 18 May 2017 External links editReference implementation Retrieved from https en wikipedia org w index php title Video Multimethod Assessment Fusion amp oldid 1194920606, wikipedia, wiki, book, books, library,

article

, read, download, free, free download, mp3, video, mp4, 3gp, jpg, jpeg, gif, png, picture, music, song, movie, book, game, games.