Are streaming providers delivering their A-game when it comes to the video quality of the most-watched sports event of the year?
While the world was counting the goals in the first World Cup match of Qatar vs. Ecuador, our virtual eyes were scoring the perceptual video quality. The result: all three streaming providers scored below a SSIMPLUS Viewer Score of 80, the minimum recommended level for delivering such important world sports events at an acceptable level.
The three providers we monitored for the opening game of Qatar vs. Ecuador delivered with Viewer Scores of 78.1, 71.1, and 73.7. The last score is for YouTube TV (read our detailed analysis of YouTube TV’s sports streaming performance here). That’s subpar for any event, but especially for a sporting event that draws more viewers than any other, including the Olympics. Fortune predicts that the 2022 World Cup will attract 5 billion viewers worldwide.
Soccer/football can achieve high scores: Providers delivered a Viewer Score of 80 or above for the English Premier League matches earlier this year (read our detailed video quality analysis). For example, the September 4th match of Manchester United vs. Arsenal had the following Viewer Scores from 4 different providers: 83.3, 80.4, 74.5 and 76.5.
What is the main reason behind these lower video quality scores for the World Cup game? While monitoring the first game, we saw many source quality issues where the camera is out-of-focus which causes the picture to be very blurry. This has a big impact on the perceptual viewer experience, especially for the key moments. For example, the image below was captured right after the second goal with a Viewer Score of 67.
Download the captured frames in high resolution (zip file).
Figure 1: With a Viewer Score of 67, this is a far cry from what good quality looks like. The best streaming provider for the English Premier League delivers soccer matches at 80 and above. – Click for higher resolution.
Figure 2: This frame, with a Viewer Score of 59, has a lot of compression artifacts such as blurriness and grass detail loss – it looks as if the teams are playing on a carpet instead of a soccer field. There are also ringing artifacts – ripples surrounding the player on the far left. Click for higher resolution.
Another issue is the encoding quality: overall the viewer experience is low, primarily due to blurred pictures and loss of details. No grass texture is present and sometimes faces are not recognizable, even in “simple” scenes.
The two images above show encoding issues. These two images have Viewer Scores of 65 and 69 respectively.
Apologies, our website doesn’t support a moving slider – but if you go here, there is a version of these graphics that does allow you to move the sliders.
The two images above are taken from two different providers who are delivering the same game from the World Cup. All the FIFA World Cup games are available across several streaming services in the USA, so soccer fans can choose the service that delivers the best quality. We clearly see that Image 1, with a Viewer Score of 69, is very blurry, with a lot of detail loss on the grass and objects, like the numbers on the jerseys. Image 2 is crisper, with an image quality of 87. For example, compare the numbers on the back of the number 17 jersey. We recommend that you download the images and open them up on a 55” TV to replicate the actual viewer experience.
One of the reasons for the lower scores for the World Cup could be the high cost of network delivery in and from Qatar. A representative from SVT Sports, a Nordic broadcaster, said that: “The main challenges have been how to get the FIFA feeds from Qatar because of the high costs involved.”
For an important event like the World Cup with billions of eyeballs following the games your broadcasting/streaming engineering teams need to answer the most pressing question: “How do you know your viewers are receiving the best quality using the least amount of bits.”
SSIMWAVE’s Emmy award-winning video automation solutions enable your teams to know the answer and to ensure that live sports (especially UHD quality), that are exceptionally tricky to get right, are delivered with Viewer Scores of 80 and above at the lowest possible cost to keep soccer fans happy without breaking the bank.
Analyst Comment
I have been watching SSIMWAVE for a number of years as I really appreciate the firm’s efforts to bring numbers and better measurement to video quality, and I appreciate the firm’s collaboration in allowing us to reproduce this blog article. Personally, I have access to the BBC’s UltraHD HLG iPlayer coverage of the World Cup and the performance, so far, has been excellent. I have a Virgin Media cable broadband connection and Virgin have a bit more control over their network than most ISPs, but the result, so far, has been completely glitch-free and with good quality. We’ll see if that persists as audiences get even bigger as England move into the knockout phase of the competition. (BR)
