Prof. Nick Feamster’s Research Drives Wall Street Journal Investigation of Streaming Video

Over three-quarters of today’s internet traffic comes from streaming video, a number that is only projected to rise over time. To meet this demand, internet service providers offer consumers faster data speeds at premium prices, with gigabit-per-second tiers available in some areas. But do these pricier plans actually improve the quality of video streaming? A Wall Street Journal investigation, published August 20th, answered this question with the help of UChicago Professor Nick Feamster, in a collaboration that both informs consumers and advances science.

The story, “The Truth About Faster Internet: It’s Not Worth It,” began three years ago with what the reporters thought would be a simple question: do faster speeds matter for streaming video? To answer it, they turned to Feamster, then at Princeton University, for his expertise on networked computer systems. Feamster’s group had developed software systems that could reliably measure the actual internet speeds consumers received at home, one important part of the WSJ inquiry. But gathering the other half of the data — the performance of streaming video services in those homes — would require a mix of systems, machine learning, and recruitment.

While video providers such as Netflix, Amazon, and YouTube can collect data on the quality that users receive through their software, ISPs and outside researchers are in the dark. With the help of the Wall Street Journal, the researchers recruited more than 60 households, collected information about their internet service, and installed a monitor of the data passing through their network. But challenges remained. 

“We're basically looking at nonsense traffic; it's all encrypted, we can't see the contents of any of it, and we somehow want to know: Is the user streaming a Netflix video, and if so, what's the resolution? And, how long did it take to start playing?,” said Feamster, Neubauer Professor of Computer Science and Faculty Director of the Center for Data and Computing. “That's an interesting and very challenging machine learning inference problem. We see a bunch of encrypted data, and we’re trying to figure out the quality of this video—not just ‘speed’, but the quality of the user’s actual experience.”

The work led to a research paper and a new tool called Net Microscope, which infers video streaming quality metrics such as startup delay and resolution in real time from the encrypted data stream. By gathering data from more than 200,000 video sessions from the volunteer homes, the team trained a model that that can look at encrypted data and identify which streams are from Netflix, YouTube, Amazon, and Twitch, as well as the quality that end users experience.

From there, the researchers could finally answer the question posed by the Wall Street Journal: How do these services perform at different internet speeds? The study found that streaming video performance plateaus long before the upper tier plans offered by ISPs, with plans above 100 megabits-per-second only marginally improving startup delays and resolution, even when multiple devices are viewing videos simultaneously.

That’s valuable information for consumers, who might feel compelled to upgrade to a more expensive plan if they’re unsatisfied with their streaming video quality at home. But it’s also useful information for ISPs, Feamster said, who can better help their customers find the true cause of unsatisfactory performance instead of just recommending faster service.

“Anything we do that can help basically shed more light into that question from consumers can also ultimately help the operation of the network itself,” Feamster said. “So it kind of goes both ways.”

The project fits Feamster’s broader research focus on the performance and security of communications networks, which encompasses work on Internet of Things technologies, censorship and information control over online platforms, and policy questions such as net neutrality and broadband access. Like the Wall Street Journal project, many of these research areas and policy issues require creating new software and systems that can collect data and measure performance in the real world. From the lens of his new role at the Center for Data and Computing, Feamster sees this project as the beginning of what he hopes will be more work at the intersection of data science, public policy and investigative journalism.

“Addressing policy problems depends on having access to good data, because what is needed to inform the debate is accurate information about what's actually going on,” Feamster said. “These datasets generally don't exist, there's no tranche of speed test data that gets dropped in our lap that could answer these questions. We have to design the method and build the system to gather data that nobody else has…then we can provide an answer.”

Related News

More UChicago CS stories from this research area.
UChicago CS News

NeurIPS 2023 Award-winning paper by DSI Faculty Bo Li, DecodingTrust, provides a comprehensive framework for assessing trustworthiness of GPT models

Feb 01, 2024

“Machine Learning Foundations Accelerate Innovation and Promote Trustworthiness” by Rebecca Willett

Jan 26, 2024

Nightshade: Data Poisoning to Fight Generative AI with Ben Zhao

Jan 23, 2024
UChicago CS News

UChicago Undergrad Analyzes Machine Learning Models Used By CPD, Uncovers Lack of Transparency About Data Usage

Oct 31, 2023
UChicago CS News

Research Suggests That Privacy and Security Protection Fell To The Wayside During Remote Learning

A qualitative research study conducted by faculty and students at the University of Chicago and University of Maryland revealed key...
Oct 18, 2023
UChicago CS News

Five UChicago CS students named to Siebel Scholars Class of 2024

Oct 02, 2023
UChicago CS News

UChicago Researchers Win Internet Defense Prize and Distinguished Paper Awards at USENIX Security

Sep 05, 2023
In the News

In The News: U.N. Officials Urge Regulation of Artificial Intelligence

"Security Council members said they feared that a new technology might prove a major threat to world peace."
Jul 27, 2023
UChicago CS News

UChicago Computer Scientists Bring in Generative Neural Networks to Stop Real-Time Video From Lagging

Jun 29, 2023
UChicago CS News

UChicago Team Wins The NIH Long COVID Computational Challenge

Jun 28, 2023
UChicago CS News

UChicago Assistant Professor Raul Castro Fernandez Receives 2023 ACM SIGMOD Test-of-Time Award

Jun 27, 2023
Michael Franklin
UChicago CS News

Mike Franklin, Dan Nicolae Receive 2023 Arthur L. Kelly Faculty Prize

Jun 02, 2023
arrow-down-largearrow-left-largearrow-right-large-greyarrow-right-large-yellowarrow-right-largearrow-right-smallbutton-arrowclosedocumentfacebookfacet-arrow-down-whitefacet-arrow-downPage 1CheckedCheckedicon-apple-t5backgroundLayer 1icon-google-t5icon-office365-t5icon-outlook-t5backgroundLayer 1icon-outlookcom-t5backgroundLayer 1icon-yahoo-t5backgroundLayer 1internal-yellowinternalintranetlinkedinlinkoutpauseplaypresentationsearch-bluesearchshareslider-arrow-nextslider-arrow-prevtwittervideoyoutube