Netflix keeps the ratings for its shows a closely guarded secret, primarily to maintain a strategic advantage over competitors. If Amazon, for instance, knew that Netflix’s top two shows were, say, Fuller House and Stranger Things, that would certainly help inform the type of original programming Amazon would seek to develop. What’s more, Netflix’s chief content officer Ted Sarandos has said that releasing ratings data would be counterproductive because some content is released niche audiences.

“There is a very natural inclination to say, ‘Relative to this show, this show is a failure.'” Sarandos explained last year. “That puts a lot of creative pressure on the talent.”

In light of Netflix’s secrecy with respect to show ratings, third-party companies have come up with roundabout ways to figure out what Netflix’s massive 109 million subscribers tend to watch most often. Most recently, Nielsen began boasting about a new service that relies upon audio data to figure out what shows and episodes Netflix subscribers are watching.

Here’s how it all works. Nielsen’s technology is essentially an opt-in service that monitors audio when users are watching Netflix on their HDTV via a set-top box. The statistical problems with this approach are numerous. For starters, Nielsen’s technology — as an opt-in service — is prone to self-selection bias. Imagine, for example, that the demographic of users likely to agree to use Nielsen’s technology are sci-fi enthusiasts. This would skew Nielsen’s ratings towards programs like Stranger Things and perhaps downplay the popularity of crime thrillers like Mindhunter.

More importantly, Nielsen’s technology doesn’t incorporate any data from content watched via mobile devices like smartphones and tablets.

Read More Here