Online ratings make the modern world go round. We consult Yelp before making restaurant reservations, we skim Amazon reviews before buying light bulbs, and when someone suggests we watch a movie or a show, we inevitably end up on Rotten Tomatoes or IMDb to see if it’s really worth our time. These systems work until they don’t. Not every waitress at every restaurant has an “attitude problem,” as most bitter Yelp reviewers might have you believe. Five stars on Amazon will not guarantee that your new teddy bear will be appropriately proportioned. And the best television show of all time is probably not a retelling of a real-life nuclear disaster featuring distracting British accents.
Last week, HBO’s Chernobyl shot to the top of IMDb’s all-time TV rankings, outperforming other mega-popular hits like Breaking Bad, Game of Thrones, and various stoner-friendly seasons of Planet Earth. And as of Tuesday, it had a 9.6-star (out of 10) average rating from more than 200,000 users on the Amazon-owned entertainment site. To the knee-jerk press, the limited series’ ascension was evidence of a historic hit. The Economist ran with the numbers, comparing them to traffic spikes on the “Chernobyl nuclear disaster” Wikipedia page, declaring the show “the highest-rated TV series ever,” and marveling at the reach of its subject matter.
“Documentaries typically struggle to gain as much attention as action and fantasy shows such as Game of Thrones, and the same is true of historical dramas,” The Economist wrote. “Even series about well-known subjects, such as Queen Elizabeth II or the trial of O.J. Simpson, earn user ratings that are typically below those of the most popular shows. In comparison, Chernobyl scores better than the greats.”
It’s a compelling narrative that lets you fill in the blanks. A society plagued with worries of environmental destruction and dogmatic leadership finds meaning in a series meant to encapsulate both. Forget the tits and dragons, forget the meth-making chem teacher—the people want history! But while Chernobyl is extremely compelling television, the unblinking enthusiasm with which outlets have embraced these rankings—both to sing the praises of the series and to declare it more well-liked than an obvious cultural phenomenon—is less a testament to its quality than it is the flaws of our pop culture rating ecosystem.
As one of the 50 most visited sites in the world, IMDb wields considerable power over the entertainment industry. And, from the company’s very early days, its role as an all-purpose indexing site has been exploited to influence releases both big and small. At times, that has meant studios massaging the information on their films’ pages to purposefully shape coverage. (In 1999, Artisan sought to market The Blair Witch Project by arranging for the IMDb pages of its actors to read “missing, presumed dead.”) Today, that manipulation is more often coming from online communities. As Wired wrote in 2016, IMDb’s “voter system has increasingly become a soft weapon in all sorts of online turf wars.” And the subject of these turf wars ranges from geopolitical disputes to plain sexism. When Angelina Jolie’s 2011 directorial debut, In the Land of Blood and Honey, made its way to international audiences, it was overwhelmed with 1/10 votes, likely due to its disputed depiction of the Bosnian War. (Current rating: 4.4/10.) Bangladeshi users launched a similar attack on the hit 2014 Bollywood film Gunday, after being angered by its sloppy depictions of 1971’s Bangladesh Liberation War. (2.2/10.) The 2016 remake of Ghostbusters was sabotaged by a faction of fans who appeared to be upset by its all-female cast. (5.2/10.) This kind of deliberate trolling also appears to be the reason the website shut down its message boards in 2017, tragically erasing 16 years’ worth of conversations between movie buffs to duck the responsibility of moderation.
Similar issues have plagued its competitor, Rotten Tomatoes. In 2017, ratings for The Last Jedi dipped to a dismal 44 percent on the site, while its “Critics Consensus” from verified reviewers sat at 91 percent. Many outlets theorized that this was due to a targeted campaign against the movie, headed by longtime Star Wars fans who felt it had betrayed the franchise. Brie Larson’s remake of Captain Marvel was also preemptively “review-bombed” this year, presumably because it featured a woman in the role of a typically male hero. The site ultimately axed its anticipatory “want to see” scores before the film’s release to prevent further abuse of the system. “We’re doing it to more accurately and authentically represent the voice of fans, while protecting our data and public forums from bad actors,” the site said in a blog post.
Though the most public instances of IMDb-rating sabotage center on film-related controversies, plenty of other movies have been swallowed or buttressed by the ranking system without any apparent motive. In 2016, the creators of the indie film Kicks were surprised to discover that a wave of negative reviews from IMDb users with no biographical data had suddenly lowered what was initially a promising rating, and suspected it was due to vote brigading. Conversely, IMDb’s Top 250 list, which ranks the site’s highest rated movies of all time, is itself living evidence that so-so projects can get ahead as long as they resonate with the site’s most active users. The top film on the list is The Shawshank Redemption; Avengers: Endgame is ninth; Citizen Kane ranks 120th; When Harry Met Sally and Moonlight, for comparison, don’t even make the list.
The company is predictably secretive about how it determines its scores, but we do know a few things about how its ranking system is designed. The score of a movie does not reflect the mean of all user votes—rather, “various filters are applied to the raw data in order to eliminate and reduce attempts at vote stuffing by people more interested in changing the current rating of a movie than giving their true opinion of it,” according to the IMDb website. But in its top ranking lists, the platform places more value on the votes of regular users than those of newbies. And a 2017 report in Mel Magazine found that those regular users often skew to an overwhelmingly male international audience between the ages of 18 and 29.
With this information, a pattern emerges. The internet’s go-to movie database is being shaped by a highly specific group of people with highly specific opinions and, quite frankly, a lot of free time. This problem is not unique to IMDb. Reference sites like Wikipedia are still struggling to diversify their voluntary staff which is overwhelmingly made up of white, male editors. Nor is this unique to online communities. The Academy Awards’ lack of diversity in both its voting body and award winners has been a recurring topic of conversation for the past few years. That Chernobyl has flourished under this system is not a surprise: It’s an excellent TV series, but also one that happens to have an all-white, mostly male cast, and a story line that appeals to exactly the kind of regular voters that IMDb prioritizes.
For the record, there is nothing wrong with certain demographics loving a certain set of movies that cater to their sensibilities. As a white woman raised in California who loves Lady Bird, I can relate! But IMDb’s influence is particularly insidious because the opinions of a very specific group of people are presented as empirical data: foolproof numbers that follow a title wherever it goes on the internet. At this point, we should all know better than to blindly take that information as fact—whether we enjoy pale actors muttering Britishisms in concrete rooms or not.