Ah, Halloween. A time for Lycra and seasonal photo shoots and arguing about the comparative merits of different candies.
Have you been mad online about the above map this week? If you have, you’re not alone. It was debated on the Today show. “This Controversial Map of State-by-State Halloween Candy Favorites Is Dividing Our Nation,” chirped Time. “A Survey of States’ Favorite Halloween Candies Shows Something Is Seriously Wrong on the West Coast,” wrote Digg. On Twitter, people raged. What’s wrong with you, New Hampshire? Who the hell is giving out Toblerone? And candy corn — fucking candy corn! It’s disgusting, and apparently the most popular Halloween candy in the country.
Friends, I am here to tell you this map is bullshit. It is bullshit in so many ways, and we are going to count them, and we are going to miss some because there are so many to get through, and it will still be OK, because even discussing just a fraction of the bullshittery will prevent you from ever having to defend California’s Life Savers consumption again.
Let’s start with the most obvious problem: The numbers are more than a year old! Influenster, the company behind the “study,” first published the map a year ago — hence the big ol’ 2015 at the top. It prompted a similar wave of fury the first time around: “The Map of Favorite Halloween Candy by State Is a Downright Travesty to the U.S. of A.,” decreed Barstool.
Influenster, for its part, seems content to let the confusion stand: It republished the numbers last month in a blog post titled “America’s Favorite Halloween Candy State by State,” presenting year-old data as brand-new. Asked about this, an Influenster spokesperson said, “The 2015 numbers are stable and people’s favorite Halloween candies aren’t going to change over just one year for those numbers to be statistically significant.”
So what exactly is Influenster? It pitches itself as a “product discovery platform,” which means, essentially, it’s a Yelp for everything. The idea is not merely to review the book you just bought or the headphones you really like, but rather every single product you have ever encountered in your entire life. Spectrum Coconut Oil Organic? 754 reviews. Boboli Pizza Crust Italian Mini 8-Inch? 4.6 stars. Elmer’s Liquid Glue? “This glue has always been my favorite.” Maruchan Ramen Noodle Soup Roast Chicken Flavor? “I always make sure to have this in our pantry.”
Users (it’s hard to get exact numbers, but Influenster claimed to have more than 1.7 million users in April) are wooed with periodic boxes of free samples; myriad blog posts are devoted to instructing readers on how to preen their profiles on the site to increase their chances of getting one. The key, mostly, is to surrender your social media accounts, spread the gospel of whichever brands come knocking, and leave product reviews. Lots and lots of reviews.
So let’s return to that Halloween candy map, which claims to have an impressive 42,238 responses, and uses fancy statistical language like “margin of sampling error.” Because I hate fun and also Life Savers, I asked Influenster to explain its methodology. This was the response: “We aggregated the top-rated candy products from Influenster.com based on product ratings and frequency of the word ‘Halloween’ mentioned in reviews, and then conducted an online survey giving them a list of Halloween candies to choose from.”
What? WHAT?! This is (deep breath) so deeply flawed for a (deep breath) multitude of reasons. What happens if someone uses the word “Halloween” more than once? Would that count for each mention? Did someone in Arizona publish a review with “Halloween” written 60 times and “Toblerone” stuck in there once for good measure? Users are frequently reviewing multiple candies, so are their elections being double- (or triple- or more) counted?
More importantly, this explains the victory of candy corn, and the extent to which it’s nonsense. Unlike every other candy on the map, candy corn is sold more or less exclusively as a Halloween treat — so of course reviews are likely to mention the holiday. Candy corn sales around Halloween are presumably many times what they are during the rest of the year — whereas Reese’s, which sells year-round, probably only sees an incremental bump.
And then we come to the bigger question of the sanctity of Influenster’s reviews. Stop me if you’ve heard this one before, but sites that incentivize users to leave as many reviews as possible do not always get ones that are of very high quality. An overwhelming majority of Influenster’s reviews appear to be four or five stars — no great shock considering many are written with the goal of receiving samples in exchange. Still others seem to be the work of spambots, posting near-identical reviews months apart from different accounts:
You know what’s a better way of looking at candy popularity than paid, spammy reviews? Here’s one: sales. CandyStore.com, which is pretty much what it says, created its own state-by-state map using sales from the three months leading up to Halloween over the eight-year period of 2007 to 2015. Its conclusions had little in common with the Influenster map:
But if this map is also making you raise your eyebrows (saltwater taffy in California? Hot Tamales in Indiana?) good. The fact that you’ve never heard of CandyStore.com (sorry!) is indicative of the fact that it is a relatively niche vendor — one people might turn to for retro or otherwise uncommon sweets like Dubble Bubble (sup, Montana?).
Most Americans buy their Halloween candy at grocery stores, drug stores, and big-box retail outlets like Walmart, and the truth is that the reality, probably, is much more boring than either of these maps suggests: The vast majority of states probably consume just a handful of the map’s suggested most popular candies. A realistic map would more than likely show little variation from heavyweights like Hershey’s, Reese’s, and the like — it’s just very unlikely that candies with comparatively small market shares, like Airheads, could find their way to the top with any frequency. But wouldn’t that be boring?
So here we are. That map you’ve been so mad about: It says nothing about you or your state or your childhood. All it shows is what reminded a bunch of shady, heavily incentivized users of Halloween.
