NFL

The NFL’s Analytics Revolution Has Arrived

Football is still well behind baseball and basketball when it comes to embracing advanced metrics, but teams have made significant progress in recent years. Those who do not adapt will be left behind.

NFL

The NFL’s Analytics Revolution Has Arrived

Football is still well behind baseball and basketball when it comes to embracing advanced metrics, but teams have made significant progress in recent years. Those who do not adapt will be left behind.

By Kevin ClarkDec. 19, 2018, 5:08 pm UTC • 19 min

The football analytics revolution may not be obvious, but it is happening in front of you all the time. There is an NFL team that plans to run more offensive plays to the side of the field farthest from its opponent’s bench. It has figured out, using player-tracking data, that a defensive lineman will sometimes run more throughout the course of a game by shuffling from the bench to the field during a substitution than he will during actual gameplay. Thus, running plays to the far side of the field can help tire out rotating defensive linemen.

This strategy is unique but the logic behind it is not. Stories like this are common around the league: A team stumbles upon some shred of data and builds a play, a playbook, a personnel decision, or an entire scheme around it. It changes how a team drafts, calls plays, and evaluates opponents. All of these trends point to one thing: Football’s analytics moment has arrived.

We’ve reached this high point for a couple of reasons. The rise of smarter, younger GMs and coaches is part of it. A bigger part of it, though, is the spread of the NFL’s player-tracking data, which is being shared leaguewide for the first time this season. Having access to that data allows teams to build models to analyze plays and players differently, and to simply know more about the game. That’s been a boon to a movement that had already been embraced by a handful of the smartest teams. As other teams try to catch up, they’ve created an arms race to get the best numbers. Essentially, the smartest teams are getting significantly smarter, the average teams are trying to get better, and the dumbest teams are going to be very dumb if they don’t act soon.  

“It’s about translating that data ASAP and being very, very in tune with the numbers. You can’t be a year behind, you can’t be a month behind,” said Falcons general manager Thomas Dimitroff.

Teams are examining details they’ve never studied before to get an edge. One scouting department graded a defensive back prospect as an undrafted free agent due in part to his slow 40-yard dash. When that department was able to measure his game speed using tracking data, it determined that it should have listed him as a midround pick. Other teams in the market for linebackers have homed in on what kind of closing speed elite tacklers need. For instance, Zebra Technologies, a company whose MotionWorks service collects game-day data, found that the Cowboys’ Leighton Vander Esch reaches 16 to 17 miles per hour on his best plays.

“Teams can go deep on rosters to leverage the tracking data to scout players for the future, maybe in free agency,” said John Pollard, vice president of Zebra Technologies.

The new information can also help teams simplify play-calling, like going for it on fourth down more often. The Eagles won the Super Bowl in part because of their aggressiveness in such situations—and in part because they went for two-point conversions when it was mathematically smart to do so. Analytics are not new to football, but this depth of knowledge is. The Eagles have had an analytics department for nearly two decades. “We confirmed,” said Joe Banner, the former Eagles president who helped set up the department, “that there’s a competitive advantage in analytics in a league that is structured to prevent you from having a competitive advantage.”

The Patriots have incorporated some level of analytics for years. (Multiple people joked to me that the movement would have taken hold leaguewide sooner if the Patriots ever discussed their efforts publicly.) Earlier this decade, the Jacksonville Jaguars launched one of the NFL’s most notable analytics departments under executive Tony Khan. When former Jaguars general counsel Sashi Brown joined the Browns front office, he employed one of the most forward-thinking, analytics-based approaches in the league. He was fired last year during Cleveland’s 0-16 season, but is credited with helping the franchise accumulate draft picks and young talent. Brown’s successor, John Dorsey, said he’ll continue to use analytics. The Vikings practice facility features a massive analytics hub, and general manager Rick Spielman told me he uses the numbers often. Because of the secretive nature of the sport, it’s impossible to tell what all 32 teams are doing, but the feeling is that just about everyone is in on analytics, which wasn’t the case until very recently.

During the 2018 draft, Giants GM Dave Gettleman mocked those who used analytics to question the team’s decision to take a running back with the second overall pick. Football will likely never be baseball, where statistics can basically explain anything. There are too few games and too many variables. But there’s always been a lot of room for more data. “There’s been a shift and I think it’s a lot more recent than people think,” said Neil Hornsby, the founder of Pro Football Focus, one of the sport’s top statistical services. “There are very, very few teams that aren’t doing at least some sensible things with analytics. I would say even two years ago there were still a lot of teams paying lip service.” Hornsby said he thinks at least a handful of teams employed an analytics department simply to say they had one in case an owner asked about it, even if it didn’t have any influence.

Matt Swensson, the NFL’s vice president of emerging products and technology, said Next Gen Stats—the NFL’s advanced player-tracking data service—became a full-fledged initiative around five years ago. The data had a limited rollout at the start, and this year is the first time teams have been able to get leaguewide data. Next Gen Stats collects information from chips placed in players’ shoulder pads that reveal their location at all times; the chip can track how fast a player is moving, whether he is sprinting or jogging, the average separation between an offensive player and his defender, and many other metrics. Some of this data is public—skill-position players’ speed, for instance, is published on an NFL website—but the vast majority is available only to teams, who are creating other proprietary stats from the raw numbers. NFL franchises are absorbing millions of data points for the first time, and many football lifers are getting their first deep experience with analytics.

The NFL’s relationship with numbers—and in some cases, with smart people—has been complicated, to say the least. Two years ago, I reported on infighting within the league over the release of data and the use of technology. Some teams didn’t want the league to produce automated maps of their route trees because they thought writing them out by hand was a good dues-paying process for younger coaches. Now, there’s no denying the technology and the data are here, and wins and losses are at stake.

“The point we made with our coaches is: We have all this information but so does everyone else,” said Steelers general manager Kevin Colbert. “What advantage does it give us to get it? None. It’s what we do with it, the way we use it. It’s no different than when we go sit at the combine and get all the same information. It’s about finding an advantage in what we do with it.”

Colbert is right, and the race is on to figure out how to use the numbers better than everyone else.

“It is amazing,” Warren Sharp said, “how many teams anonymously follow me on Twitter.” Sharp is an engineer with his own analytics site and has been playing around with football statistics for about 20 years. He is among the top minds in football not working full time for a team. In fact, when you talk to people inside the league, some think he might be the top mind, period. Though he’s been writing on the internet for many years, he said it wasn’t until 2018 that teams started reaching out to him to discuss analytics. He says he’s heard from at least five and has done work as a consultant.

It makes sense that teams have become interested in outsiders like Sharp. Unlike other sports, which have staffed up their analytics departments over the past decade or so, football doesn’t employ many of the best analytics minds studying the game. (One team decision-maker sarcastically said this is because analytics employees inside NFL buildings are too beaten down from having their ideas ignored by old-school coaches.) Sharp is in high demand because he can help answer a question vexing front offices: Which stats matter? He became interested in how teams win games when he was in college and became convinced of two things, both of which would foreshadow the modern NFL. The first is that an offensive emphasis on passing correlated to wins. The second is more complicated than it sounds. Sharp found that third-down efficiency, long the obsession of announcers and old-school coaches, was not the key to an effective offense. He found that it was better for teams to scrap third downs entirely and move the chains by gaining the necessary yardage on first and second down.

“Announcers love to say, ‘This team is 10-of-13 on third down,’ and there’s never any comparison to early-down success,” Sharp said. In his view, teams should run the ball on first down much less than they do. This revelation came to him in the late 2000s, as he watched quarterbacks like Drew Brees, Peyton Manning, and Tom Brady at their peaks.  

“I don’t think their strategy was to avoid third down, but I think there was just more aggressiveness. It’s about calling efficient plays,” Sharp said. “You always hear TV announcers, it drives me crazy, they’ll say on second-and-short, ‘Good time to take a shot down the field,’ and there’s a lot of risk in that. The interception rate is higher than it is on a regular play, the success rate converting it is low. Run a play that gets you the first down and take a shot on the next first-and-10 if you want.”

There is some evidence that teams are coming around to Sharp’s ideas about early-down aggressiveness. The leaguewide yards per attempt average on first down is 7.4, up from 7.1 in 2017. On second down, it’s 7, up from 6.5 last year. On third down, yards per attempt has actually dipped slightly, from 6.2 in 2017 to 6.0 now. This season, the Rams and Chiefs lead the NFL in first downs, yet are 26th and 31st, respectively, in third-down attempts. Sharp pointed to the Chargers as a team that did not pass enough on first down and did not call efficient plays on early downs last year. This season, Philip Rivers’s yards per attempt average on first down has climbed from 8.1 to 9.7; the 11-3 Chargers are enjoying their best season in years.

Hornsby, the Pro Football Focus founder, said he is getting more queries from NFL teams on how to utilize wins above replacement, a statistic that spells out the value of a player. That metric listed Saints receiver Michael Thomas as the most valuable non-quarterback in the league last season largely because of his ability to get first downs. Hornsby said this year the metric ranks Seahawks linebacker Bobby Wagner ahead of Rams star Aaron Donald because pass coverage carries more weight than pass rush, as offenses can compensate for the latter. As an example, Hornsby points to when Donald was single-teamed against the Bears in Week 14: Chicago took an average of 1.5 seconds to get rid of the ball, thus making it next to impossible for Donald to wreck a play in his usual way.

Hornsby, like Sharp, said teams are finally figuring out the value of never giving the ball up. In baseball, analytics staffs had to work to convince managers that you shouldn’t give up outs via sacrifice bunts, since there are only 27 outs for each team in a game. In the NFL, this logic applies to punting. Teams this season average between 10 and 13 drives per game, and 11 teams have fourth-down conversion rates of at least 60 percent. Of the six teams with conversion rates of at least 70 percent, five are locks for the playoffs. The Chiefs have converted 90.9 percent of their fourth downs. The lesson: Do everything you can to keep the ball.

This has long made sense to Carolina coach Ron Rivera, who in 2013 earned the “Riverboat Ron” moniker following a string of aggressive fourth-down decisions. He told me that he’d met a banker at an awards dinner earlier that year who specialized in analytics. At Rivera’s request, the banker compiled numbers on fourth-down conversion rates, which Rivera still references. That same year, Rivera said, he became interested in The New York Times’ “Fourth Down Bot,” which indicated when a team should go for it.

Still, Rivera admits that it was initially hard to figure out what to do with all the information that player-tracking technology provided. We spoke earlier this season about the Panthers’ run to the Super Bowl following the 2015 season, a game they lost to Peyton Manning and the Broncos. “That was the year we were trying to dive into the analytics,” Rivera said. The Panthers were getting daily practice readings on players’ energy levels, but Rivera said he couldn’t quite contextualize them until the following spring. I asked him whether there was anything he could have done differently. “Probably, probably a lot of things we could’ve done differently,” he said, pointing to optimizing practice and meeting times. Now, he said, the team uses monitoring devices to see how much players’ core temperatures go down during their mandated two-minute water break in practice. Player health monitoring is a massive part of the league’s data revolution. “We’ve learned that if a player is going 6,000 yards every day at practice it’s like constantly driving a car at 1,000 miles per hour—something’s gonna blow, so slow down,” Titans general manager Jon Robinson told me.

Figuring out what matters in the data is a hot topic in front offices. Swensson said there are more than 100 data points given to teams, “so all of it is a little mind-numbing.” Dimitroff told me he’s monitoring pass defense closely, as he’s long thought there were better ways to measure defensive back play. Next Gen Stats can now track how close a defensive back is to a receiver on a given play.

“I think now every team has some level of personnel support to manage the data itself,” said Pollard of Zebra Technologies. “How teams interpret that data still varies, but there’s not a club I’ve interacted with that’s not using a deeper level of analytics, that isn’t building predictive models.”

“When they write the history of all of this, it will start with Virgil Carter,” said Brian Burke, ESPN’s senior analytics specialist. “It’s amazing. Just imagine if Bill James were a pitcher for the Cincinnati Reds.”  

Carter is a driving force in two separate football revolutions. In 1970, the Cincinnati Bengals turned to Carter, the backup quarterback, after their starter went down with an injury. The coaching staff changed its offensive strategy to play to Carter’s strengths, using shorter passes and rollouts to compensate for his lack of arm strength. The Bengals won seven straight games to end the regular season, and in 1971 Carter led the NFL in completion percentage. The quarterbacks coach on that team was Bill Walsh and the offense the Bengals designed eventually became known as the West Coast offense, used not only to great acclaim in San Francisco under Walsh, but by dozens of coaching descendents thereafter, including Andy Reid and Jon Gruden. It is still widely used today.

If that were Carter’s only contribution to the sport, it would be notable. But during that same stretch, Carter introduced something else, albeit to much less fanfare. It was a 1971 academic paper called “Operations Research on Football” written by Carter and Robert Machol, a systems engineering legend credited with pioneering research in wake turbulence. The paper featured landmark data on the value of possession and quantified “expected points” according to field position, now the backbone to modern football analytics.

When I asked Carter this year about the reaction to his paper when it was published, he said there was “none.” He cowrote it while studying in Northwestern’s MBA program during his offseasons with the Chicago Bears in the late 1960s. The Bears helped Carter get the phone numbers of every public relations head in the NFL, and he called them and asked for play-by-play data, which was not yet widely circulated. (He told me the only team to deny his request was the Raiders.) Carter placed punch cards into an IBM computer to process data from 8,373 plays run that year, each with 53 variables on each play, anything from the down and distance, to the score, to the temperature. In the process, he came to believe football stats could never be as all-encompassing as baseball ones; there are simply too many variables. “But I started to realize there were decisions being made that probably didn’t make the best use of expected value,” he said. Head coach Paul Brown did not take to any of his ideas, such as kicking off in overtime instead of receiving, or running out of bounds before the 5-yard line, after which it becomes statistically more difficult to score a touchdown. “Somebody could lose their job,” Carter said, explaining why people were wary of making decisions deemed wild in 1970.

Carter ended up writing columns about football statistics as part of a sponsorship agreement with the A.O. Smith Corporation, which sold data processors. He answered reader questions and built on his previous research, but no one in the league took much notice of it at the time. Carter had, however, provided the basis for a lot of advanced football statistics going forward. “It wasn’t the moon landing, but for some respects it was a big deal,” he said. The influential 1988 book The Hidden Game of Football introduced more metrics similar to expected points.

Burke, who joined ESPN in 2015, went into football analytics in 2006 as a way of settling office debates. He’s something of a godfather of that era of analytics. He started a blog and built his own models. He expanded on Carter’s work and created “expected points added” and “win probability added.” His work is featured in win probability models widely used by NFL teams to help build charts on when to go for two and when to go for it on fourth down. Burke jokes about how many teams have built more advanced models: “I’m a proud papa, but that baby has grown up and has left home.”

In 2005, University of Pennsylvania professor Cade Massey and University of Chicago professor Richard Thaler (who would later be awarded the Nobel Prize) published a paper called “Overconfidence vs. Market Efficiency in the National Football League,” which argued that top draft picks are overrated and that trading back for more picks is almost always the right decision. The instant reaction to their findings was not positive, Massey told me: “I met an NFL owner at a cocktail thing, he was all with me, and as soon as I started talking about trading back, he couldn’t hear anything else. He thought I was a complete idiot.”

The idea of trading back has since become widely accepted. Bill Belichick is credited with popularizing the idea in the NFL, and the Browns have recently used it to help stockpile the assets needed to build one of the best young rosters in the league. Similarly, sites like Football Outsiders have popularized dozens of influential stats—among them DVOA—that are now widely accepted in the NFL. One NFL decision-maker mentioned “speed score,” which Bill Barnwell, now with ESPN, developed at Football Outsiders.

Aaron Schatz, the founder of Football Outsiders, said, “I never wanted to revolutionize the way football teams were run—I wanted to revolutionize the way they were covered.” He thinks this has happened, at least in part, and points to how many people scoffed when Jon Gruden ripped analytics earlier this year instead of simply dubbing his reaction “old school.” (As Hornsby points out, Gruden did quote Pro Football Focus numbers at a press conference this year, so Oakland’s anti-analytics sentiment may have been overblown.) Schatz also thinks the public availability of all-22 film, which shows every angle of a play, has made analytics better because fans can chart their own plays and develop their own metrics. “The analytics revolution is also a film revolution,” Schatz told me. For those who’ve preached about analytics in the NFL, the promised land is near.

“I’ve always said one day we’re going to have a GM who read Football Outsiders growing up,” Schatz said. “I don’t think we’re far off.”

Schatz said analytics will be fully accepted only once owners start going along for the ride, when a coach can go for it on fourth down because the numbers say it’s advantageous without fear of getting fired. Hornsby said that he can tell analytics is being embraced more because people are starting to separate the decision from the execution of that decision. The example he brings up comes from Super Bowl XLIX: Malcolm Butler’s game-sealing interception. “It wasn’t the pass, it was the type of pass,” Hornsby said. “[The Seahawks] had run that exact play six times during that year and it had been totally unsuccessful each time; the best they got out of it was a 2-yard outlet on a third-and-5 against Carolina.”

The decision to pass was sound. The pass itself was not.

The shift in the NFL’s attitude toward analytics is perhaps summed up most succinctly by the story of Matt Manocherian, who, to Schatz’s point, is a Football Outsiders reader. After working in the scouting departments of both the Saints and Browns, Manocherian was let go by Cleveland in 2014 and explored graduate school. He talked to Vince Gennaro, then the director of Columbia’s sports management program. Gennaro is also the president of SABR, the baseball research society, and serves as a consultant for MLB teams. Manocherian had never looked into the analytics side of football, but Gennaro told him that it was the way in. “He said to look at the analytics movements,” Manocherian said. “Baseball is past theirs, basketball is in theirs now, and football’s is still to come. Those are the guys who are going to be running teams in five years.”

Manocherian said that when he was working in the Saints scouting department nearly a decade ago, he was told to ignore most numbers except combine minimums; a cornerback should run a 4.6-second 40-yard dash or better, for instance. He said that New Orleans was considered forward-thinking for the time—one offensive coach told Manocherian that the staff didn’t see interceptions entirely as a negative, instead preferring to focus on explosive plays—but noted that he didn’t get much of an analytics education. (The Saints use analytics now, sources say.) “Ten years ago the average team was saying, ‘We don’t care to hear about this,’” Manocherian said. “Now the average team is saying, ‘We’re kind of starting to understand that it would be a really good idea to figure this out.’”

Manocherian is currently a director of research and development and football at Sports Info Solutions, a company that helps teams with analytics. His evolution from old-school-ish scout to analytics guru is indicative of what the next generation of NFL front offices will look like. In the same way that many of the power brokers of the sport—such as executives like Mike Tannenbaum, Howie Roseman, and Mickey Loomis—made their names when the salary cap debuted in the early 1990s, owning a new and previously overlooked corner, analytics employees will rise up the ranks over the next few years as well. Hiring as many smart people as possible to translate data has become a priority. Eagles owner Jeffrey Lurie told me earlier this season that reducing all the data into metrics that truly matter is one of the biggest keys in the modern NFL.  

“From a personnel arms race, the better and smarter teams have war ships and they are getting the very best missiles and guns for their ship because they know how difficult it is to stay on top with this,” Sharp told me. “Then there are the other teams fighting the same battle and they don’t even know what to do with one gun. They just want to stay afloat, they are trying not to get their coach fired and they’ll be fine being passed over by whoever as long as they keep their jobs.”

The arms race is just beginning. Some general managers think it will take years to produce a sample size large enough to start making massive conclusions. “You’ve got to have a history,” the Titans’ Robinson told me. “You can look from a player health standpoint and get returns pretty quickly, but as far as X plus Y equals Z stuff, everyone is so different it might take a few years to get a real database.”

The vast majority of the analytics data the NFL collects is not public. Swensson said this is partially for competitive reasons, but also because the league wants to make sure the stats are telling the stories they are supposed to tell, “and that it’s not too overwhelming.” He said that it took a few years for player miles per hour—which has been mentioned during broadcasts for three years—to be put in proper context by fans, and thinks a similar thing will happen with other metrics from Next Gen Stats. He pointed out that the NFL still only does location data and not anything approaching motion capture, so you can’t pick up the specific movements of a player. There is a lot more that could come on the horizon.

So this is the start of something new. There will be more data, more decisions made based on that data, and more evidence of what works. It is an exciting time to be smart in the NFL.

“In every field there’s way too much, ‘This is the way we’ve always done it,’ and in the NFL that’s particularly extreme,” Joe Banner said. “Now comes the major shift.”

Kevin Clark