Lessons From Four Years of College Football Playoff Selection Committee Decisions
The best strategy is to be Alabama
The College Football Playoff selection committee has gone to great lengths to help the public understand how it makes its decisions. All the committee theoretically needs to do is pick four teams to make the field after conference championship weekend, but instead it releases six sets of 25-team rankings each year, a breathtaking and unnecessary step taken in the name of transparency. (And I suppose in the name of giving ESPN weekly programming, but transparency, too.) When those rankings are released, the committee’s chairperson typically speaks at length about the committee’s thought process surrounding its most controversial choices. The committee allows media members to participate in a mock selection event to get a feel for how everything works, and the committee’s membership list, voting process, and criteria for selecting teams are all publicly listed online.
Which is why it’s so amazing that basically nobody grasps how the selection committee makes its decisions four years into the playoff era.
While the choice between Alabama and Ohio State was clearly the toughest the committee had to grapple with in 2017, most thought that Ohio State would nab the fourth and final playoff spot. Seven of eight CBS writers picked the Buckeyes to make the field; FiveThirtyEight’s projection model gave Ohio State significantly better odds than Alabama; ESPN analyst Kirk Herbstreit gave an impassioned speech about how he personally would pick Alabama, but felt the committee would go with Ohio State; while Vegas favored the Crimson Tide to be selected, so many people placed money on the Buckeyes that it was speculated the committee’s decision had been leaked to big-money bettors. Of course, the committee gave the last spot to Alabama, causing Herbstreit to walk off ESPN’s live set in apparent shock.
Surely, there has to be some logic in the committee’s decisions if we just look hard enough. So we analyzed each of the playoff’s four final selections to date, along with the explanations given by committee chairpeople after those decisions were announced. The goal was to see if we could learn anything that would help predict future decisions—or determine if the committee is just making everything up as it goes along.
2014
The Field
1. Alabama (12-1)
2. Oregon (12-1)
3. Florida State (13-0)
4. Ohio State (12-1)
Was There Controversy?
Yes. Ohio State got the final playoff spot over Baylor (11-1) and TCU (11-1). The Buckeyes were ranked two spots below the Horned Frogs entering championship Saturday, but moved up after throttling Wisconsin 59-0 in the Big Ten title game while the two Big 12 schools remained idle. This was during the stretch when the Big 12 lacked a conference championship game, claiming instead that its round-robin schedule would produce “One True Champion.” That branding massively backfired. Baylor was also criticized for its abysmal nonconference scheduling; the Bears played SMU, Northwestern State, and Buffalo.
How Did the Committee Justify Its Decision?
Then-committee chairman Jeff Long didn’t specifically say that the Big 12 was penalized for its lack of a championship game, but he winked at it with statements like, “Ohio State’s performance in a 13th game gave them a quality win over a highly ranked team.”
Long also said the “committee noted that Ohio State’s nonconference schedule was stronger than Baylor’s.” The Buckeyes played Navy, Virginia Tech, Kent State, and Cincinnati outside of its Big Ten slate, losing 35-21 to the Hokies.
What Did We Learn?
The committee’s point was strong and clear: Having a championship game victory allowed Ohio State to distinguish itself from the two other teams in the mix.
Ah, what were you thinking, Big 12! You had two potential playoff teams, and if you’d just been smart enough to have them play each other one more time, one team would have had a chance to make the playoff over Ohio State. Sure enough, this snub convinced the Big 12 to bring back its league championship game. (As it happened, the reintroduction of that title game this year cost the Big 12 an important bowl slot. The Big 12 really can’t win—it must be taking lessons from Kansas.)
There was also a hint that teams should strengthen their nonconference schedules, but that was buried in the widespread zinging of the Big 12. And the fact Baylor was ranked fifth and TCU sixth seemed to indicate head-to-head victories could serve as a tiebreaker between teams with similar résumés, as noted in the committee’s criteria: Baylor beat TCU 61-58 in 2014, the Horned Frogs’ only loss.
2015
The Field
1. Clemson (13-0)
2. Alabama (12-1)
3. Michigan State (12-1)
4. Oklahoma (11-1)
Was There Controversy?
Not really. A pair of undisputed conference champions, Michigan State and Oklahoma, took the final two spots over Iowa (12-1), Ohio State (11-1), and Stanford (11-2). Both the Hawkeyes and Buckeyes lost matchups against Michigan State.
How Did the Committee Justify Its Decision?
The committee suggested the decision was clear cut: “I don’t think I could characterize it that Stanford or Ohio State was close to getting in the top four,” Long said on the selection show. He spent most of his time discussing the difference between the third-ranked Spartans and fourth-ranked Sooners. Long also made clear that the Big Ten championship game between Michigan State and Iowa had served as a de facto play-in game.
What Did We Learn?
The fact the Spartans and Sooners were considered so obviously superior to the Hawkeyes, Buckeyes, and Cardinal heavily implied that a team winning its conference championship and head-to-head games against other teams in the mix would cement its playoff case. After all, it didn’t make a difference that Ohio State started this season 10-0 and was in the top four during the first three installments of the 2015 playoff rankings.
2016
The Field
1. Alabama (13-0)
2. Clemson (12-1)
3. Ohio State (11-1)
4. Washington (12-1)
Was There a Controversy?
A big one. Ohio State and Washington got the nod over Penn State (11-2), in spite of the Nittany Lions winning the Big Ten and beating the Buckeyes in a head-to-head matchup in October.
How Did the Committee Justify Its Decision?
Kirby Hocutt, who took over the chairmanship from Long, clarified that the primary debate in 2016 was not Ohio State versus Penn State. Although the Nittany Lions beat the Buckeyes 24-21 and won the division and conference in which the Buckeyes play, Hocutt indicated that the committee felt strongly that “Ohio State was a better football team than Penn State.”
But he was light on details. “We come back to our protocol and that is identifying the four very best teams in college football,” Hocutt said about the gap between the Buckeyes and Nittany Lions, explaining that conference championships were just one factor the committee looked at. He did praise Ohio State’s scheduling—“I would say you need to play a good schedule and win those games. And Ohio State, I think going on the road and winning against a top-10 Oklahoma team this year, shows that the selection committee respects that kind of schedule and those wins that come from those kind of nonconference games.”
As for explaining the committee’s choice to go with Pac-12 champion Washington over Penn State—the one that Hocutt said was tougher—he was again sparse with details, lightly chiding Washington’s nonconference schedule (Rutgers, Idaho, and Portland State) while saying that holistically the committee felt the Huskies were better.
“Had Washington had a stronger strength of schedule, I don’t think the conversation and discussion would have been as difficult,” Hocutt said. “We looked at key statistical categories, which translate to performance on the field each week—statistical values that we see value in. Washington has the advantage. We talked about performance and what the coaches [on the committee] say over the course of 13 games, and Washington seemed to have the advantage.”
What Did We Learn?
The committee’s 2016 decision seemed to contradict itself—Hocutt both hinted at how Ohio State’s strong nonconference schedule was a decisive factor in its inclusion and said that Washington’s weak nonconference schedule didn’t bump it out of the field. The lesson: Scheduling big games was neat if a team won, but if a team didn’t schedule big games it might have been fine anyway.
This choice also contradicted the logic used in previous years. Ohio State made the 2014 field largely because of its conference championship victory; Michigan State made the 2015 field based on its conference title and head-to-head wins over Ohio State and Iowa. How were the Buckeyes seen as so far superior to the Nittany Lions team that beat them and went on to win the Big Ten?
Hocutt didn’t say it, but the answer seemed simple: Penn State had lost twice, while Ohio State and Washington had lost only once. For those counting at home, two is more than one.
2017
The Field
1. Clemson (12-1)
2. Oklahoma (12-1)
3. Georgia (12-1)
4. Alabama (11-1)
Was There Controversy?
You bet. Alabama got the final playoff spot over Big Ten champion Ohio State (11-2). The Crimson Tide had fewer big wins and a weaker schedule, but the Buckeyes had two losses, one of which came by 31 points at Iowa. The Buckeyes were blown to smithereens by the Hawkeyes’ corn-powered artillery.
How Did the Committee Justify Its Decision?
Hocutt referred to “one-loss Alabama” and “two-loss Ohio State” as he laid out the committee’s choice, going on to specify that of Ohio State’s two losses, “[the] more damaging [one] was the 31-point loss to unranked Iowa.”
He also explained that while conference championships are a factor in the decision-making process, the committee deemed that Alabama was “unequivocally better” than Ohio State, so the conference championship tiebreaker did not come into play and bump the Buckeyes over the Tide. “Our charge is, the selection committee has to identify the four very best teams in the country,” Hocutt said. “When there are close separations between the teams, we're instructed to look at certain criteria. In this case, the margins weren't close enough to look at those.”
What Can We Learn?
I can find only one trend through these four years of playoff selections: A team can’t lose two games during a season.
When the committee needs to decide between teams with the same number of losses, as happened in 2014 and 2015, it will use head-to-head wins, conference championships, and strength of schedule as determining factors. When the committee needs to decide between teams with differing amounts of losses, as happened in 2016 and this year, it will ignore everything else and say that the number of losses created a gap that made one team “unequivocally better” than the other, negating the consideration of any other factors.
To be fair, this means the committee is following its criteria, which say head-to-head games and conference titles are to be used as tiebreakers between otherwise comparable teams. But it raises the question: Why do we have a 13-person committee deliberating six times a year if its entire philosophy boils down to looking at the loss column? Wasn’t this one of the main knocks against the BCS? It seems like the only decision the committee makes beyond observing how many losses each team has is keeping non-power-conference schools as far away from the playoff field as possible.
If this is the way the committee thinks, teams shouldn’t aim high when it comes to scheduling. Sure, the committee values strong schedules and conference championship games. But while conference titles and marquee early-season victories are viewed as noteworthy, the risk of a team losing games against quality opponents outweighs those games’ potential benefit. A team seeking a playoff berth should just endeavor to survive a season rather than raising its degree of difficulty, because the main goal is to get to December with as few losses as possible.
Either that, or the committee has a mandate to pick Alabama every year. The Tide have rolled into the playoff in all four years. Maybe “blackmailing committee members” is part of Nick Saban’s famous Process.