SMQ: What is the value of a win versus a loss in the College Football Playoff?

Can we make sense of how the College Football Playoff selection committee rates wins and losses in their rankings? Let’s try to figure it out…

Welcome back for another edition of the revamped Sunday Morning Quarterback here at Saturday Blitz. In this week’s edition, you’ll get our usual Sunday morning segments. On second down we’ll punch the ball forward with a look at Cincinnati’s hopes of making history out of the Group of Five. I’ll air it out on third down with my best guess of how the College Football Playoff selection committee will justify each team’s position in their Top 25 rankings on Tuesday. When we hit fourth down, we’ll boot the ball toward the uprights and look ahead to the big games coming up in Week 12.

For our first down this week, though, we need to talk about how wins and losses are evaluated by the College Football Playoff committee.

1st Down: The value of wins and losses in the College Football Playoff

Before we begin to discuss the issue of how the College Football Playoff selection committee rewards some wins more heavily and penalizes some losses more strictly than others, we must first set aside the notion that any system for determining the No. 1 team at the top level of college football has ever truly had meritocracy at the heart of its calculations. Whether a human aggregate like the AP Top 25, a mathematical system like the old Dickinson ratings, a modern computer algorithm issued by a statistician like Jeff Sagarin, every system has biases baked into the formula.

We cannot escape our subjectivities as human beings, and all systems start with a human element. Especially in a sport with so few data points between teams and leagues, there is no possible means of achieving an illusion as alluring yet as inconceivable as true objectivity.

That out of the way, we must also acknowledge the very human inclination to seek out patterns and make sense of the nonsensical. That inclination led me to begin compiling data on how wins and losses have affected movement in the College Football Playoff rankings.

Acknowledging that the committee’s rationales change from year to year and that this dataset only covers the first three years of the College Football Playoff, let’s draw out some patterns from the data.

• Of the 58 teams that entered a bye week ranked in the College Football Playoff rankings, nine were ranked lower exiting the bye week. Only one, Houston in 2016, fell completely out of the Top 25 after a bye. More frequently, teams climb in the poll after a bye thanks to attrition in other games around the country. Between 2014 and 2016, 31 of the 58 teams coming off a bye climbed at least one spot, with two teams rising three positions and 11 more climbing two spots.
• From 2014 to 2016, there were 18 contests that featured matchups between top-10 programs in the College Football Playoff rankings. The 18 winners climbed an average of 1.7 spots in the rankings, while the 18 losers dropped 3.9 spots on average. In the 69 contests featuring two ranked Top 25 teams, the winners climbed an average of 2.6 spots. Nearly one-fifth of the losing teams dropped completely out of the rankings, while those who remained fell an average of 3.5 spots.
• In a system where 25 teams are ranked at any time and 130 teams play at the FBS level, there are always bound to be far more matchups between Top 25 teams and unranked teams than between two ranked teams. There were 195 of these ranked-versus-unranked contests between 2014 and 2016. About 75 percent of the games ended in victory for the ranked team but yielded a bump in the rankings of just 1.5 spots in the process.
• That means that the other one-quarter of those games ended in losses for the ranked team. Half of the 48 teams that lost to an unranked opponent while ranked found themselves unranked when the committee released their fresh hierarchy the following Tuesday. The other two dozen teams fell an average of six spots, with Texas A&M suffering the most precipitous plummet in 2016. After losing to 4-5 Ole Miss in Week 11, the Aggies dropped 17 places from No. 8 to No. 25.

Again, this is still incomplete data at the moment. But a superficial glance at the shift in the rankings between 2017 and 2020 reveal similar patterns. The exact averages and ratios might be slightly skewed, but we can at least begin to find baselines in the general behaviors of the 12 people tasked with determining who gets a chance at the mythical national championship and who is left to hope that one of the NCAA’s many official selectors give them a chance at an alternative claim to the crown.