Until six years ago, college football’s championship was decided not by a committee but an aggregate. Let’s look at how the BCS would rate teams in Week 6.
Over the past few weeks in this space, we have broken down the movement in the AP Top 25 and in the USA Today Coaches Poll. For years these two human polls served as part of the Bowl Championship Series, which from 1998 to 2013 determined which two teams would meet on the field to play for the national championship. The BCS went the way of the dodo in 2014, when the College Football Playoff selection committee replaced the formula as the method for picking the top four teams for the plus-one playoff.
But the BCS left behind its unique aggregation of human and computer rankings for the world to remember — and to plug in the numbers whenever they see fit.
The AP poll recused itself from the formula after the 2004 season, when the Harris Interactive Poll was hastily created to replace it as one leg of the tripartite aggregate. When the BCS died, the Harris Poll died with it as a superfluous appendage of a bygone era. That is one reason why we can no longer replicate the BCS rankings in 2019, though we can still approximate the formula by substituting the AP Top 25 back in for academic purposes.
At this point, further, only four of the six BCS computers are currently producing their weekly rankings. The BCS formula combined the six computers, dropping each team’s highest and lowest score from the sextet, and then added together a quarter-share of each team’s reversed Borda count to come up with the third leg of the system. With only four of the six computers currently in operation this season, we can include all four without dropping any weighting to approximate the final leg.
Thus we are able to come up with a composite that mimics the BCS system even if it doesn’t completely replicate the methodology.
Most of the usual suspects feature on here, though beyond the top two teams the order is shuffled a bit going down the list.
What can we learn in 2019 from a look at the BCS formula?
At first glance, it seems that the formula mostly gets it right. But by looking at how the computers shift things around throughout the rankings, we can see where the human polls seem to throw inordinate weight on skewing the landing spot for each team.
- Consider, for instance, Ohio State. The Buckeyes sit at No. 4 in the AP Top 25 this week and No. 5 in the Coaches Poll. But the computers overwhelmingly love the Big Ten frontrunner. Ryan Day’s team tops the computer aggregate, with Jeff Sagarin rating Ohio State the top team in the country and two other computers ranking them second. Nobody has them lower than No. 3, something no other program can boast at this time of the season. The humans seem to be turning around to this same way of thinking, as the Buckeyes claimed seven first-place votes from AP voters and four first-place votes among the coaches this week.
- Similarly, the computers have a lot of love for Auburn. The Colley Matrix places the Tigers at No. 1 in the country, Richard Billingsley slots in Gus Malzahn’s team at No. 4, and the other two computers have them at No. 6. Even with both human polls putting Auburn at No. 7, the computer aggregate puts them at No. 4 behind only Ohio State, Clemson, and Alabama. Like Ohio State, it is the media and the coaches that are depressing where Auburn lands in the final formulation.
- Conversely, while both human polls dropped Clemson — and some individuals thought they might deserve to drop even further — the computers are less skeptical about the defending national champions. Billingsley has the Tigers at No. 1 in the country, and no computer has them lower than No. 5. That is good enough on aggregate to place Clemson right at… No. 2, behind Ohio State, the rare case where the humans and the algorithms created by humans are in agreement.
- Alabama, on the other hand, has received far more love from the human voters than from the computer programs. While Kenneth Massey has the Crimson Tide at the top of his ratings after crunching the numbers, and Jeff Sagarin has Nick Saban’s squad at No. 2, the other two computers have them outside the top four. Given that they took over the top spot in the AP Top 25 despite seeing their vote-shares go down by one-ninth of a point per voter, it is not a surprise that the computers are conflicted on the Tide.
But what about the variance between the computers?
One reason why the BCS was designed in its later iterations to include six computers was a reaction to outliers in different algorithms. Computer systems can rate teams into wildly different slots in the hierarchy depending on what they privilege in the equation. Since there are only four of the six computers currently spitting out results, though, we have to accept those outliers for what they are and work with what they offer.
It is interesting to see the variance between where each computer has a team. In the BCS, only the top 25 computer slots are included in the Borda count. Thus every team outside the top 25 receives a 26, in order to nullify the default rating (26-X is the formula that determines the reverse order for the purposes of calculating the aggregate) and count their contribution to the poll at zero.
Which teams do the computers agree on the least?
- Boise State — The Broncos are the top Group of Five team in both the AP Top 25 and the Coaches Poll, sitting at No. 16 in the former and No. 15 in the latter. The aggregate of the computers also has Boise State at No. 15, but none of the computers agree on where Bryan Harsin’s team should fall. The Colley Matrix has the Mountain West leaders all the way up at No. 3. Sagarin has the Broncos completely outside his top 25, putting Boise at No. 27. That 24-point gap is the second-biggest gap found among top-25 teams this week in the BCS standings.
- SMU — SMU just found their way into the AP Top 25 for the first time since 1986, coming in at No. 24, and they hover on the cusp of the Coaches Poll at No. 27. Three of the four computers have the Mustangs between No. 24 and No. 45. Then there is Colley, which put SMU at No. 5. As the only computer transparent about its formula, Colley has no bias to separate Group of Five teams from Power Five programs. Still, that 40-point gap would likely be thinned down if the other computers were available to separate the outliers.
- Florida — On the other hand, the Colley Matrix does not like Florida at all, positioning Florida at No. 25, while Billingsley puts the Gators all the way up at No. 7. The 18 points separating the two algorithms has gradations in between, with Massey putting Dan Mullen’s squad at No. 11 and Sagarin slotting them in at No. 15. All except Billingsley have the Gators lower than either human poll; the media put Florida at No. 10, and the coaches have them as the No. 8 team.
- Oklahoma — The coaches think Oklahoma is the fourth-best team in the country. The AP voters have the Sooners a little lower at No. 6. Three of the computers have the Sooners between No. 3 and No. 8. The Colley Matrix, though, treated Lincoln Riley’s team much it did Florida. That algorithm put the Sooners all the way down to No. 20. That 14-point gap is likely a result that would be filtered out once all six computers are up and operational, but for now it sinks the Sooners down to No. 7 behind Auburn.
- Oregon — The Ducks are ranked No. 17 overall in the composite of the computer rankings, but those computers aren’t certain how to fill about Mario Cristobal’s crew. Sagarin has Oregon as a top-10 team at No. 9, and Massey sees the Ducks as the No. 15 team in the nation. Billingsley rates the Ducks at No. 18. Yet despite losing to the team that Colley sees as the No. 1 team in the ratings, Oregon is ranked at No. 31 by that system. With both human polls putting Oregon at No. 13, the 22-point gulf between computers only dropped them one spot in the BCS setup, but it nevertheless had an impact.
How much would this matter at this time of the season?
In the BCS era, the ratings came out in mid-October. Thus this is an exercise in projection, though one that offers some interesting insights into how accurate computer systems can be at this point of the season.
The variance reveals the flaws in each system. Colley, for instance, shows much greater variance than the other systems. Even as things balance out over the course of the year, though, it must be noted that Colley’s system is the one by which UCF claimed a national title in 2017. (It is likely that the transparency of the system is the root of its lack of any conference bias in the algorithm, and while merely speculative there is almost certainly some sort of conference weighting built into the other systems due to their publicly-hidden nature.)
That, in the end, is the beauty of aggregation. By compiling the polls together, the BCS offered a human element that was baked not just into the voter polls but also the algorithms themselves. Each programmer came up with his way of judging each team, and then let the computer do the magic from there. Beyond the initial subjectivities of the design, the beauty is the objectivity thereafter.