College Football Playoff: Lessons learned evaluating teams as a member of a mock selection committee


cfp mock committee College Football Playoff: Lessons learned evaluating teams as a member of a mock selection committee

Grapevine, Texas — When I was a student journalist at Baylor in 2014, ESPN’s final graphic was no. 4 transferred to Ohio State. The Bears, a famous co-champion of the Big 12 with TCU, found themselves sitting on the outside of the first College Football Playoff. Eight years later, to my great chagrin, the College Football Playoff Mock Selection Committee did the same.

But we’ll get to that.

12 members of the media, myself included, were invited to participate in the annual mock test hosted by the College Football Playoff. The training takes place behind the doors of the same swanky meeting room at the Gaylord Texan as the real team that holds the fate of countless athletic fields in its hands.

For the day, I played the role of Navy athletic director and current CFP board member Chet Klatchuk. “Get some guts to play Chet,” joked a 72-year-old CFP employee in the seat I occupied. But if that’s what they want, a deep-throated recap of the controversial 2014 playoff run is the perfect place to start.

The challenge is to rank the four “best” teams

The process for ranking teams has been reported in detail, but here’s a quick rundown: The rankings are divided into seven groups — each of three places 1-9 and each of four 10-25. We vote for each trio by considering six teams at once. Group One: Alabama, Ohio State, Oregon, Florida State, Baylor and TCU. Staying there.

CFP president and NC State athletic director Boo Corrigan charged us with finding the 25 “best” teams in the country. From the beginning, however, it became clear that each of us had different definitions of that metric. Some valued postseason success, while others wanted a better team leading up to Selection Sunday. Regardless, resumes ended up being a more important consideration than which team would win on any given Saturday.

Arguing with some of the most famous names in the game was probably the part I was most unprepared for. Former players in the room were adamant that Florida State’s undefeated season was a trump card. I don’t agree. Pro Bowl running back Deuce McAllister and former first-round quarterback E.J. Try to be a schlub like me and tell Manuel that success isn’t all about metrics.

The CFP features an analytics system developed by SportSource Analytics that compares four teams simultaneously with a wide range of metrics displayed on giant screens in front of the team. Baylor vs. TCU vs. Ohio State vs. Florida State Comparison Spot No. Everything that happened in 2014 made sense when it came on screen between number 3 and number 4.

Jayaraja (left) and former NFL linebacker Kirk Morrison

Kevin Jayraj/College Football Playoff

Group tables are displayed on the side with the results list and common opponents or head-to-head are highlighted. However, the most striking part of the page is the color gradient with ranks ranging from green (good) to red (bad). When the schedules for TCU and Baylor came around, the red and the rest of the field was shiny.

Never underestimate the power of data visualization!

Also, one of the only real criteria listed by the committee is that similar teams are prioritized for head-to-head wins or conference championships. CFP director Bill Hancock confirmed to the panel that since both Baylor and TCU were submitted as co-champions by the Big 12, the teams should be considered 0.5 titles each.

Compared to Ohio State and Florida State, the decision isn’t as difficult. We can argue about the specific strength of the schedule and evaluation metrics used by the team, but it gave us a complete picture of the team’s decision.

Perhaps led by Manuel’s Florida State hand in the room, the ‘Noles actually moved up to No. 2 in our mock rankings, behind Alabama. I Florida State No. I voted for 5; 12 metrics that the CFP has found to correlate with success hated FSU and its tight margin of victory aren’t enough to convince me of other (better) conference champions.

In full disclosure, I focused more on performance numbers and quality of play than other voters. SportSource Analytics provides neatly organized and color-coded statistics that correlate to winning scores and one-point offense after extensive historical research. (A special teams metric was added to the team’s top 12 statistical factors at the behest of former Nebraska coach Tom Osborne.) Other mock committee members talked about the quality of their schedule. Some leaned on the quarterback game or tried to boost their NFL prospects.

Ultimately, that disagreement is the point. Seven of the 13 CFP board members are athletic directors. Former NFL player and MIT Ph. D Seven are former college football players, including mathematician John Urschel. Two are coaches. One, former USA Today columnist Kelly Whiteside, is a journalist. Everyone sees the game differently.

A mix of ex-players, writers and TV personalities gave our team a similarly interesting environment. San Diego State legend and NFL vet Kirk Morrison wanted to make sure the group of five competitors got the attention they needed. Athlete Ari Wasserman asked us to consider recruiting gaps. editor John Dalty asked Katy Perry to remember being at The Grove for Ole Miss’ matchup with Alabama. McAllister and former Michigan quarterback Devin Gardner pushed us to consider winning above all else. Brett Bielema told Ole Miss 30-0 that the Arkansas team won the “Borderline Erotic” Bowl over Texas. Everyone has their share.

NC State Athletic Director and CFP President Boo Corrigan

Kevin Jayraj/College Football Playoff

How to sort the best from the rest

If you’re wondering why there might be random moments in the rankings, that’s why. In each secret poll different voters weigh their own criteria while trying to convince other voters that their perspective is correct. And, in fact, it made the discussion of numbers 7-25 infinitely more interesting and competitive.

“When I was the athletic director at Army, we would have been happy to finish in the top 25,” Corrigan told the group. “As much importance should be given to those at the bottom of the rankings as those at the top.”

Which we did. The process demanded it.

Some participants tried to bring the strength of the conference into the picture or tried to reject some proposals from the Group of Five. CFP Executive Director Bill Hancock and Corrigan quickly shot them down. After examining the matchups and revisiting the green-to-red scheduling gradient, it became clear that it wasn’t necessary.

The data visualization featured records against the team’s previous top 25 and previous top 10 against teams over .500, but in reality, very few of those high-level data points have been picked. Instead, tables filled with green drew our attention. Ranked games — or borderline ranked games — were discussed on a case-by-case basis.

We drew Boise State and some were shocked by the amount of quality opponents in the “Group of Five” conference. It moved up. Conversely, Wisconsin’s “Power Five” table is filled with red. SEC rivals Ole Miss and Georgia were filled with green wins and red losses. Michigan State and Kansas State haven’t had a streak of wins, but their only blemish has come against the top teams in the country.

After each round, we’ll vote on new teams. Finally, we had an opportunity to discuss any major discrepancies. UCLA was first in Tempe, Arizona, behind an Arizona State team that was walloped by the Bruins. It has been addressed. Georgia moved further down, while Arizona moved up several spots. Marshall entered the field comparing favorably to teams like Minnesota and Louisville in 12 factors.

The discussion lasted for five hours. The real thing is six weeks of talking, hours of discussion and countless weekends watching cutups of every relevant football game. Hancock noted that every game he watches has a team member who decides to rate each player individually. Every year is different.

What will change (and what won’t)

Naturally, that leads us to a 12-team playoff. In many ways, the decisions made by the group are variable. With six auto bids, there will be less emphasis on figuring out which teams are seeded and more emphasis on ranking and seeding teams.

When the playoffs expand, the team will be 13 members. They still plan to field 25 teams and feel comfortable that at least six conference champions will fall within that number. Instead of filling bowl attachments with teams, they now fill at-large playoff spots. Basically, the process won’t change much as the playoffs unfold, and those in the room believe it will be sooner rather than later.

In the end, I gained a lot of respect for the process. It’s not perfect, it’s not flawless, but it’s perfect. With the amount of information and video available to team members, I feel more confident that they have all the tools they need to make better decisions. More importantly, the secret ballot process makes it a little more difficult to jimmy the rankings for branding or matchup purposes. Transparency doesn’t confirm any conspiracy theories, but it gives me more faith in the process.

Hancock, whenever you need a new name to add to your crew list… I’m ready.



Please enter your comment!
Please enter your name here