Open Rants by Ellis Zusel

QB Key Model: Looking Beyond Sports to Evaluate QBs

As the NFL draft approaches, the eyes of talent evaluators, front office executives, and prominent
media figures will be focused on one position above all else: quarterback. Each year, teams use premium
picks on QBs hoping they will become “the dude.” While we hope that whichever promising prospect our
team selects becomes the franchise player he is expected to be, the reality is that teams typically do not
draft this position effectively. Since 1999, only 8/49 (16%) of the QBs selected in the first round have or
are on track to reach a winning percentage of .545 over 10+ seasons or won a Super Bowl (definition of a
successful first round QB pick). The current methodology is not working. In order to improve the current
way quarterback talent is evaluated, successful, non-sports related predictive models should be studied.

For my research, I focused on Allan Lichtman's model for predicting US presidential elections.
Lichtman, a history professor at American University, has correctly predicted every election since his
model debuted in 1984. The model can even be used to correctly forecast every election since 1860.
Lichtman’s model, known as “The Keys to the White House,” consists of 13 indicators, called keys. Each
key is a true-false statement about the political conditions leading up to the election. For instance, key #3
is “Incumbency: The incumbent party candidate is the sitting president.” If that statement is true, the key
favors the incumbent party. If not, it favors the challenging party. If the challenging party has six or more
keys in its favor, they are projected to win the election. If not, the incumbent party is expected to remain
in power.

With Lichtman’s work in mind, I studied a sample of 76 quarterbacks, including every single first
round QB selected since the 1999 draft. After finding seven strong data trends that could be used as
quarterback keys, I created the “QB Key Model.” The system yields a binary grade, either pass or fail. In
order to receive a passing grade, the prospect must receive 5/7 keys. A quarterback that receives a passing
grade is projected to be a worthwhile first round pick (as defined earlier), while a player that receives a
failing grade is projected not to reach that threshold. The keys are as follows:

1. Winning: The prospect’s winning percentage was at least 66.7% in their final college
season.
2. Competition: The prospect played at a Power 5 conference, excluding the Big 12.
3. Experience**: The prospect is a starter for at least two college seasons.
4. First Round Talent: The prospect is projected to be a first round pick by a major news
outlet (such as ESPN) in the week before the draft.
5. Sitting in the NFL: The prospect sits for at least one year in the NFL.
6. MLB: The prospect has been either drafted or offered a contract by a professional
baseball team.
7. Pass Attempt to Interception Ratio: The prospect’s PA/int ratio is greater than 39 PAs / 1
int.
** The experience key is mandatory, as no QB since 1999 that started less than two years
in college was able to meet the successful career benchmark.

Out of the 76 quarterbacks that have received grades, 17 QBs are not factored into the system
because they haven’t been in the league for a minimum of five seasons yet. Out of the remaining 59
quarterbacks, the model has correctly predicted pass or fail for 42, yielding an overall success rate of
71%. The model is a much stronger predictor of failure, as its accuracy when giving a failing grade is
86% (32/37), while its accuracy when giving a passing grade is 45% (10/22).

The divergence in accuracy when giving a pass vs a fail shows how the model should be used:
Because the model is very good at predicting failure, it should be used as a tool for teams to filter out
busts with near certainty. Teams could then use subjective factors (film, leadership, injury proneness, etc.)
to further evaluate the quarterbacks that received passing grades, knowing that there would be about a
50% chance of that QB succeeding. Considering that QBs drafted in the first round are only successful
16% of the time, an accuracy rate of 45% when assigning passing grades is a large improvement.

These results show that the QB Key Model can serve as a powerful addition to the current way
college quarterbacks are evaluated. Data table can be found here: http://bit.ly/393J5XB