Rookie Scouting Portfolio contributor Mark Schofield takes a second look at former NFL quarterback Colt Brennan and discovers lessons for the future of evaluating the position. 

I loved Colt Brennan. I loved him at the paddock, before the second race, and outside the men’s room when I placed my bet.

I loved him before you even got up this morning.

Before there was any of this—RSP Film Rooms, Inside the Pylon, Locked On Patriots, or “17 Drives,” Colt Brennan was my guy. Even when I was just a struggling lawyer who spent too much of his spare time — or billable time — thinking and studying the quarterback position, my football sensibilities were dreamy over Colt Brennan.

Okay so we got off on a little “Ocean’s 11” tangent there for a moment, but let’s recenter ourselves. Before my second career began as a football analyst began and I was just posting on message boards in a pre-Twitter Era, I thought Brennan had a shot to succeed in the NFL.

I watched every moment of that Sugar Bowl against Georgia, his final college game, frustrated at the fact that the Bulldogs harassed him all night long and forced three interceptions from him. But I held on to the belief that Brennan could be a good NFL quarterback, and living in the Washington, D.C. area I was excited when the local team acquired him in the sixth round.

When Brennan lit it up in the preseason, I thought my beliefs would be confirmed. But his NFL career faded away into a fog of injuries and inexperience.

I had not thought about Brennan for years. Then, one day, I was reading through “Coaching Quarterbacks: By the Experts 3rd Edition”  and “Coaching the Passing Game: By the Experts” in preparation for a piece I was writing about the evaluation of the position.

Trying to illustrate the importance of competitive toughness when studying quarterbacks—a non-negotiable trait for the position—I was pulling out quotes about leadership and toughness from coaches like Chris Petersen and former quarterbacks like Ken Anderson when I stumbled across a presentation from June Jones, Brennan’s college coach at the University of Hawaii.

Here’s what Jones told an assemblage of other coaches about his quarterback, during a presentation on the Levels concept from “Coaching the Passing Game: By the Experts”:

Colt Brennan was our quarterback this year and led the nation in passing. He had no idea what we were doing. Maybe the last two games, he understood something. I think it is simple, but had it not been for this pattern and couple of other little things we did, he might have struggled all year.

I kept scaling back the offense, trying to make it simpler. He thought 781 and 781-Change were two different plays. After he went over the pattern a million times it finally sunk in. The last three or four games, he was something else. He was not that way in all areas. Next year, I am cutting out more things from the offense. I will not ask him to do as much.

He completed 70 percent of his passes, threw 38 touchdowns, and had 10 interceptions. We watched the tape and he read from one to two, but never could get to three and four. He was a tremendous scrambler and made big plays because he could avoid tacklers, but he did not really understand what was going on.

During Brennan’s pre-draft process, Jones was perhaps his biggest defender. But these words stuck out to me and forced me to revisit my opinion of Brennan from years ago.

Every draft season, those of us on the outside looking in wonder how NFL franchises can continue to get quarterback evaluations “wrong.” Believe me, I’m a card-carrying member of that crowd.

This anecdote about Jones and Brennan is a perfect example of the wealth of information that NFL organizations have access to, that those of us on the outside likely never hear. Unless we found ourselves at that Jones presentation, odds are we would not have known about how the coach needed to scale back the offense for Brennan.

NFL teams have this access. They can get the coach on the phone to work through an evaluation of his player. Still, even with such access, teams miss.

It leads us to wonder: How can the evaluation of quarterbacks be improved?

Part of the answer might be to “grind the tape more.” Had I sat down and charted all of Brennan’s reads and throws from his time in college, I might have uncovered that the offense was more simplistic than I believed.

That might have been a difficult proposition in 2007, but today with increased access to collegiate All-22, thanks to the various dark corners of Football Twitter, it is easier to get a more complete picture of a quarterback. So perhaps that is simply the answer.

Yet, that also ignores some of the advancements in football analysis that are being made on a weekly or even daily basis. Thanks to the rise of analytics and more advanced statistical study, we have access to more information than we dreamed of years ago.

This has led to various outlets coming up with new ways to study the position and create models for quarterback evaluation. Just last year Josh Hermsmeyer from FiveThirtyEight and Bill Connelly from SB Nation opined on the study of quarterbacks before the draft. Hermsmeyer debuted Completion Percentage Over Expected (CPOE) while Connelly examined success rate, Adjusted Passing Yards per Attempt (AY/A) and Adjusted Net Yards per Attempt (ANY/A) to outline the advanced stats that identify quarterbacks who will actually be good.

Both put Kyler Murray atop their lists.

Interestingly enough, those metrics would have pointed to Brennan as having a shot at NFL success. CPOE takes into account completion percentage, depth of target and total QBR, so it is difficult to create a CPOE for Brennan (lacking depth of target and total QBR for him). But looking at his completion percentage (70.4% for his career) and as Hermsmeyer argues completion percentage is also a strong indicator of NFL success, it would follow that Brennan would likely be viewed well by some of these models.

In addition, some of the metrics that Connelly relies upon, such as AY/A, also would work in Brennan’s favor. He left Hawaii with an AY/A of 9.4, behind only Baker Mayfield from the 2018 first round class. Furthermore, Brennan’s completion percentage of 70.4 would be higher than any of the quarterback that came out last year.

And yet…

Now, as Hermsmeyer himself states, “…all models are universally wrong, but some can be useful.” The CPOE model he created also predicted that Johnny Manziel would go on to NFL success, along with Kellen Moore.

The model also is very bearish on Lamar Jackson, although time may prove the model right depending on how Jackson’s next few seasons fare. Hermsmeyer pointed out those instances in his write up of CPOE.

Models are certainly useful as another data point or a bit of information to consider when putting together an evaluation of a quarterback. After all, the point underlying this piece is that even with all the information available to NFL teams, they miss on quarterbacks.

Those of us on the outside, with much less information to work with, still try and grade these players. So adding statistical models and studies to the process makes sense. While they might not be perfect, they can still be useful as one more data point to consider. Additionally, as more and more data is gathered and these models are put through successive years of testing and refining, they will improve. So will the underlying analysis.

However, there is one final aspect to consider: Putting together the entire tool kit. The film study still needs to be conducted as well. As the Brennan example illuminates until you see the player in action you cannot truly be informed about their skill-set.

The numbers are a part, but what the player is tasked with doing, and how he accomplishes this task, are also a part. Sure, depth of target can uncover whether a player is just “throwing screens” or actually pushing the ball downfield, but it does not uncover the player’s processing speed, or whether he is making the right read on a given play.

And the numbers might fail to uncover that the player’s coach needs to simplify the offense because the QB is having trouble figuring it out.

Quarterback evaluation is a difficult process. Even with all the information available to them, NFL teams still miss. For those on the outside of the league going down this path, having as much information as we can get just makes sense. Use all of it. Let the numbers guide you to areas of further inquiry, and let the film guide you to areas of deep statistical study. The future of quarterback evaluation lies in tying the two together.

Mark Schofield is a former quarterback at Wesleyan. In addition to is work at the Rookie Scouting Portfolio, you can find him at Inside the Pylon, Pro Football Weekly, Big Blue Review, and hosting the Locked-On Patriots Podcast. You can follow Mark Schofield on Twitter @MarkSchofield.

For the most in-depth analysis of offensive skill players available (QB, RB, WR, and TE), get the 2019  Rookie Scouting Portfolio. If you’re a fantasy owner the Post-Draft Add-on comes with the 2012 – 2018 RSPs at no additional charge.

Best, yet, 10 percent of every sale is donated to Darkness to Light to combat sexual abuse. You can purchase past editions of the Rookie Scouting Portfolio for just $9.95 each. You can pre-order the 2019 RSP now (available for download April 1).