2014 DQR and Evaluating QB Play Around the League

[deljzc]

I created DQR (deljzc’s quarterback rating) back in 2010. I have since tweaked the formula a few times as I continue to try and properly weight and proportion the five criteria I used in determining a “rating”:

Completion %
Yards per Pass Play
Points Scored per Possession
Turnovers per Pass Play
“Big Play” Ratio (20+ yard plays minus sacks per Pass Play)

These are the scores and rankings of the 32 NFL teams this past season, by division:

AFC East (75)
T. Brady, NE = 97.3 (7th)
R. Tannehill, MIA = 86.6 (16th)
K. Orton, BUF = 77.8 (23rd)
G. Smith, NYJ = 68.3 (29th)

AFC North (51)
B. Roethlisberger, PIT = 103.8 (4th)
J. Flacco, BAL = 95.3 (10th)
B. Hoyer, CLE = 81.0 (18th)
A. Dalton, CIN = 80.7 (19th)

AFC South (79)
A. Luck, IND = 95.7 (9th)
R. Fitzpatrick, HOU = 91.9 (11th)
Tennessee = 70.1 (28th)
Mettenberger = 70.2
Whitehurst = 78.5
Locker = 59.8
B. Bortles = 58.8 (31st)

AFC West (62)
P. Manning, DEN = 107.1 (3rd)
P. Rivers, SD = 89.0 (14th)
A. Smith, KC = 88.6 (15th)
D. Carr, OAK = 60.8 (30th)


NFC East (51)
T. Romo, DAL = 110.8 (2nd)
E. Manning, NYG = 89.1 (12th)
Eagles = 83.3 (17th)
M. Sanchez = 83.8
N. Foles = 82.5
Washington = 80.4 (20th)
R. Griffin = 72.4
K. Cousins = 90.5
C. McCoy = 82.9

NFC North (70)
A. Rodgers, GB = 116.8 (1st)
M. Stafford, DET = 80.2 (21st)
T. Bridgewater, MIN = 79.7 (22nd)
J. Cutler, CHI = 74.7 (26th)

NFC South (71)
D. Brees, NO = 97.7 (6th)
M. Ryan, ATL = 96.7 (8th)
C. Newton, CAR = 76.7 (25th)
J. McCown, TB = 53.6 (32nd)

NFC West (69)
R. Wilson, SEA = 101.4 (5th)
Arizona = 89.1 (13th)
C. Palmer = 93.4
D. Stanton = 84.8
C. Kapernick = 77.6 (24th)
St. Louis = 71.8 (27th)
A. Davis = 66.7
S. Hill = 78.3


Remember, I don’t consider this a ranking of the actual quarterbacks but more of their actual PLAY and PERFORMANCE for 2014. This does not take into account strength of schedule or surrounding talent, just production and measurement of the five criteria listed above.

 

[Bigappleyinzer]

*Enters thread looking for Manziel's piddling score

*Leaves thread disappointed

 

[slashsteel]

good work Del !

 

[oicu8ad]

deljzc, I always enjoy your posts & analysis, & I always wonder: how the hell do you find the time?

Thanks.

 

[ark steel]

If you are not trying to consider talent surrounding the QB, why is points per possession included?

I guess it depends on the weighting, but if a QB throws 3 passes/completions for 10 yards each and the next play is the RB running for a 50 yard TD run, it seems to boost the rating, when the RB got most of the yards and the score.

 

[Wingman]

Interesting point ark. I am wondering how say play action would affect the running backs ability to get that long run or a series of short runs to get a score. The long run is I think a bit rare but a grind it out run game that keeps moving the chains with a short or limited passing game would make that qb look better than one that relied on him to do more. I suspect that might be true in Dallas with Romo although I do not watch the games so have no idea other than the few I've seen when they were playing the Steelers.

 

[Wingman]

Interesting stuff del thanks for sharing.

 

[ark steel]

Interesting point ark. I am wondering how say play action would affect the running backs ability to get that long run or a series of short runs to get a score. The long run is I think a bit rare but a grind it out run game that keeps moving the chains with a short or limited passing game would make that qb look better than one that relied on him to do more. I suspect that might be true in Dallas with Romo although I do not watch the games so have no idea other than the few I've seen when they were playing the Steelers.

That is why weighting would be important. for example, there points/possession didn't have a lot of weight and completion % and yards/pass had a lot of weight?

 

[tapeANaspirin2it]

That is why weighting would be important. for example, there points/possession didn't have a lot of weight and completion % and yards/pass had a lot of weight?

speaking of weighting, i always thought that there should be situational weighting for QBs. If you are down by 30 then passing yards come much easier. Turnovers in the redzone should be weighted more. Then there's situational awareness. Does a QB throw a deep incompletion on 3rd and 4 when he could have moved the chains? Does a QB take a sack that moves them out of FG range?

How do you factor in yards after catch for a QB? The first thought is to remove them from the equation because it's the receiver who gained those yards. But sometimes YAC comes because the QB leads the receiver into open space with the throw. A bad QB may lead the WR into a big hit but the great QB sees that big hit lurking and forces the WR to stop and that leads to more YAC.

 

[obxsteeler]

Terry Bradshaw said the thing he valued most in his career was that only he and Montana had won 4 SB's without a loss. Second was that he had made a big play in the 4th Q of each. Really might be the ultimate rank.

 

[deljzc]

Remember, I designed the new formulas to replace the traditional quarterback rating system which only uses the following criteria (about equally ranked):

Completion %
YPA
Int%
TD%

I wanted to keep the basic idea of this being a "statistic", thus you have to be able to gather the information from the game book stats. It's not subjective. It's not situational (although you can measure things like a quarterbacks DQR in 4th quarters when within 7 points or something like that).

I tweaked the 4 criteria above to things that just make more sense to me:

1. Completion % stayed the same.
2. YPA turned into "Total Yards per Pass Play" and includes scramble yards, most rush yard (I take out sneaks and kneeldowns as best possible) and divides them between ALL attempts (including sacks).
3. INT% turned into Turnovers/Pass Attempt and now includes fumbles.
4. TD% turned into Points per Possession. This is obviously the most controversial, but in my opinion, the quarterbacks job is to get into the endzone however possible. I don't want to only reward him for TD passes. Yes, there are drives that are just runs and score, but there are also drives that are mostly passes and then get run in from the 1 yards like. I think those cancel each other out over the course of a season.
5. I added a minor category (it counts about half as much) to try and make a negative score for sacks. Instead of just measuring sacks, I thought it was reasonable to say "Sacks are okay if the QB/offense was aggressive downfield", so I created the idea of Big Plays vs. Sacks and then measured this ratio per Pass Play.

I consider this statistic a measure of quarterback play, not the quarterback himself. We all know there is no stat in a gamebook that fully represents a quarterback's ability because there are just too many variables that he does not control: offensive system, coaching, play calls (sometimes), surrounding talent and execution, defense against (and all the variables that brings).

I still believe the best way to value a quarterback is scout him and watch film.

What this number represents is how the quarterback is working with all those variables around him. In some the score might reflect bad coaching or surrounding talent or tough opponents. I am of the opinion, if a teams starts to drift below the half way point (around 85.0), then you have to ask yourself "How can we improve" and that answer might be change the quarterback or it might not. That is up to you scouting correctly and making changes correctly.

 

[ark steel]

speaking of weighting, i always thought that there should be situational weighting for QBs. If you are down by 30 then passing yards come much easier. Turnovers in the redzone should be weighted more. Then there's situational awareness. Does a QB throw a deep incompletion on 3rd and 4 when he could have moved the chains? Does a QB take a sack that moves them out of FG range?

How do you factor in yards after catch for a QB? The first thought is to remove them from the equation because it's the receiver who gained those yards. But sometimes YAC comes because the QB leads the receiver into open space with the throw. A bad QB may lead the WR into a big hit but the great QB sees that big hit lurking and forces the WR to stop and that leads to more YAC.

I'd almost bet that there is a PFF 'premium' stat that purports to come close to this, or something like it. As del says, there would be a lot of subjectivity involved. As has been stated here with PFF, we have no idea who the graders are, there ability to properly diagnose what is going on or how the final stat is calculated. Or even their objectivity.

A couple of experiments would be interesting to see. 1) Have 3 or 4 guys independently watching the same game and rating each play like you state to see what differences arises. 2) Do something similar (for 10-15 plays, not a whole game), but somehow blocking out the ability to see the team and/or players.

 

[ark steel]

Remember, I designed the new formulas to replace the traditional quarterback rating system which only uses the following criteria (about equally ranked):

Completion %
YPA
Int%
TD%

I wanted to keep the basic idea of this being a "statistic", thus you have to be able to gather the information from the game book stats. It's not subjective. It's not situational (although you can measure things like a quarterbacks DQR in 4th quarters when within 7 points or something like that).

I tweaked the 4 criteria above to things that just make more sense to me:

1. Completion % stayed the same.
2. YPA turned into "Total Yards per Pass Play" and includes scramble yards, most rush yard (I take out sneaks and kneeldowns as best possible) and divides them between ALL attempts (including sacks).
3. INT% turned into Turnovers/Pass Attempt and now includes fumbles.
4. TD% turned into Points per Possession. This is obviously the most controversial, but in my opinion, the quarterbacks job is to get into the endzone however possible. I don't want to only reward him for TD passes. Yes, there are drives that are just runs and score, but there are also drives that are mostly passes and then get run in from the 1 yards like. I think those cancel each other out over the course of a season.
5. I added a minor category (it counts about half as much) to try and make a negative score for sacks. Instead of just measuring sacks, I thought it was reasonable to say "Sacks are okay if the QB/offense was aggressive downfield", so I created the idea of Big Plays vs. Sacks and then measured this ratio per Pass Play.

I consider this statistic a measure of quarterback play, not the quarterback himself. We all know there is no stat in a gamebook that fully represents a quarterback's ability because there are just too many variables that he does not control: offensive system, coaching, play calls (sometimes), surrounding talent and execution, defense against (and all the variables that brings).

I still believe the best way to value a quarterback is scout him and watch film.

What this number represents is how the quarterback is working with all those variables around him. In some the score might reflect bad coaching or surrounding talent or tough opponents. I am of the opinion, if a teams starts to drift below the half way point (around 85.0), then you have to ask yourself "How can we improve" and that answer might be change the quarterback or it might not. That is up to you scouting correctly and making changes correctly.

Thanks for the explanation. I agree that trying to incorporate all of the variables would be too much and introduce too much subjectivity.

You may be right about the TD's. After a quick skim, only 7 teams didn't have a pretty good gap between the TD's by pass over TD's by rushing. Unless I missed something, these are the Bengals, Browns, Chiefs, Vikings, Jets, Seahawks and Redskins.

 

[deljzc]

Thanks for the explanation. I agree that trying to incorporate all of the variables would be too much and introduce too much subjectivity.

You may be right about the TD's. After a quick skim, only 7 teams didn't have a pretty good gap between the TD's by pass over TD's by rushing. Unless I missed something, these are the Bengals, Browns, Chiefs, Vikings, Jets, Seahawks and Redskins.

A field goal isn't always a bad result either (that's 3 points per drive) and that isn't really reflected at all in the old formula.

Sometimes I just think a good quarterback finds ways to keep the chains moving more than a bad quarterback. How they accomplish it varies. They might scramble, they might audible to the right play or run the correct direction based on how he reads the defense. The better quarterbacks have more "check-with-me" allowances and that should reflect in more successful player, more first down and more points per drive.

Again, if there is a problem with these things it's not ALL on the quarterback, but he certainly bares some of the blame as well because the good ones seem to find a way to get it done. Maybe it's experience. Maybe the coach is being too controlling. But those ARE things to be worried about if I was evaluating a quarterbacks performance.

 

[Litos]

nice work, thanks for sharing

 

[deljzc]

I will add this:

I do not like the move towards taking yards after catch away from measuring quarterbacks. I think there are too many offenses that build yards after catch into their offensive systems and good quarterbacks are expected to "lead" receivers into areas that allow them to gain those yards.

I think this is just as important a quarterback skill as just "throwing it up for grabs" on a bomb and having a very skilled receiver out-jump a defender and make the play. I don't see how that type of play reflects of some great, inherent quarterback skill. But according to ESPN and their rating system, throwing it up on a bomb for 40 yards is a better skill than leading a receiver on a 20-yard deep in perfectly and having the receiver gain 20 yards after the catch. I don't really see the difference. Both take some degree of receiver talent to succeed. To try and pick and choose when a receiver helps "too much" on a yard gained by a pass is WAY too subjective for my liking.