One of my goals has always been to create a fair and true player evaluation model. Myself and many others have attempted to do this to varying degrees of success. The recent trend is to stray away from goals for and against type methodologies towards shot attempts for or against methodologies (i.e. corsi analysis). One of the key reasons for doing so is to increase sample size to reduce errors in the evaluation including errors that might be associated with nothing more than pure luck. There is merit to wanting to achieve this goal and to some extent looking at shots attempts for/against achieve this goal but you will also find the method fail in many situations due to the fact that it treats all shots equal, which we know is not true. Some shots are simply better than others and more likely to produce goals. A prime example of where corsi analysis fails is with the Toronto Maple Leafs of recent years. Recently the Leafs have produced a ton of shots but relatively few goals in large part because while they have had quality players to move the puck up the ice and get opportunities they haven’t had the elite level offensive players to convert those opportunities into great scoring opportunities. Corsi Analysis can make many Leaf players appear like top level players and make the team itself appear to be an average or even better than average team but in the standings this clearly hasn’t been the case.
Because of this I still prefer to consider a goal based approach because really, goals are what matter in hockey. If you are on the ice for more goals for than goals against, that is a good thing. The stat that measures this that everyone knows and understands best is plus/minus but it has major drawbacks. A below average player that plays on a great team with great teammates can have a very good plus/minus while a great player that plays on a bad team with below average teammates can have a poor plus/minus. Furthermore, a defensive specialist that goes head to head with the oppositions best offensive players may also have a poor plus/minus but still be getting the job done for the most part. The standard plus/minus stat is, in general, a very poor stat for evaluating players.
To improve on straight forward plus/minus we need to take into consideration the quality of teammates/linemates a player has along with the quality of the opposition players he plays against. One way to account for his teammates is to look at team statistics when that player is on the ice vs off the ice. If a team has better statistics when he is on the ice than when he is off, his play has seemingly made the team better. For the most part this works, but it fails in certain situations as well. In particular, it fails when a team sticks mostly to set lines and the discrepancy in lines is significant. One example I recall is a couple years back on ice/off ice adjustments made Kris Draper look downright awful but in reality he wasn’t nearly as bad as it was portrayed to be. The problem is, when Draper was on the ice he was playing with Kirk Maltby and Dallas Drake neither of which were very good but his off ice comparisons were against Datsyuk, Zetterberg, Lidstrom, Rafalski, Holmstrom, Franzen, Cleary, Hudler, etc. none of whom Draper got to play much with. The problem was Draper’s off ice comparables were elite level players but his own stats never really benefitted from playing with them. Conversely, a team like the Islanders had bad players throughout their lineups so an equivalent player on that roster had a much weaker ‘off ice’ comparable. When dealing with really good and really bad teams, the team on ice/off ice method for adjusting for quality of teammates failed in a similar way to how corsi analysis fails at the extremes.
With this in mind, when I developed my all encompassing player evaluation method I choose to consider line mates performance levels when that line mate is playing with the individual being evaluated and when he is not playing with the player being evaluated. Similarly, when evaluating strength of opponent I chose to consider each opposition players performance levels when he is not playing against the player being evaluated. I always consider production on a per 20 minute of ice time basis. The following is the formula for calculating the defensive rating of Player A.
DefRating(A) = ExpectedGAper20(A) / GAper20(A)
GAper20(A) = The goals against per 20 minutes while Player A is on the ice.
ExpectedGAper20(A) = (TMExpGAper20[A] + OppExpGFper20[A]) / 2
TMExpGAper20[A] = SUM [ TMGAper20Apart(A,B)*TOI(A,B)] / SUM[TOI(A,B))]
OppExpGFper20[A] = SUM[OppGFper20Apart(A,B)* TOI(A,B) ] / SUM[TOI(A,B)]
TMGFper20Apart(A,B) = Goals Against per 20 minutes by teammate B when teammate is not playing with player A.
OppGFper20Apart(A,B) = Goals for per 20 minute on ice by opposition player B when opponent is not playing against player A
TOI(A,B) = The amount of time that player A and B are on the ice together (or against each other)
Similarly an offensive rating can be calculated using the following formula:
OffRating(A) = GFper20 / ExpectedGFer20(A)
GFper20(A) = The goals for per 20 minutes while Player A is on the ice.
ExpectedGFper20(A) = (TMExpGFper20[A] + OppExpGAper20[A]) / 2
TMExpGFper20[A] = SUM [ TMGFper20Apart(A,B)*TOI(A,B)] / SUM[TOI(A,B))]
OppExpGAper20[A] = SUM[OppGAper20Apart(A,B)* TOI(A,B) ] / SUM[TOI(A,B)]
TMGFper20Apart(A,B) = Goals for per 20 minutes by teammate B when teammate is not playing with player A.
OppGFper20Apart(A,B) = Goals against per 20 minute on ice by opposition player B when opponent is not playing against player A
An overall rating can then be calculated by averaging the offensive and defensive ratings. It should also be mentioned that when I consider offensive I do not include goalies in the calculation (i.e. I assume that goalies have no influence on offense). In short, I do not include goalies in the summation of goals for per 20 minutes in either the offensive or defensive rating calculations.
There are a couple of things I like about this model:
- One model can be applied to every player, regardless of position. Ratings for forwards, defensemen and goalies (only defensive rating applies to goalies) can be calculated using the same formula.
- Because we are using a ratio of actual performance vs expected performance we get a nice result where numbers greater than 1 indicates a player better than average and a number less than 1 indicates a player below average.
- It is relatively easy to understand. The concept of ‘does player B perform better or worse when playing with/against player A’ is a straight forward one. All we do here is calculate a time on ice together weighted average of that performance difference across all players who played with/against player A.
I have struggled with whether the defensive rating for goalies using this method does in fact produce a reasonable assessment of the goalies performance but I have been reasonably happy with the results I have seen so far.
Here are a few results to consider but over the next few days/weeks I’ll delve into the numbers in greater detail. Here are the top 10 offensive forwards for last season with a minimum of . Note: For now I am only considering 5 on 5 even strength performance.
DANIEL SEDIN, 2.154
HENRIK SEDIN, 2.097
SIDNEY CROSBY, 1.889
NATHAN HORTON, 1.639
MARIAN GABORIK, 1.635
STEVE DOWNIE, 1.608
NIK ANTROPOV, 1.603
PATRIK ELIAS, 1.58
CHRIS STEWART, 1.558
The bottom 5 offensive forwards are Stephane Veilleux (0.435), Nate Thompson (0.482), Steve Begin (0.582), Todd Marchant (0.599) and Jeff Halpern (0.600).
Here are the top 10 defensive forwards.
DREW MILLER, 1.867
KYLE WELLWOOD, 1.765
MANNY MALHOTRA, 1.654
DUSTIN BOYD, 1.603
CURTIS GLENCROSS, 1.572
RADIM VRBATA, 1.545
TAYLOR PYATT, 1.506
TRAVIS ZAJAC, 1.504
MICHAEL RYDER, 1.493
NICK FOLIGNO, 1.475
The worst 5 defensive forwards are: Malkin (0.648), Afinogenov (0.679), Brad Richards (0.701), Brind’amour (0.704) and Patrick O’Sullivan (0.709). It’s interesting that 2 of those guys are no longer in the NHL (Brind’amour retired and Afinogenov is off to Russia) and a third (O’Sullivan) is an unrestricted free agent who can’t find a contract.
The top offensive defensemen with 750 minutes of 5v5 ice time are Green, Markov, Boyle, Jeff Schultz, and Ehrhoff while the top 5 defensive defensemen are Mark Fistric, Mark Giordano, Mike Weaver, Keith Yandle and Jeff Schultz.
The top five goalies with at least 1000 5v5 minutes are Tuukka Rask, Ray Emery, Miikka Kiprusoff, Henrik Lundqvist and Tomas Vokoun. The bottom five are Michael Leighton, Pascal Leclaire, Ondrej Pavelec, Tim Thomas and Jeff Deslauriers.
Now before you all jump all over me about how one of the players on the list should not be on the list or someone not on the list should be or that there is no way Kyle Wellwood is that good defensively let me point out that you may be right. This analysis is still far from perfect. Specifically, it still suffers from the small sample size issues that the people witnessed and switched to corsi analysis to attempt to resolve. There are other issues too so in the coming days I plan to present two improvements that hopefully make things a little better and I’ll also get the ratings up for all the players so you can see how everyone stacks up.