Revised Player Rankings

A few weeks ago I posted my initial work on developing a player rankings system which got mixed reviews among the more theoretically-based, statistically minded crowd. I am still not going to go down the road of applying pure statistical theory to developing a ranking system (such as what Javageek is doing) but I have significantly rejigged my algorithm and also incorporated powerplay and penalty killing time into the process and I think I have come up with some fairly good player rankings. I have decided not to divulge the secrets of my player ranking algorithm but I have decided to (for now anyway) post the rankings of all the players in the NHL.

The algorithm produces four numbers for each player: an offensive rating, a defensive rating, and overall rating and an overall contribution. The offensive, defensive and overall ratings are ice time independent. By that I mean that players who play less ice time don’t get penalized when compared to players that get more minutes. Someone who plays a lot of minutes on the power play will get a better opportunity to rack up goals and assists and this has an advantage when it comes to scoring more points but not when it comes to getting a better offensive rating. My algorithm accounts for that. When looking at the offensive, defensive and overall ratings consider a rating of 1.00 to be the average NHLer. Ratings less than 1.00 are below average and ratings over 1.00 are above average.

The other number produced is an overall contribution statistic. This overall contribution combines a players offensive, defensive, and overall ratings with a players ice time (not quite overall rating*icetime but close) to get a number representative of what the player has contributed to his team over the course of the season. This statistic isn’t the best when evaluating who is the best player since a lesser player playing more ice time on a worse team has the potential to rank better than a better player playing less ice time on a better team (or due to injuries), it does do a good job at evaluating which player has contributed the most to his team. So while this may not be the best statistic in evaluating who is the best player in the NHL, it might be a good statistic in evaluating who has contributed the most to his team and thus might be a good starting point for evaluating the MVP and Norris trophy candidates.

Ok, before we start looking at some actual numbers, let me say that I have produced results for the second half of last season and also results for the current season and have compared the two. Using a requirement of a player having to have played 300 minutes of ice time to get a valid rating for the season there were 318 players who had a rating for both last season and this season and I did a quick correlation calculation on these players. The offensive ratings produced a correlation coefficient of 0.44, the defensive ratings produced a correlation coefficient of 0.13, the overall ratings were 0.37 and the overall contribution was 0.52. These correlations are generally much better than those I got for my algorithm of a couple weeks ago and I am somewhat satisfied with them. Many of the players who have significant differences between last year and this year are understandable because by no ones evaluation would you say they are having a similarly good or bad seasons. For example, the player with the biggest improvement in overall rating is Thomas Vanek and the player with the biggest drop in overall rating is Jonathan Cheechoo. This makes total sense. Vanek has been awesome this season (16 goals, 30 points, +16) while Cheechoo was awesome in the second half of last year and pretty ordinary this season. That said, I am a little perplexed as to why the defensive ratings don’t produce better correlations. When developing this algorithm I went through several steps of progress and at every step the defensive ratings produced correlations significantly below that of the offensive ratings. And yet when I look at the ratings I can seem to justify why each player gets a good or poor result. It is strange because essentially the offensive and defensive ratings are produced in the exact same way, except opposite in the sense of what is good (producing goals vs stopping goals). I am hoping that as the season goes on that the correlations for all the these ratings improves. In another month or so I’ll maybe revisit the correlations and see if anything has changed.

Ok, so on to some results. Player ratings for every player in the NHL can be found by clicking the division links in the menu on the left. I have also included a page with the top 20 rated players for each rating. Take some time to browse through them and let me know what you think.

Now, I know that there will be some players ratings that will create some controversy so let me address some of them now.

Defensive Ratings in general: As I explained above, defensive ratings are a bit perplexing because they don’t correlate well with last years defensive ratings but also because guys like Lidstrom (0.91) isn’t rated that well when he has the league best +/-. But there is an explanation for it. That is, the Red Wings have one of the worst penalty kill percentages in the NHL and considering that Lidstrom gets a lot of ice time killing penalties his defensive rating gets negatively impacted. I think this is perfectly fair. But there are other examples, such as Pronger, where explaining his 0.90 defensive rating is more difficult since the Ducks have a pretty good penalty kill so his defensive rating might be a bit of an anomaly this early in the season. Incidently, Lidstrom had a 0.96 defensive rating last year and Pronger had a 1.11 defensive rating so Lidstrom is right in line with last year and Pronger is a bit low. I would expect to see Pronger’s defensive rating rise a bit over the course of the season.

Tom Preissing – There are some in the media and among the fans that feel that Preissing has been a disappointment but I find this strange. He has a respectable 10 points in 24 games and easily has a team best +13 rating. Maybe the reason fans are disappointed is because he is only getting 14 and a half minutes of ice time. But don’t blame that on him as in those 14 and a half minutes he has been pretty effective in both producing offense and not letting many goals in.

Anders Eriksson – Somehow this guy managed to get the 12th higest overall rating which doesn’t make a lot of sense. A lot of it is driven by the fact that he is a +3 on a weak Columbus team but being rated that highly doesn’t make much sense. I expect him to drop over the course of the season. But having overly high rated defensemen is par for the course for the Blue Jackets as Ron Hainsey was rated quite highly last season but has dropped to more expected levels this year.

Ryan Getzlaf – Getzlaf is an interesting player to look at because he ranks very highly in the offensive rankings with a 2.18 offensvie ranking. That might strike you strange for a guy who is 6th on his team in points. But it doesn’t seem to be an anomaly as he had a 1.95 offensive rating last season as well. He’s doing something right in the 13 minutes of ice time he gets.

New York Rangers – The trio of Jagr, Straka and Nylander lead the league in overall contribution. It’s no surprise really considering all three are in the top 15 in scoring and top 10 in +/- but it makes you wonder why the Rangers are just 4 games over .500. Inconsistent goaltending seems to be the answer to that (Lundqvist .900 save% is not good enough).

Goaltending- Speaking of goaltending, you may have noticed that I have taken goaltenders out of the rankings. The reason for this is because I intend to develop a goaltender specific ranking systems because although I think this processed used for developing these rankings would do a decent job on goalie rankings I think I can produce better results with a goalie specific algorithm.

Anyway, that’s all for now. I’ll let you browse through the rankings yourselves and I away all of your comments, suggestions and criticisms. I look forward to hearing them.

This article has 14 Comments

  1. You might graph your results as a quick sanity test for your algorithm. If you get a “long tail” (or whatever you want to call it) you’re probably on the right track. I’m no mathematician or statistician but when ranking the ability of a common set of items (athletes, students, businesses, etc.) it seems to me that you would get that type of downward sloping distribution with most being in the “tail” and rated as near average.

  2. “That is, the Red Wings have one of the worst penalty kill percentages in the NHL and considering that Lidstrom gets a lot of ice time killing penalties his defensive rating gets negatively impacted. I think this is perfectly fair.”

    Some quick calculations:

    – The Wings have allowed .93 shots on goal per time short-handed. League average is 1.26. So the Wings have allowed 43.8 fewer shots than an average team would have in the same number of times short-handed. The league-average SV% while short-handed is .867. So with average goaltending, the Wings’ penalty kill would be performing at 5.8 goals above expectation, tops in the NHL.

    The reason for their poor overall numbers is that Wings goalies have only a .783 SV% on the penalty kill. Now, there might be a shot quality issue here, driving down the save percentage through no fault of Hasek/Osgood/MacDonald, but regardless, I find it very hard to believe that the Wings penalty killers have been subpar.

  3. I have yet to be convinced that shots are a good proxy for evaluating how good a team or player is but regardless of that you are using the wrong statistic. You shouldn’t look at shots per time short handed, you should look at shots per minute short handed. Teams with poorer penalty kills are inherently going to be playing fewer minutes short handed per penalty because fewer penalties have the full 2 minutes played. Then there is the whole issue short penalty kills because the opposing team takes a penalty cancelling the short handed situation. We don’t know how many 25 second penalty kills the Wings have had. It’s possible they have had more than other teams.

    But, let me say, my player ranking system does factor in the goalie so unless for some unknown reason Red Wing goalies play worse on the PK than even strength I’ve correctly (as best I can) adjusted for the goalie influence.

  4. “You shouldn’t look at shots per time short handed, you should look at shots per minute short handed.”

    Of course. But that data isn’t readily available (at least not without working through the play-by-play files – correct me if I’m wrong). And of course you’re right that the Wings are playing fewer minutes per time short-handed than most teams, but I doubt that it makes much of a difference in this case. The Wings penalty kill is 4.2 goals worse than average this year… how many minutes of PK time can that have cost them? No more than a handful.

    “unless for some unknown reason Red Wing goalies play worse on the PK than even strength I’ve correctly (as best I can) adjusted for the goalie influence.”

    That’s exactly what I’m suggesting. Wings goalies are above average in even strength save percentage, and absolutely horrendous in short-handed save percentage. I do recognize that shots allowed is far from a perfect way of evaluating defense, but it seems a much better solution than assuming that the Wings’ penalty killers are so bad that an average goalie would have a save percentage around .800 playing behind them.

  5. I guess there are 3 reasons why Detroit’s goalies have a worse save percentage on the PK.

    1. Detroit penalty killers give up more high quality shots.
    2. It’s early in the season and bad bounces are the difference.
    3. Detroit goalies suddenly play worse as soon as the PK begins.

    Now to me the most reasonable answer is #1. #2 is certainly feasible and if it is the case we should see the goalies save percentages improve as the season goes on. #3 doesn’t make much sense. I just can’t think of an explanation of why a goalie might suddenly get worse when their teams gets a penalty and suddenly get better when the penalty is done.

  6. #2 makes the most sense to me, by far. The Wings’ goalies have faced 120 shots while shorthanded; anything can happen over a sample that small. I’d say that the Wings’ goalies have just happened to play worse while on the PK, in the same way that Mike Smith just happened to stop 111 of the first 117 shots he faced this year (when his true ability level is almost certainly nowhere near that).

    Small-sample flukes like that happen. But given that we have to assign the blame to someone when doing these sorts of player rankings, I think it makes more sense to blame the goalies than to blame the PK unit (unless, of course, there’s actual data supporting the hypothesis that the Wings PK is surrendering higher-quality shots).

  7. But by the same token, if Wings goalies are overperforming against their expectation on even strength, Lidstrom is receiving the benefit of that as well.

    It is still early in the season – the goalies’ performance should normalize over the course of a season.

  8. The Wings give up fewer shots than any team in the League. Perhaps the reason PK shots get through on their goaltending is the increased regularity with which they occur. More rebound chances, one less man in front to clean up the garbage, etc.

    It would be nice if there were a way to determine what KIND of shots are getting through on the Wings goalies when they’re a man down. I sincerely doubt they’re higher quality shots. Ugly rebound goals, and mad scrambles in front of the net would appear to me to be the most likely culprits, especially since the Wings D isn’t amazingly large, and their advantage of mobility and puck possession would be reduced when down a man.

    Either way, I think this whole debate is a bit irrelevant in comparison to the larger question of how your Defensive rankings show such a low correlation from season to season. While it is still early in the year, it must be a bit disconcerting to think that players Defensive abilities fluctuate wildly from year to year, that doesn’t make a lot of sense to me. I would think it indicates that there’s a flaw in the algorithm you’re using somewhere… but since you’re keeping that quiet I guess we’ll have no idea where.

  9. Essentially the defensive algorithm is the reverse of the offensive algorithm so it seems odd that one would correlate and the other wouldn’t if there was an issue with the algorithm. I could be wrong though. I have a few ideas on how to improve the algorithm which I hope to test over the next week or so. I am hoping that that plus having more games played would improve things a bit.

  10. Defence is a team first process. If any one player loses his check, then a goal is quite likely. In your algorithm it penalizes all players on the ice. One individual player who is great defensively cannot keep his teammates from making a mistake.

    Offence is more of an individual pursuit. If one player makes a great pass or a great rush he has a great chance to score. Again in your algorithm it benefits all players on the ice. But at least it definitely captures the success of the goal creators.

    It is far easier to isolate good offensive players by the nature of hockey because offense is much more individualistic.

  11. I suspect that defense being more of a team thing is part of the issue but even in my algorithm a player who doesn’t make mistakes should still see their defense ratings higher than those that do. My guess (or hope) is that you just need a larger data set to isolate the differences between players defensively than offensively.

  12. In principle, the better defensive player should rise to the top with a large enough data set, but I suspect that data set might be hundreds or even thousands of games (I have little to back up that claim other than the random seeming results that come from one season or less). If it requires an unreasonably sized sample, then its bound to fail.

  13. Another point… which I admit you’ve previously made reference to and tried to incorporate, would be the defensive effects of stellar goal tending.

    A team with a goalie capable of standing on his head will inevitably seem superior defensively to one that lacks such a goalie. Team D can limit chances and make average goalies seem like all-stars (i.e. Manny Legace, Patrick Lalime), but a stellar goaltender can compensate for a lot of defensive faults on a sub-par team (i.e. Luongo in Florida over the past few years, Belfour/Joseph in Toronto for a number of seasons previously, Lundqvist in NY).

    I don’t know how you could factor OUT such an impact and isolate individual defensive skills. I think it could explain wide variation in defensive ratings from year to year. Any player on Atlanta would benefit from having Lehtonen in for the majority of play this year. As would any player on Vancouver from the pickup of Luongo. Similarly, LA might be seriously suffering from the addition of Cloutier as their starter, and Ottawa could be taking a hit on the exchange of Hasek for Gerber in the early part of the year.

    I know you’re trying to account for it but I think it’s just too much of an overall team dynamic to eliminate the effect entirely.

Comments are closed.