Nov 202008
 

James Mirtle from From the Rink and the Edmonton Journal’s David Staples are questioning the usefulness of the +/- stat and maybe abandoning it altogether. I have wrote about this here before and I have to agree, it is the most useless stat in the NHL. The big flaw in it is that a player gets penalized if he plays with a group of bad players or gets a big benefit if he is playing with a group of really good players, regardless of how good or bad the individualy player is.

James Mirtle goes on to refer to sites such as Behind the Net which has better stats such as on ice vs off ice comparisons. But these stats, while significantly better than +/-, are still flawed. The idea is that by comparing a players +/- (or goals for or goals against individually) when he is on the ice to his teams +/- when the player is off the ice you will factor out some of the unfair benefits or penalties a player receives in his +/- stat based on who the player plays against. But even this doesn’t really solve the problem.

The problem still exists because Chris Draper is having his on ice stats compared to off ice stats that are racked up by guys like Zetterberg, Datsyuk, Holmstrom, etc. why a guy like the Islanders Trent Hunter has his on ice stats compared to Mike Comrie and Mike Sillinger. Chris Draper might be a very good defensive player, but when you compare his +/- to the rest of his all-star stacked team mates, most of whom he doens’t get the benefit of playing with, his on-ice to off-ice ratio stats become misleading. The goal of Behind the Net’s ratio numbers is to eliminate the unfair benefit of playing on a great team. The problem is, if a player who plays on a great team, such as Draper, never gets to play with his great team mates, like Datsyuk, Zetterberg, Holmstrom, Franzen, Cleary, etc. and instead plays with pretty ordinary players, such Maltby and Drake, then you end up factoring out a benefit that never existed in the first place.

When considering all players who have played 60 games with at least 10 minutes of even strength ice time per game last season, David Perron, Viktor Kozlov, Dany Heatley, Michel Ouellet, and Pavel Datsyuk came out as the top 5 rated forward in Behind the Nets ‘Rating’ statistic which is the players +/- rating compared to the team. The bottom 5 were Radek Bonk, Alexander Semin, Kris Draper, Jarret Stoll and Mike Fisher.

Now, can you honestly believe that Kris Draper, who had the most short handed minutes of any Red Wing player is the second third worst defensive player in the NHL, especially considering that the Red Wings had an 84% penalty kill success rate? Would the Ottawa Senators use Mike Fisher, supposedly the 5th worst rated player in the NHL, for nearly 3 minutes a game on the PK? Would the Edmonton Oilers have used Jarret Stoll, the 4th worst rated player according to Behind the net Rating, for the same amount of time on their PK unit? Conversely, if Ouellet is such a good player, why did he get zero PK time last year? Same with Perron on the Blues and Kozlov on the Capitals.

Clearly Behind the Net’s on ice/off ice +/- ratio ratings system is still failing to really tell us the whole story even if it is probably a bit better than straight +/-.

A year or so ago I worked on developing my own ratings system which admitidly is far more complicated than +/- or Behind the Net’s ratio system and while still far from perfect I am confident is much more revealing. What I have done is taken into account who a player has played with and against and tried to factor out any benefits they might have by playing with better than average players or against worst than average players or penalties they may get by playing with worse than average players or against better than average players. Instead of looking at how his team plays when he is off the ice vs when he is on the ice, I look at how his teammates play when they are playing with him vs how they play without him. The result is a player will get a good rating when he consistently makes his team mates better when they play with him vs when they aren’t playing with him. I also take into account PP and PK time which I think is important when evaluating a players value.

The results can be seen at Stats.hockeyanalysis.com and select the links under ‘Player Rankings”. I calculate both an offensive and defensive rating plus an overall rating which is independent of ice time and then an overall value, which factors in ice time. A player with a high rating and plays lots of ice time will get a really good value. In the ratings, a rating of 1.00 is approximately average while anything above is above average and anything below is below average.

In my ratings, Kris Draper had a 1.23 defensvie rating and a 0.98 overall rating which is far from the bottom 5 in the NHL. When compared to his all-star team mates, Draper doesn’t look to good, but when compared to the average NHLer, Kris Draper can more than hold his own. Mike Fisher had a 0.93 defensive rating and a 0.94 overall rating and Stoll had a 0.92 defensive rating and a 0.90 overall rating which are both a bit below average but are both far from the worst in the NHL. Interestingly, Ouellet and Perron and Kozlov still do quite well in my ratings so it seems Behind the Net’s flaws might be more significant at the bottom of the list than at the top.

I need to point out that my ratings are still not perfect. I still haven’t found a way to successfully factor out the impact of the goalie and players who play on teams with a bad goalie may have their defensive stats overly impacted in a negative way and players on teams with great goalies will have their defensive stats biased to the positive side). Also, the more a team juggles their lines the better and more accurate my ratings should be because it makes for better with team mate, with out team mate comparisons. But overall I believe they are reasonable and I personally have more confidence in them than Behind the Net’s on ice/off ice ratings comparison. At some point I hope to revisit my ratings system and see if I can improve it further.

  10 Responses to “The Useless +/- Stat”

  1.  

    sometimes i wonder if a simple takeaways/giveaways ratio might be a better measure of a player’s defensive play.

  2.  

    The problem with that is I do not believe takeaways/giveaways is a reliable stat. It is a stat based on interpretation and I can show flaws and inconsistencies with how game monitors hand out takeaways and giveaways. Plus good players who have the puck more are more apt to give away the puck just because they have more opportunities to do so. In basketball they have the assist to turnover ratio which is an attempt to identify how well a player is at passing the ball around with out it getting intercepted. That isn’t bad but still doesn’t necessarily tell the whole story and in hockey assists wouldn’t even come close to being representative of the total number of passes that occur in a game.

  3.  

    Bang on about that David. Really good read and I am looking forward to looking into your system.

    Giveaways and takeaways are VERY susceptible to interpretation of the rink officials, as I have multiple instances this year only of players I have watched make a blatant g/a, been benched and have nothing show up on the official NHL scoresheet. The odd time would be acceptable, but I think that sometimes these guys/gals are standing in line for a soda….

  4.  

    You guys are right about the stat watchers in different rinks being… unreliable. But the league hasn’t really been recording all these non-traditional stats for all that long, have they? The league can change that easily with increased training, standardization, etc.

    David, you and everyone else who have been looking critically at the +/- stat are totally right. But to suggest that a better solution is to dance around that fact and use a super complicated mish-mash stat, based fundamentally on +/-, is something that I consider to be a bit of a wasted exercise. Eventually you end up with a stat that we are still dubious about but cannot understand how it works enough to figure out why we’re dubious about it!

    If I’m playing in the beer leagues with my mates, I know that if I steal a puck or intercept a pass more often than I give away a puck to the other team, then my net defensive effort is positive for my team. You could even incorporate a negative influence for every “defensive” penalty that I take (the hooks, holds, and trips), because I am taking those penalties when I am otherwise unable to take the puck from the opposing player. The league is increasingly more and more dedicated to advancing its statistic tracking. They’re improving every year. If the league leadership can be convinced that tracking giveaways/takeaways more accurately would help the fans (and their own GMs) to understand the effectiveness of players, then they will put the effort into making sure that those stats are recorded more accurately in the future.

    The method outlined in your blog here is a good attempt at making the best out of a poor situation. But the great thing about the blogosphere is that if we make a ton of noise about something, eventually someone will hear about it then the actual problem can be addressed.

  5.  

    David, you and everyone else who have been looking critically at the +/- stat are totally right. But to suggest that a better solution is to dance around that fact and use a super complicated mish-mash stat, based fundamentally on +/-, is something that I consider to be a bit of a wasted exercise. Eventually you end up with a stat that we are still dubious about but cannot understand how it works enough to figure out why we’re dubious about it!

    I disagree. The concept of +/- is a good one. The two most important things in hockey are how many goals you allow vs how many goals you score. The problem with +/- is not that it is an inherently bad thing to measure, but that it doesn’t measure an individuals performance but rather the performance of the individual, all his linemates, the goalie, and his opponents. The result is an even +/- on Atlanta may very well be better than a +12 on Detroit. Even on a team, Khris Draper’s -2 might be better than Thomas Holmstrom’s +9 simply because Holmstrom has unfairly benefitted from playing with Datsyuk and Zetterberg.

    But, if we can somehow isolate a players performance then +/- is, at least conceptually, a good stat. What I have done is instead of looking at how a player compares to his team as a whole, I compare individual players to each other when they are on the ice together vs when they are on the ice apart from each other.

    If Draper’s teammates generally give up fewer goals when they are playing with Draper than when they are not, then that seems to indicate that Draper is a positive influence on their defensive game.

    Go to stats.hockeyanalysis.com and click on Player Performance with and Against other Players in the side menu. This will give you a list of all the players in the NHL with their goals for and against while they are on the ice, their +/- and their per 20 minutes of even strength ice time stats. Now click on the players names and you will find goals for and against stats when playing with and against other players and you can see how they do on and off the ice. These numbers are the essential basis of my calculations. If you look at Kris Draper he makes several of his team mates better in terms of goals against but most importantly opponents generally score fewer goals when they are playing Draper than they do when they play anyone else.

    The only thing I have yet to do is sufficiently factor out goaltending. This is the hardest thing to do, particularly for teams that play mostly one really good goalie such as the Devils. Players on these teams will generally get elevated defensive ratings.

    The method outlined in your blog here is a good attempt at making the best out of a poor situation. But the great thing about the blogosphere is that if we make a ton of noise about something, eventually someone will hear about it then the actual problem can be addressed.

    It isn’t going to happen. Maybe people will start quoting +/- as being such a good indication of defensive performance but I doubt we’ll get anything better that will be easily digestible by the league, hockey announcers, main stream media, and fans.

  6.  

    I’ll concede. You’ve obviously put a tonne of thought into this subject.

    What about adjusting for a team’s coaching style, or do you think that already gets taken care of when you compare vs a player’s teammates?

    An idea just occurred to me about how one might factor out the influence of goaltending. How about using shots for and shots against?

  7.  

    It is clear that coaching does have an impact on the game and one could presume that some players skill set would be better utilized by a certain system than others. But that is really hard, if not impossible, to account for. How can I say whether Gaborik would perform better overall in an offensive minded system. Yes, he will probably score more goals and produce more offense, but will his defense get hurt? Who knows.

    As for goaltending, the problem is a challenging one. The easy solution is to look at save percentage and use that as a measurement for how good or bad a goalie is. In large part, you are likely to get pretty good results by just doing this. But every time I say that ‘Toskala is a sub par goalie because his save percentage isn’t very good’ someone will come by and say ‘yeah, but the Leafs defense is horrible so he faces tougher shots than most goalies.’ How do I prove that this is true? There has been much analysis of shot type and distance (if you searched the archives you can find some stuff I did but Alan Ryder’s work is probably more thorough and comes to more or less the same conclusion) and that seems to imply that the Leafs defense over the past couple years has been average, if not a better than average.

    But while that analysis is good, it still doesn’t necessarily tell the whole story. It takes into account shot distance but distance doesn’t tell the whole story with respect to shot quality. A 20 foot shot from George Laraque with the goalie unscreened and lined up to the shooter is much easier than a cross ice pass leading to a 20 foot one timer shot from Dany Heatley. Do the Leafs have a propensity to give up those nasty cross ice passes? I am not sure. I suspect in the grand scheme of things they are not but there is no way of really knowing without watching every game and evaluating every shot one by one and I have no intention of doing that.

  8.  

    I think that the +/- stat is really only helpful for defensive players. For example, a defenseman with a +6 at this point in the season means that he has helped the team score 6 more even-strength goals than they have given up. It’s safe to assume that the defensemen have a significant role in their team scoring, as they usually are the ones cycling the puck. A +6 through 20 games for a defenseman usually indicates that he is a good defenseman, whereas a -3 usually means he is a below-average defenseman.

    For offensive players/forwards, the two stats that I look at as a determinant of a players’ overall “usefulness” are shooting percentage and turnovers per game. I think that shooting percentage tells how reliable a scorer a player is, while turnovers per game tell how reliable a passer a player is, and those are the two main things that result in goals.

    Those are just things that I look at, and I think those are the three most important stats as far as a player’s “worth” to his team and overall ability.

  9.  

    I agree the +/- can be misleading. If as a checking center I only get faceoffs in my own zone, even if i’m succesful 9 of 10 and on that one miss they score, i’m a minus 1 and never get the offensive zone faceoff to even myself out with a +.

    However sometimes the +/- can tell a story. Kaberle averaged 39 points and a plus 15 the 3 years before the lockout, now averaging 59 points but is averaging a minus 2.Is this the team he plays with or Kaberle not good enough under the no clutch and grab league, is he concentrating more on offence and the stretch pass instead of just getting the puck out of his zone, or looking for the assist instead of getting in deap ( yes I’m talking about the Trashers game and his 2 blue line turnovers)

  10.  

    Or maybe Kaberle just plays on a team with far worse goaltening post lockout.

Sorry, the comment form is closed at this time.