# My Player Ranking System and last years ratings

When developing my player ranking system I wanted to isolate the ability of an individual player as much as I can and best factor out both who the player is playing with and who they are playing against. I don’t know how many times I hear things like ‘but he has to face the opponent’s best players’ and things like that when people try to analyze how a player is doing. So I am trying to eliminate that.

For now I am going to spare you all of the gory details of the process I took but basically I looked at every shift every player has played and who they played with and against for those shifts. Thankfully the NHL makes this information available on their website. I also looked at how many goals for and goals against each player was on the ice for. By combining the goals for and goals against with the shift data of who the player was playing with and against I came up with an expected goals for and goals against. By that I mean, based on a players linemates and opponents, how many goals can one expect that he will be on the ice for and against. To get a defensive rating I divided how many goals he was expected to be on the ice for by how many goals he actually was on the ice for. I also eliminated the effect of players playing different amounts of ice time by adjusting the numbers to a per 20 minutes of ice time basis (i.e. how many goals were scored for every 20 minutes that the player was on the ice). A number greater than 1 means he was on the ice for fewer goals than expected and thus can be concluded that the player is a better than average defensive player. A number less than 1 would indicate he was on the ice for more goals than expected and is a below average defensive player. To calculate an offensive rating I took the number of goals his team scored while he was on the ice by the number of goals that were expected to be scored while he was on the ice. A number greater than one indicates more goals were scored by his team than expected and thus he is a better than average offensive player. Conversely, if the number is less than 1 the player is a less than average defensive player. I then calculated an overall rating by averaging the offensive and defensive ratings. For the time being I have just looked at even strength situations. In the future I plan on developing power play and penalty kill ratings as well or maybe finding a way to develop a combined rating system. I am just not sure how to do that yet so for now just even strength ice time was used.

The interesting thing about this method of rating players is that it doesn’t take into account how many goals and assists that that player tallied. It only takes into account how many goals were scored while he was on the ice. This is interesting because it allows us to compare forwards and defensemen and even goalies directly. We know that forwards score more goals and get more points but that doesn’t mean defensemen don’t contribute the same offensively. We also know that defensemen and goaltenders are important in stopping goals, but that doesn’t mean that forwards aren’t equally important. By not looking at goals and assists a player tallied we aren’t biasing the analysis towards point producing forwards.

Ok, I think it is time to look at some results. The NHL started making shift data available on January 18th of last season so I have ranked players using data from January 18th through the end of last season. That’s a total of 549 games or almost 45% of the season. Here are the top ranked players overall. Only players with 200+ minutes of even strength ice time are listed.

Player Team Offense Rating Offense Rank Defense Rating Defense Rank Overall Rating Overall Rank
MARTIN GELINAS Florida 1.83 13 1.56 28 1.70 1
SHEAN DONOVAN Calgary 0.85 398 2.48 1 1.67 2
HENRIK ZETTERBERG Detroit 1.77 15 1.55 30 1.66 3
JOE THORNTON San Jose 2.30 2 0.98 250 1.64 4
JONATHAN CHEECHOO San Jose 2.09 6 1.19 112 1.64 5
MICHAEL NYLANDER NY Rangers 2.23 3 1.06 180 1.64 6
MATHIEU SCHNEIDER Detroit 2.02 11 1.23 95 1.63 7
COLBY ARMSTRONG Pittsburgh 2.16 5 1.07 172 1.61 8
J-SEBASTIEN AUBIN Toronto 1.74 21 1.45 37 1.59 9
JAROMIR JAGR NY Rangers 1.70 26 1.44 38 1.57 10
TEEMU SELANNE Anaheim 2.31 1 0.82 413 1.56 11
ANDREI MARKOV Montreal 1.24 125 1.85 11 1.55 12
SHEA WEBER Nashville 1.23 129 1.88 8 1.55 13
DAVID LEGWAND Nashville 1.11 201 1.97 5 1.54 14
JAMIE MCLENNAN Florida 1.30 94 1.77 15 1.54 15
BRENDAN SHANAHAN Detroit 1.52 40 1.55 31 1.53 16
JOE SAKIC Colorado 2.03 8 1.00 228 1.51 17
MIIKKA KIPRUSOFF Calgary 0.66 511 2.36 2 1.51 18
MARK CULLEN Chicago 1.60 31 1.40 44 1.50 19
MICHAEL KOMISAREK Montreal 0.77 447 2.14 3 1.45 20
PATRICE BERGERON Boston 2.07 7 0.82 414 1.45 21
DMITRI KALININ Buffalo 1.66 27 1.21 105 1.44 22
SIDNEY CROSBY Pittsburgh 2.17 4 0.70 523 1.44 23
WILLIE MITCHELL Dallas 0.71 484 2.11 4 1.41 24
PHILIPPE BOUCHER Dallas 1.75 16 1.04 197 1.40 25
VESA TOSKALA San Jose 1.74 22 1.03 204 1.39 26
AARON JOHNSON Columbus 1.75 17 1.02 213 1.38 27
ANDY HILBERT Pittsburgh 1.84 12 0.92 295 1.38 28
BRENDEN MORROW Dallas 1.75 18 1.00 229 1.38 29
MICHAL ROZSIVAL NY Rangers 1.41 65 1.34 60 1.38 30
MIKE DUNHAM Atlanta 2.03 9 0.73 500 1.38 31
NIK ANTROPOV Toronto 1.37 76 1.39 47 1.38 32
FREDRIK SJOSTROM Phoenix 1.32 87 1.43 40 1.37 33
MARK MOWERS Detroit 1.41 66 1.32 66 1.37 34
ALEXEI PONIKAROVSKY Toronto 1.48 53 1.23 96 1.36 35
BRIAN GIONTA New Jersey 2.03 10 0.69 529 1.36 36
CRISTOBAL HUET Montreal 0.81 425 1.91 7 1.36 37
RICK DIPIETRO NY Islanders 1.40 69 1.33 63 1.36 38
ALEXEI ZHITNIK NY Islanders 1.30 95 1.39 48 1.35 39
BRIAN POTHIER Ottawa 1.04 255 1.65 20 1.35 40

Yeah, I know what you are thinking. Martin Gelinas the best overall even strength player? Are you kidding. No. I am not kidding. He was on the ice for 40 goals his team scored even strength which is two more than top Panther scorer Olli Jokinen was on for at even strength, and he was also on the ice for just 13 goals against. That’s an awfully good track record. On the season he was a +27 which is a whopping 12 points higher than his closest teammate (wouldn’t surprise me if this was a league best differential). Shean Donovan is also very surprising but mostly because hardly any goals were scored against the Flames when he was on the ice. In fact, when he was on the ice just 5 goals were scored against since January 18th. That’s pretty phenomenal really. We also need to remember that these ratings don’t represent a player’s net value. A player who is rated slightly less but gets more ice time will have a higher net value. What these ratings are trying to do is compare players when all things (teammates, opponents, and ice time) are equal. After those first two surprised in Gelinas and Donovan come some probably more expected names in Zetterberg, Thornton, Cheechoo, Nylander, Schneider and Armstrong.

I’ve kept the goalies included but their ratings are somewhat suspect because it ends up largely being a comparison with the other goalies on the team. J.S. Aubin is the top rated goalie because he was so much better than Belfour and Tellqvist but I am not sure he was the best goalie in the NHL. I plan on developing a more refined goalie ranking system but for now it is interesting to see how the goalies rate using this system.

Nik Antropov gets a lot of criticism by Toronto media and fans but I have been a big defender of his. And because of this I am glad to see that he ranked very well at 32nd overall.

At the bottom of the list were Marcel Hossa, Clarke Wilm, Eric Weinrich (while in Vancouver), Tim Taylor, Mike Ricci, Alyn McCauley, Alexander Khavanov, Martin Lapointe, Tyson Nash, and Kirk Maltby. There are a few surprising names there but most of them are lower tier players.

Evaluating a player ranking system is a difficult thing to do but one method of doing so is to look at how consistent it is. By that I mean how closely do players get ranked before and after trades and from season to season. Tomorrow I’ll look at some season to season comparisons but for now lets look at some players that were traded at the trade deadline last year and see how things look. I’ve listed offensive, defensive and overall ranking numbers for each player

Mark Recchi (Pittsburgh): 1.10, 0.76, 0.94
Mark Recchi (Carolina): 0.54, 0.97, 0.76

Martin Skoula (Dallas): 1.02, 0.74, 0.88
Martin Skoula (Minnesota): 0.66, 1.18, 0.92

Keith Carney (Anaheim): 0.76, 0.98, 0.87
Keith Carney (Vancouver): 0.88, 0.83, 0.86

Eric Weinrich (St. Louis): 1.49, 0.67, 1.08
Eric Weinrich (Vancouver): 0.56, 0.49, 0.52

Brent Sopel (Islanders): 0.84, 0.74, 0.79
Brent Sopel (Los Angeles): 0.52, 0.95, 0.73

Brendan Witt (Washington): 0.98, 1.44, 1.21
Brendan Witt (Nashville): 1.24, 0.62, 0.93

Brad Lukowich (NY Islanders): 0.91, 0.99, 0.95
Brad Lukowich (New Jersey): 1.46, 0.74, 1.10

Willie Mitchell (Minnesota): 1.14, 0.73, 0.94
Willie Mitchell (Dallas Stars): 0.71, 2.11, 1.41

There is some consistency in the overall numbers as Skoula’s, Carney’s and Sopel’s overall numbers are almost identical before and after the trade and Lukowich’s is fairly close. Overall I would have liked to see more consistency but most players got worse with their new teams which I think is a testament to how difficult it is to learn a new system and learn to play with new line mates.

1. Wow, Teemu Selanne the top offensive force in the league? Awesome.

Then again, while most players had benefit of decent linemates, Teemu was slugging it along with Andy McDonald (undrafted) and Chris Kunitz (undrafted), so there may be something to this.

2. I love that Antropov is ranked 32nd. I am a fan of the lanky Kazakh. I am also an advocate of the Antro-Poni duo but it looks like that would be putting all of our eggs into one basket.

3. When you do a statistical study you have to ask yourself if the results make any sense. Martin Gelinas the top player in the league? That doesn’t make sense.

What is missing? I think its the situation in which a player is used. Or maybe its some kind of false normalization (ie Florida hardly ever scored so this makes Gelinas’ offence look better than in should be shince it is normalized by a low value). Without a better look at the exact method, I cannot be sure, but I don’t accept that Martin Gelinas was the best player in the NHL last year. Its not a plausable scenario. I need a bit more detail to try to see what is wrong with it – but I am convinced it is not correct.

4. Steve says:

Just a few things. Basically from the sounds of it you’ve modified the +/- ranking by incorporating time on the ice. The undiscussed portion of how you determined your “expected” goals for and against ratings is obviously the key to this analysis but since you don’t divulge the details on that I’ll stick to what’s present.

First, there obviously needs to be some way of abrogating the discrepancies between solid offensive and defensive production. By working towards a factor of 1 which is relatively small, this display seems to distort the actual value. If you worked to a number like 100, by multiplying by 100 – and yes I realize that multiplying by a factor of 100 won’t change the results at all – you would display the results in a more clear fashion I think.

Part of the issue I take with this is that it doesn’t seem to me that there’s a huge discrepancy or impact for a player who is obviously at the lower end of either offensive or defensive production. If Donovan can finish 2nd overall in your system but be the 398th best offensive player in a league with 870 players (over 100 of whom didn’t play more than 20 games) then that would imply the defensive weighting is too high, but then Thornton ranked 250th defensively and still finished 4th overall. What I guess I’m saying is the variation of the impact of your numbers strikes me as a distortion in your results. A simple combination of the two values seems like an oversimplification to me. This stat also neglects numerous other fashions in which players contribute to their team. While I admit goal scoring and preventing them is the most obvious contribution, you’re ignoring puck possession, physical play, face off wins, shots on goal, etc.

Just by way of example contemplate the idea of a team that wins most of it’s games by scoring at a higher rate than it’s opponents, or a team that wins by preventing any goals from it’s opponents and occasionally chipping one in. In the first case, defensive play may be a secondary consideration, and in the latter, offensive play would be. For this reason, a player like Donovan would be unlikely to see a lot of ice on a team like San Jose, and Joe Thornton would hamper the game plan of a team like Calgary. I don’t see a way around ignoring styles of team play until you incorporate some sort of overall team style weighting. Perhaps if a player plays on an “offensive” team weighting the offensive numbers more significantly would be appropriate and vice versa for the “defensive” teams.

It is also relatively absurd to compare goaltenders to skaters. There is zero value added by including them. They are on the ice for virtually the entire game, they never handle the puck in the offensive zone, and frankly you are again ignoring styles of team play.

Either way, I am curious as to how you’re deciding the expected levels of production, since such factors would have to be entirely mutually dependent. Just because Chris Kunitz and Andy McDonald would be expected to draw down the expected production of Selanne, that doesn’t necessarily hold true. Wouldn’t their own production be expected to increase by his presence? This also ignores the concept of line chemistry. There are some bizarre line combinations in the NHL that somehow manage to work. Mike Knuble would be an example of a player that theoretically should bring down the production of top flight NHL players. But he’s been a part of two of the top scoring lines in recent history with Thornton and Murray in Boston, and Gagne and Forsberg in Philly. Jonathon Cheechoo would not have leapt out as a likely Richard Trophy winner BEFORE Joe Thornton became his center iceman. Now that he’s led the league in goals over a season does that mean we would assume he’d be likely to produce similarly with another center of different talents?

Just some food for thought, hope to read more about it.

5. David Johnson says:

Greg:

I was shocked to see Gelinas at the top of the list but when I see him at +27 and the next best Florida player at +15 that tells me that Gelinas is doing something right. If whenever Jokinen was on the ice at even strength Gelinas was on the ice too then they would have identical +/-. But they didn’t. For some reason Gelinas was on the ice when good things happened (either scoring goals or not allowing them) significantly more often than any of his teammates. That tells me something about Gelinas. Good things happened when Gelinas was playing with Jokinen and good things happened when Gelinas was playing with Horton. Good things follow Gelinas around. Is that a fluke or does that tell you something about Gelinas’s play? Interestingly enough he’s leading the Panthers in +/- this year too and tomorrow you’ll see that he is also rating reasonably well this season.

Was Gelinas the best player in the NHL last season? Probably not because he doens’t play on the PP and doesn’t get the ice time of other players, but even strength for the minutes he played he was one of the best.

Steve:

Part of the issue I take with this is that it doesn’t seem to me that there’s a huge discrepancy or impact for a player who is obviously at the lower end of either offensive or defensive production. If Donovan can finish 2nd overall in your system but be the 398th best offensive player in a league with 870 players (over 100 of whom didn’t play more than 20 games) then that would imply the defensive weighting is too high, but then Thornton ranked 250th defensively and still finished 4th overall.

I think we can conclude that there are very few players that are outstanding at both. If you take risks offensively you will take a hit defensively. If you focus on your defensive game, it hinders your offensive output.

While I admit goal scoring and preventing them is the most obvious contribution, you’re ignoring puck possession, physical play, face off wins, shots on goal, etc.

Most of that is indirectly taken into account. If you are taking shots at the net you have a better chance of scoring goals and a lesser chance of giving them up. If you posess the puck you have a better chance of getting offensive opportunities and goals than giving up scoring chances and goals. Same for winning faceoffs and for the most part physical play which might force turnovers and may also result in your being out of position defensively. What isn’t taken into account is things like physical play wearing down an opponent over the course of the game and drawing and taking penalties.

Just by way of example contemplate the idea of a team that wins most of it’s games by scoring at a higher rate than it’s opponents, or a team that wins by preventing any goals from it’s opponents and occasionally chipping one in. In the first case, defensive play may be a secondary consideration, and in the latter, offensive play would be. For this reason, a player like Donovan would be unlikely to see a lot of ice on a team like San Jose, and Joe Thornton would hamper the game plan of a team like Calgary.

I am not sure I accept this arguement. It’s possible but I suspect that Joe Thorton would be a benefit to the Flames and Donovan would probably also be a benefit to the Sharks. No team plays all offense or all defense. There are always situations where you want one or the other.

It is also relatively absurd to compare goaltenders to skaters. There is zero value added by including them.

I mostly agree but only because there aren’t enough variations in situations that can isolate a goaltenders value. By that I mean the starting goaltender most likely plays with the same players in similar proportions of time as the backup and the quality of the opponent probably approaches equivalent levels too. If there were more ways to differentiate them I think we could isolate their value better.

Either way, I am curious as to how you’re deciding the expected levels of production, since such factors would have to be entirely mutually dependent.

Without giving up too many details what my algorithm basically does is look at how a players linemates play without him. If they player better with him his ratings will be above 1, if they play worse with him his ratings will be below 1. That’s a very simple way of looking at it as there is more that is considered including the opponent they are playing against as well.

Oh and just to tack on a bit here, Smolinski was a +22 the year before last in Ottawa with 11 of his 46 points coming on the power play. That means 35 goals with him out there, but only 13 against when he was on the ice… considering your points about Sean Donovan and Martin Gelinas that would make him pretty valuable in your new system wouldn’t it??

No, not necessarily. If Smolinski played all his even strength ice time with +35 players then one could probably conclude that he dragged his line down. In fact this was probably the case. Last year Smolinski rated 0.92 offensively, 0.81 defenseively and 0.87 overall. So far this year he has been better offensively but weaker defensively and slightly better overall.

Maybe an easier way to discern whether or not it’s the team affecting the player or the player affecting the team would be to take the team’s overall ± and compare the player’s number to that number, ranking them based on the difference between their contribution and those of the rest of the league.

That is kind of what my algorithm though taking into account much more information like who the individual player plays with and against as opposed to looking at team averages like you are suggesting.

It would have been really interesting to see how Thornton rated in Boston and Cheechoo rated before he teamed up with Thornton but unfortunately the data I need is not available prior to January 18th.

6. You have done a lot of work here, but your lack of information on your algorithm with less than intuitive results makes this information almost worthless.

Expected goals:
“Without giving up too many details what my algorithm basically does is look at how a players linemates play without him.”
– This part concerns me primarily to how little data this could be. (eg. H. Sedin + D. Sedin they never play apart) Of course I don’t know what you’re doing so it’s difficult to criticize or embrace anything.

Error:
Using goals from only half the season means you’re looking at maybe 10-15 minutes per game per player for 40 games or at best 36,000 seconds with ~3 goals per hour ~ 30 goals on average for and against per player (not sure the exact values it’s not that important) or ± 5.5 goals for and against with a confidence interval of (19, 41) which of course suggests the aggregate of ± 16 (95% confidence interval). But one has to consider as well that there are +900 players in the league so we should expect about 50 players to exceed that range! Take note this is the range if each player was identical and goals happened randomly.

A better way of saying the above: how much does an effect does changing a players plus minus score by 5?

Goaltenders:
What’s interesting is Aubin is ranked 21 with a score of 1.71, if this doesn’t tell you how much error these results have nothing does. Mind explaining how Aubin could be so much better at scoring than say Belfour? Dunham score is over 2!

Gelinas?
Of all players a \$975,000 player is the best in the league? I realize that with poor information you can have an inefficient market, but paying \$975,000. Interestingly Gelinas played at basically a ±0 level his whole career until he played in the swiss league and somehow managed a +81 in 41 games!

I’m surprised you didn’t see this: but Gelinas was +27 (40 – 13 = 27) by your own statement in the last half of the season, which means he was 0 in the first part, so did he go from best to average in half a season? (This is similar to the error problem mentioned above).

Donovan?
Signed by Boston for \$925,000 also had a great plus minus in the swiss league, but other than that he has consistently been below 0.

Nylander?
So he’s the reason the New York Rangers score so many goals, and I thought it was Jagr!

Interestingly you were right to compare traded players, but the results basically showed this information doesn’t help quantify players skills. In fact the correlation is not statistically significant. Although it explains 10% of the variability (sample size: 16).

Good luck defending your methods and results.

7. DANiel says:

i still think its not fair. i mean donovan has such a good defensive record cuz he has kipprusoff backstopping him. no wonder he only has 5 goals scored against him. and like greg said, a player like donovan is good in calgary, but wouldnt see the light of day in san jose or ottawa. sam goes for gelinas. obviously hes doing something right, but it shouldnt be that high. a guy like rolston, who is a great center, super fast, plays the point on the PP and is considered lethal, and as well is one of the best penalty killers in the league, is a much much better player than say… nylander. i think u need more facotrs in there. players like chris higgins who has shorthanded goals, as well as lots of SH time, and PP time and has stats to back it up, should be much higher. im not saying higgins should be up there since your goig on last seasons stats, but im saying someone LIKE higgins. also, faceoff % is a major factor in determining a players value.

i think your ratings are more centered around goals for and goals against. so a guy like gelinas will get u a good ratio, but isnt necessarily a valuable player. again, im not saying hes not, but he certainly isnt as valuable as thornton lol, or even nylander or gionta.

8. David Johnson says:

This part concerns me primarily to how little data this could be. (eg. H. Sedin + D. Sedin they never play apart) Of course I don’t know what you’re doing so it’s difficult to criticize or embrace anything.

Certainly some pairs or trios of players will be difficult to isolate but generally this is in the minority. Over the course of a season or even a half season most players will play time apart from each other. So far this season the Sedins have player 8-9% of the time apart from each other. I’ll grant you that that isn’t much and not a lot of data to work with this early in the season but these two guys are probably the most connected pair of players in the NHL.

Mind explaining how Aubin could be so much better at scoring than say Belfour?

I am a firm believer that good goaltending leads to better all round play by the players in front of him. If players are confident that the goaltender can stop the puck they won’t be so paranoid about taking chances offensively. Of course this is just my hypothesis so an analysis would need to be done to prove it but I suspect there is some truth in it.

I’m surprised you didn’t see this: but Gelinas was +27 (40 – 13 = 27) by your own statement in the last half of the season, which means he was 0 in the first part, so did he go from best to average in half a season?

I did notice it but the conclusion you draw is invalid without looking at how his teammates did for the first half of the year. If you recall, Florida was a much better team in the second half than the first half.

Jokinen was +14 on the year and +19 over the course of my study period so he was -5 for the first half.

Stumpel was a +12 on the year and a +15 over the course of my study so he was a -3 for the first half of the season.

Funny how things work out.

i still think its not fair. i mean donovan has such a good defensive record cuz he has kipprusoff backstopping him.

More on this tomorrow but Donovan has a 1.23 defensive rating this year so he is better than average. Maybe he is a good defensive player afterall.

a guy like rolston, who is a great center, super fast, plays the point on the PP and is considered lethal, and as well is one of the best penalty killers in the league, is a much much better player than say… nylander.

I made it clear that these are even strength ratings. I do plan on doing PP/PK ratings too and hopefully figure out a way of combining them.

i think your ratings are more centered around goals for and goals against. so a guy like gelinas will get u a good ratio, but isnt necessarily a valuable player.

Maybe you can explain to me how scoring more goals than you give up doesn’t make you a valuable player?

9. Concerning Donovan: if he was getting a high score merely because of the team and goaltender he plays with, you would expect to see most, if not all of Calgary’s lineup stacked near the top of the defensive list. Same would go for Ottawa’s players on the offensive list.

No statistical ranking system will ever tell the whole story, but I for one am impressed and intrigued and can’t wait to see this year’s rankings.

10. Steve says:

To respond to the following :

“No, not necessarily. If Smolinski played all his even strength ice time with +35 players then one could probably conclude that he dragged his line down. In fact this was probably the case. Last year Smolinski rated 0.92 offensively, 0.81 defenseively and 0.87 overall. So far this year he has been better offensively but weaker defensively and slightly better overall.”

I would make two points. 1) The +22 I quoted was for 03-04, so you have no stats on his numbers for that season in your rating’s system. 2) The players he played with regularly in 03-04 were Spezza and Schaefer, who were both +22. Not +35. Considering all three of them had the same +/-, it strikes me as a bit strange to insist that it was the OTHER players responsible for the high +/- rating. To say he was “probably” playing with +35 players implies you don’t actually know who he was playing with 2 seasons (or 3 years) ago, which worries me a bit. To wit, the Sens didn’t have ANY +35 players 3 years ago… they had one +33 player in Zdeno Chara but nobody else on the team broke +22. Guess that means Smolinski was one of the guys at +22 who was helping raise Daniel Alfredsson and Martin Havlat to their “solid” +12 ratings??? This is twice in two days where you’ve said something without checking into it first on this SAME topic. Before replying you might want to look into your stats more thoroughly.

My point is, you’re original reason for comparing the players with your newfound system was a flaw in the old one. Frankly, you still haven’t offered me a decent comparison that demonstrates a larger issue than any of your OWN comparisons. You have JS Aubin and Shean Donovan ranked ahead of Jaromir Jagr and Joe Sakic. That strikes me as a far more strange comparison than just grasping the definition of +/- as a statistic and understanding that players who produce a lot on the power play might be weak defensively. To insist that there are serious problems in the +/- ranking because a player with fewer points might have a better +/- than a player who is considered elite baffles me.

You obviously KNOW what +/- is measuring. Why worry that it points out flaws in a player’s game? The fact that Crosby was -1 last year despite having over 100 points doesn’t mean he was a bad player. It just means he needed serious improvement defensively.

I’m not saying determining the value of a player to their team isn’t a valiant goal, I just think you’re barking up the wrong tree here by over-analysing the +/- stat. I mean another simple way to factor in a contribution would be to take the +/- ratings, and then add on all the points from the power play, and then take off points at a rate determined by taking the player’s penalty minutes and dividing by two, then multiplying the result by 1 minus the team’s penalty kill success rate. This should give you a rough estimate of how many goals were given up while the player in question was in the penalty box (though if they’re a solid penalty killer their presence in the box could hurt their team even more obviously). There’s a lot of ways to go at the problem, I just don’t know if it’s a problem in the first place.

11. Steve says:

Oh and if Gelinas being on the ice for 13 against makes him a solid player it’s a tad weak of you to say it doesn’t make Smolinski solid since he MUST have been playing with better players. Especially when the distinction of how many goals he was on the ice for is 40 goals for, for Gelinas, vs. 35 for Smolinski. That’s a difference of 5 goals for, I’m pretty sure if you had a player who was on the ice for 15 goals for but 40 against, and compared him to a player who was on the ice for 10 goals for but 40 against, you wouldn’t insist that the player with 15 goals was the MUCH better player. You also wouldn’t attribute his obviously superior play to all those -10 players raising his +/- either. All of this strikes me as a tad absurd since you’re comparing mutually dependent numbers that vary in import to you relative to their totals. The difference of 5 goals should be of equal value no matter WHERE you consider it, but as we all know… it isn’t.

12. David

If you truly believe there is something to this system then you would believe that had Atlanta traded Ilya Kovalchuk and Marian Hossa (both are nowhere to be seen in these rankings) for Martin Gelinas and Shean Donovan (top two guys) then they would have made a huge improvement for the last half of the season last year.

Would you make such a statement?

13. David Johnson says:

I would make two points. 1) The +22 I quoted was for 03-04, so you have no stats on his numbers for that season in your rating’s system. 2) The players he played with regularly in 03-04 were Spezza and Schaefer, who were both +22. Not +35. Considering all three of them had the same ±, it strikes me as a bit strange to insist that it was the OTHER players responsible for the high ± rating.

I just picked the +35 numbers out of my head to make a point because you seemed to automatically equate +22 with having a good rating just as you tried to equate Gelinas 0 for the first half of last year with mediocrity. There is a whole lot more that goes into these numbers that someones +/-. But yes, you are correct, it appears that Smolinski would have rated much higher in 2003-04 but you also have to remember that that was a significantly different style of game. A lot of players who were effective then aren’t now and vice versa.

To insist that there are serious problems in the ± ranking because a player with fewer points might have a better ± than a player who is considered elite baffles me.

There are serious problems in the +/- ranking system because you cannot easily compare players on different teams. A +2 might be good on one team where a +15 might be mediocre on another. I can’t compare players +/- ratings without considering a lot of other information. I am trying to wrap +/- and all that other information into one number.

Frankly, you still haven’t offered me a decent comparison that demonstrates a larger issue than any of your OWN comparisons.

What kind of comparison do you want?

The fact that Crosby was -1 last year despite having over 100 points doesn’t mean he was a bad player.

No. Did I ever say he was a bad player? If you look at the table you will see Crosby was ranked 23rd overall. He’s ranked very highly this year too.

I mean another simple way to factor in a contribution would be to take the ± ratings, and then add on all the points from the power play, and then take off points at a rate determined by taking the player’s penalty minutes and dividing by two, then multiplying the result by 1 minus the team’s penalty kill success rate.

+/- is a crap statistic. It is almost useless because you always have to put it into context. A guy playing in front of a bad goalie will have more -‘s that are no real fault of his own. You cannot start with +/- and get anything useful unless you somehow take into account things like that.

And yes, this is not the be all and end all statistic. I am not done either. I want to create PP and PK ratings and look into several other factors to which I would consider good and bad attributes.

If you truly believe there is something to this system then you would believe that had Atlanta traded Ilya Kovalchuk and Marian Hossa (both are nowhere to be seen in these rankings) for Martin Gelinas and Shean Donovan (top two guys) then they would have made a huge improvement for the last half of the season last year.

No, I never said that. These are even strength rankings, not all encompassing rankings. And you are getting into team building issues. Kovalchuk and Hossa have huge benefits on the PP over Gelinas and Donovan that aren’t taken into account. But if I am trying to hold a lead late in the game I would probably prefer Gelinas and Donovan on the ice.

Concerning Donovan: if he was getting a high score merely because of the team and goaltender he plays with, you would expect to see most, if not all of Calgary’s lineup stacked near the top of the defensive list. Same would go for Ottawa’s players on the offensive list.

No, not necessarily. Amonte, Warrener, Iginla, McCarty, Regehr, Huselius, Ference, Lombardi, Leopold, and Kobasew all ranked below 1.

14. Greg: I never once said that I would take this (or ANY, for that matter) statistical analysis and decide who are the “better” players based purely on that. As I mentioned, no statistical analysis can ever tell the whole story. There are simply too many factors to take into account. What I am is interested in seeing a more complete list of players, because this kind of analysis tends to point out players or trends that you wouldn’t notice otherwise. The trick is that the reader then has to absorb that information and decide what it really means.

For example, JS Aubin ranking so highly last year. Considering that Aubin was 9-0-2 last year, and how well the Leafs did while he was in the net, I’m not surprised. I’m smart enough to know that if Aubin were the starter through the entire season, his stats would certainly plumet. However, he didn’t play the whole season. He played in 11 of the last 12 games, and no other goaltender in the league boasts the kind of record he has. These rankings don’t take the limited number of games Aubin played in mind.

IF we could extrapolate Aubin’s stats from last year, over the course of a season we would expect him to finish with about 12 OT or SO losses and about 60 wins. His save % and GAA would also have led the league. Realistic? Of course not. But if that had happened, would you not expect Aubin to be the “best” goaltender?

So it’s my responsibility to look at these stats and realize that Aubin at least did not play enough games to make this a fair assessment.

I find it very interesting that Gelinas’ +/- is so much better than the rest of his teamates. It’s up to me to decide exactly what that means, however.

15. Others beat me to a lot of my comments, but I’ll add some ‘me toos’ anyway.

– To evaluate the system, we would absolutely have to see how your expected goals calculations work.

– Like Javageek said: from what I can gather about the expected goals, it could be a huge source of error. Say Gelinas played with the same two linemates for the entire sample period except for a single shift, during which his line was burned for two flukey goals against. That would skew his ratings immensely. Including error bars would add a lot of credibility.

– It seems like a lot of top defensive players played in front of great goaltending (Luongo, Kipper, Huet, Vokoun) over your sample period. That suggests the method of isolating skaters from goalies might need tweaking.

– You are absolutely right in ignoring hits/blocked shots/possession time etc. Those contributions are only valuable insofar as they lead to more goals for and/or fewer goals against.

– *If* a player can be *perfectly* isolated, and ignoring the fact that creating PP time/avoiding SH time is part of the ES game, using only +/- is a perfectly legit way to evaluate ES contributions. No need to give extra credit for goals or assists. If the player is isolated perfectly, they’re only plusses.

– That Gelinas and Donovan are high on the chart explains why coaches like those guys so much. Of course, you’d have to be a fool to think they’re the two best ES players in the league. Their placements are largely due to error, but there is a grain of truth behind them too.

It’s a lot easier to criticize stuff like this than to put in the effort to do it yourself. Great work.

16. David Johnson says:

To evaluate the system, we would absolutely have to see how your expected goals calculations work.

I don’t totally agree. I think you can do some subjective analysis without looking at the actual algorithm.

Like Javageek said: from what I can gather about the expected goals, it could be a huge source of error. Say Gelinas played with the same two linemates for the entire sample period except for a single shift, during which his line was burned for two flukey goals against. That would skew his ratings immensely. Including error bars would add a lot of credibility.

First off, that doesn’t happen that three players play every shift together except one of them plays one other shift and gets burned for 2 players. If you look at this years ratings which I just posted I think you see some of that and you may even see a tiny bit of it in last years ratings but over time, and certainly over the course of a half season or more, this diminishes.

It seems like a lot of top defensive players played in front of great goaltending (Luongo, Kipper, Huet, Vokoun) over your sample period. That suggests the method of isolating skaters from goalies might need tweaking.

I do agree but I think it has more to do with defensive teams vs offensive teams as opposed to just the goalies. It’s not going to be easy to isolate that but I think overall ratings are less affected. A team like Calgary will likely have higher rated defensive ratings than they should and lower rated offensvie ratings than they should because they generally play a defense first system. Part of that is because of their players are more defensive minded but part of it is a coaching decision. But when you average Calgary’s bias towards higher defensive ratings with the bias towards lower offensive ratings the overall ratings even out a bit.

That Gelinas and Donovan are high on the chart explains why coaches like those guys so much. Of course, you’d have to be a fool to think they’re the two best ES players in the league. Their placements are largely due to error, but there is a grain of truth behind them too.

Possibly. I think this is probably more true for Donovan than Gelinas (see this years rankings where Gelinas also does well) but if you remember back to Calgary’s cup run, these two guys big parts of that run, espescially Gelinas.

It’s a lot easier to criticize stuff like this than to put in the effort to do it yourself. Great work.

I always enjoy hearing fair criticism as well as it keeps me thinking of flaws in the system and of new ways to imrpove it. This is still a work in progress and I hope to tweak it and improve it as time goes on.

17. A few more questions:

1. What are the units?

2. What’s the distribution of talent look like?

3. What is the expected error (assuming goals occur randomly).

4. Ultimately the question for all sabrmetric type stuff is: how does it correlate to wins? But of course we’ll need PP and SH info first.

“To evaluate the system, we would absolutely have to see how your expected goals calculations work.”

“I don’t totally agree. I think you can do some subjective analysis without looking at the actual algorithm.”

Really? Mind explaining?

See when I look at save percentage I know that a difference of 1% will mean he stops 1% more shots. If I give you a goals against per hour I can multiply that by ice time to calculate expected goals.

You tell me Sidney Crosby is 2.17 what would it mean if he was 2.18 or 2.19, would that be really different or slightly different. All we have here is a list of to 10% of the distribution. Including goaltenders who are better offensively Zetterberg. You’d think the whole NHL would be lining up to sign Aubin.

18. Steve says:

Ok a few problems with your response to my comments. Firstly, I wasn’t the one that made the point about Gelinas having a 0 in the first half of the season which would make him mediocre. Secondly, I disagree that +/- is such a crap statistic since you obviously are taking it into account for your own ratings. You’re using it to justify and explain your results, which are still skewed in a fashion that makes little sense to me. This would be why it’s necessary for you to explain the algorithm that gave you your results. Until other people can understand the process, you’re not going to have much luck getting people to buy into them.

I still don’t really see how you can say +/- is crap because guys might play in front of a bad goalie in one response, then two responses later when someone says “it seems a lot of top defensive players played in front of great goalies” you come back with the fact that you think it has more to do with offensive vs. defensive teams. I admit goalies have an effect on players +/- ratings, but to throw the stat out as if it’s entirely the goalies’ or other players’ fault that a player is receiving a minus isn’t entirely justifiable. I just feel like you’re goal post moving here.

Obviously the style of play of a team has an impact. That was one of the points I tried to originally make, and as far as I can tell, you don’t take it into account ENOUGH.

When isolating an individual players value you’re never going to remove the affects of the systems they’ve played in unless you can come up with numbers for them in every possible system of play. Since you’ll never find that complete a data set, at best these results are an approximation.

As far as saying hits, puck possession, etc are only valid statistics in regards to their contribution to goals for or against I can come up with a few situations where that may or may not be the case.

Firstly, how does the player who is sent out on an energy line that is worse both defensively and offensively rewarded for the fact that their consistent hitting on the forecheck wears down the opposition over an extended period of time? They might be on the ice for 2 goals against, score zero themselves, but their team could win with 3 goals late while they’re stapled to the bench because they pounded the snot out of the other team’s top defenders throughout the first two periods. I still think those players deserve better than being ignored for those efforts entirely.

As for puck possession, you initially said it only has value as it relates to scoring goals or preventing them. To an extent I agree, but again you could have players who keep the puck on a string but can’t put the thing in the net. Players like Sergei Berezin, or Maxim Afinogenov come to mind. They’re not necessarily the most solid defenders, they don’t necessarily score every time they’re on the ice, but by maintaining possession they’re definitely contributing to a win. I guess my point is… it would be possible to maintain possession for an entire game and neither score any goals nor give any up. That would theoretically result in a perfect defensive rating, but the worst offensive rating possible… averaging out to 1.0 which would at best make them “average” players… again I just think if you’re isolating skill sets or talent these stats are limiting.

Back to the point about what sort of comparison I want. What I want is for you to make the case that a player with a high +/- isn’t better than a player with a low +/- in the general scheme. Again it’s a relative comparison, but I just think you’re arguing that the +/- stat doesn’t let you withdraw information from it that isn’t actually contained within the statistic. That doesn’t make it a bad stat, it just isn’t the stat you want it to be. That’s like being mad that the temperature scale doesn’t tell you how humid it is outside. “Is 40 degrees C in Phoenix REALLY worse than 30 degrees C in Toronto? Not if you include the humidity.”

19. David Johnson says:

“What are the units?”

There are no units, it is a ratio of actual goals for/against to expected goals for/against (on a per minute basis).

“What’s the distribution of talent look like?”

What’s your definition of talent? Point producers? Defensive specialists? Playmakers?

“3. What is the expected error (assuming goals occur randomly).”

Don’t know, don’t care. Goals don’t occur randomly.

“Ultimately the question for all sabrmetric type stuff is: how does it correlate to wins?”

Yes, of course. That’s my goal down the road. I could more easily determine value which would be the players rating * ice time.

“To evaluate the system, we would absolutely have to see how your expected goals calculations work.”

“I don’t totally agree. I think you can do some subjective analysis without looking at the actual algorithm.”

Really? Mind explaining?

To quote yourself: “Ultimately the question for all sabrmetric type stuff is: how does it correlate to wins”. And of course there is the “Does it make sense” test which is what we are doing in this thread.

“See when I look at save percentage I know that a difference of 1% will mean he stops 1% more shots. If I give you a goals against per hour I can multiply that by ice time to calculate expected goals.”

Sure, but not all shots or goals against are necessarily created equal so it doesn’t create a true understanding of a goalies value. If a goalie gives up 4 goals to Buffalo and another goalie gives up 4 goals to Columbus, are the goalies equal? Is it fair to make a direct comparison when the competition is different?

Think of my stat the same way. Is it fare to compare a player who plays the Senators and Sabres 8 times a year the same way as a player who plays the Blue Jackets, Blues and Blackhawks 8 times a year? Is a goal score against you by Alfredsson-Heatley-Spezza equal to a goal scored agaisnt you by McGratton-Kelly-Hamel? If one player plays 15 minutes of a game against the Spezza line and gives up 1 goals while on the ice is that player equivalent to his teammate who played 8 minutes on the ice against the Kelly line and gave up 1 goal? +/- says yes. My rating says no. The one player may very well be expected to give up a goal a game agaisnt the Spezza trio.

You tell me Sidney Crosby is 2.17 what would it mean if he was 2.18 or 2.19, would that be really different or slightly different.

The difference between 2.17 and 2.18 is marginal. It would mean he scored 2.17 times the +/- he was expected to get.

20. David Johnson says:

Ok a few problems with your response to my comments. Firstly, I wasn’t the one that made the point about Gelinas having a 0 in the first half of the season which would make him mediocre. Secondly, I disagree that ± is such a crap statistic since you obviously are taking it into account for your own ratings. You’re using it to justify and explain your results, which are still skewed in a fashion that makes little sense to me. This would be why it’s necessary for you to explain the algorithm that gave you your results. Until other people can understand the process, you’re not going to have much luck getting people to buy into them.

1. Sorry about mixing up my responses.

2. +/- is a crap statistic for comparing players because a +12 might be bad on one team and a +2 might be good on another. I don’t know how many times I heard the Ottawa media brag about having 3 or 4 players in the top 5 of +/-. That doesn’t tell me squat about those 3 or 4 players, just that Ottawa was a good team that scored a lot of goals and didn’t give up a lot. +/- can only be useful if you put it into context like how does one player compare to his teammates. But even that doesn’t take things into full account because it doesn’t factor in things like a player who might have his sole duty of shutting down the opponents top line. If he doesn’t score any goals and give up 5 for a -5 that may very well be an outstanding result. My rating takes into account all of this so we don’t have to.

“Firstly, how does the player who is sent out on an energy line that is worse both defensively and offensively rewarded for the fact that their consistent hitting on the forecheck wears down the opposition over an extended period of time?”

I actually aknowledged that this was one exception.

“They’re not necessarily the most solid defenders, they don’t necessarily score every time they’re on the ice, but by maintaining possession they’re definitely contributing to a win.”

Exactly. They contribute to the win by not giving up goals. Unless of course they hold possession of the puck for 30 second, turn the puck over for a 2-on-1 the other way giving up a goal in the last 5 second of their shift.

“That would theoretically result in a perfect defensive rating, but the worst offensive rating possible…”

Yeah, so he might get a 2 defensive rating and a 0 offensive rating for a pretty good 1.00 overall rating. That to me sounds like he would be getting credit for what he is doing. He contributes to a win but not exceptionally so (i.e. he doesn’t both score goals and stop them). That sounds reasonable to me. I’d rather have Crosby who can also put the puck in the net while keeping the puck away from the opposition.

“What I want is for you to make the case that a player with a high ± isn’t better than a player with a low ± in the general scheme.”

Is player who was +8 in Ottawa last year (Smolinski) better than a player who was -1 in Pittsburgh (Crosby)?

21. Steve says:

Ok you addressed most of my issues aside from the last one, and in response to that, a player who was +8 for Ottawa last year (Smolinski) was better than the player who was -1 in Pittsburgh (Crosby) DEFENSIVELY. That’s all the stat tells me, and I’m ok with that. I don’t look at +/- as a defining be all and end all stat.

22. David Johnson says:

“Ok you addressed most of my issues aside from the last one, and in response to that, a player who was +8 for Ottawa last year (Smolinski) was better than the player who was -1 in Pittsburgh (Crosby) DEFENSIVELY.”

Really? How do you come up with that conclusion? Is Chris Kunitz (+19) a better defensive player than Jere Lehtinen (+9)?

23. Steve says:

Well considering that 10 of Kunitz’s 41 points came on the power play, leaving him with 31, the fact that he was a +19 means he was only on the ice for 12 against.

21 of Lehtinen’s 52 points came on the power play, so he was out there for 31 points at even strength also. Since he was a +9 I guess that means he was on the ice for 22 goals against. Last I checked both teams have stellar goaltending and solid defensive records. They finished 6th and 7th in the league for Goals Against per game, and 4th and 5th for 5 on 5 Goals For/Goals Against.

Lehtinen averaged over 18 minutes of ice time per game, while Kunitz was out there for around 14 minutes. So YES, based on the stats I would say Kunitz did better… he produced the exact same number of even strength/short handed points in less time, and fewer games. He also was on the ice for fewer against on teams with relatively similar defensive and goaltending stats. What more justification do you really want here? Do you want me to say Lehtinen must be better because he’s won the Selke multiple times? I will grant you in their entire careers Lehtinen has demonstrated superior play over a longer period of time and thus deserves recognition in the form of greater salary. That doesn’t mean based on the numbers from last year that he was better overall.

24. Steve says:

I’d also say Kunitz did a BETTER job producing 5 on 5 since he played fewer games to get the same number of points. You could also factor in things like the fact that since Lehtinen was on the ice more he was out there for an extra 10 goals against. But if he had an extra 13 games to give up 10 more goals he also had an extra 13 games to score 10 more goals. Every argument that would justify the increase in goals against would require you to explain why they didn’t observe an increase in goals for. Since you can’t point at statistical production of the players in question and justify the differences you’re looking for explanations in the players they played with. Well lets do that quickly. Are you implying that the 5 men that shared the ice with Lehtinen for most of his ice time last season were inferior to the 5 men that Kunitz shared the ice with? Selanne, McDonald, Neidermayer, Beauchemin, and Bryzgalov/Giguere are somehow BETTER than Modano, Morrow, Zubov, Boucher, and Turco? Is that your argument? That Kunitz played with BETTER players??? that doesn’t hold water sorry.

25. Steve says:

You seem to be struggling with the dichotomy of justifying inferior play by superior players. Isn’t it possible that a superior player has problem areas or a poor season? Why is it so impossible to accept that younger players have break out seasons that provide them with superior numbers in the short term.

Given your analysis, I assume you’re trying to justify that Jere Lehtinen is a worse player than Shean Donovan. While I would accept that your analysis indicates this, I don’t personally feel that Donovan was necessarily superior off the bat. According to a more simplistic examination, Donovan was a +9, as was Lehtinen. Donovan only contributed 20 points, in the exact same number of games. None of which were on the power play. So Lehtinen produced 11 more points during the run of even strength or short handed play. He also happened to be on the ice for 7 more minutes a game, and spent more time killing penalties than your BEST DEFENSIVE PLAYER DONOVAN. How does that logically make any sense? If he’s the best defender in the league why haven’t his own coaches noticed Donovan’s superior shut down skills and decided to put him out there to kill off penalties as much as possible? There are 6 forwards on the Flames who were on the ice short handed MORE often than Shean Donovan. That’s not exactly indicative to me of the best defensive forward in the league.

The fact that Lehtinen shared the exact same +/- as Donovan just indicates that while he scored more he was also on the ice for more… that’s reasonable and understandable since he was out there for an extra 7 minutes per game. Somehow you seem to want to explain away +/- because it doesn’t let you justify a good player as being better than a player you’re biased against. Yet you completely ignore the exact same arguments against your OWN system. It’s like you’re happy with the results just because they aren’t what other people would expect, and you’re scrambling to justify them by insisting other statistics don’t do an adequate job… which doesn’t mean yours is doing an adequate job at this point either. I guess I just find this to be all a bit circular in the reasoning department.

26. Steve says:

And one way of justifying the point about Smolinski’s defensive play in comparison to Crosby’s would be empirical evidence collected from years of watching the man play. He is a better defensive center than Sidney Crosby demonstrated himself to be last season. Are you saying you think Sidney Crosby was a better defender than Brian Smolinski last season? If you could demonstrate that somehow I’d be willing to listen to the argument. Again you’re not actually explaining how you determine a players relative value offensively or defensively which makes it a tad hard to use the numbers you come up with as justification.

For all I know you’re inventing them off the top of your head. So until you can explain it in detail, perhaps this conversation should end.

27. David Johnson says:

Well considering that 10 of Kunitz’s 41 points came on the power play, leaving him with 31, the fact that he was a +19 means he was only on the ice for 12 against.

21 of Lehtinen’s 52 points came on the power play, so he was out there for 31 points at even strength also. Since he was a +9 I guess that means he was on the ice for 22 goals against. Last I checked both teams have stellar goaltending and solid defensive records. They finished 6th and 7th in the league for Goals Against per game, and 4th and 5th for 5 on 5 Goals For/Goals Against.

Lehtinen averaged over 18 minutes of ice time per game, while Kunitz was out there for around 14 minutes. So YES, based on the stats I would say Kunitz did better… he produced the exact same number of even strength/short handed points in less time, and fewer games. He also was on the ice for fewer against on teams with relatively similar defensive and goaltending stats. What more justification do you really want here? Do you want me to say Lehtinen must be better because he’s won the Selke multiple times? I will grant you in their entire careers Lehtinen has demonstrated superior play over a longer period of time and thus deserves recognition in the form of greater salary. That doesn’t mean based on the numbers from last year that he was better overall.

I’d also say Kunitz did a BETTER job producing 5 on 5 since he played fewer games to get the same number of points. You could also factor in things like the fact that since Lehtinen was on the ice more he was out there for an extra 10 goals against. But if he had an extra 13 games to give up 10 more goals he also had an extra 13 games to score 10 more goals. Every argument that would justify the increase in goals against would require you to explain why they didn’t observe an increase in goals for. Since you can’t point at statistical production of the players in question and justify the differences you’re looking for explanations in the players they played with. Well lets do that quickly. Are you implying that the 5 men that shared the ice with Lehtinen for most of his ice time last season were inferior to the 5 men that Kunitz shared the ice with? Selanne, McDonald, Neidermayer, Beauchemin, and Bryzgalov/Giguere are somehow BETTER than Modano, Morrow, Zubov, Boucher, and Turco? Is that your argument? That Kunitz played with BETTER players??? that doesn’t hold water sorry.

No, my arguement is that you have to write up a short essay of extra analysis to draw any conclusions about what a players +/- really means. That is why +/- isn’t a good stat.

As for your analysis, your very first statement is invalid. Just because Kunitz only scored 31 points at even strength and was a +19 does not mean he was only on the ice for 12 goals against. If Thornton and Cheechoo scored a goal in which Kunitz was on the ice but didn’t get a point that still counts as a +. Kunitz benefits from his linemates.

But since

28. Steve says:

Ok, you’re correct. Kunitz was on the ice for a number of goals without actually getting a point, and that would increase his +/-.

I must say though I still find it a concern where you mistakenly state that Kunitz was getting points for playing with Cheechoo and Thornton. They played for San Jose, Kunitz is on Anaheim… playing with Selanne and McDonald.

I don’t disagree about the benefit of linemates, but I notice you completely ignored my point about the players playing with Kunitz and Lehtinen. Are you saying that the players on Kunitz’ line were doing MORE to increase his +/- than the players playing with Lehtinen? You’re implying they did. It is a tad distressing when you read a comment to respond to it and you ignore the whole discussion of the 5 guys playing regularly with him and insert 2 players on an entirely different team.

29. David Johnson says:

I didn’t address it because my point of pointing out players was not to get into a Lehtonen vs Kunitz debate but to point out the flaws of +/-. But to be honest, I have no clue which of Lehtonen or Kunitz got the most benefit from playing with good players or against weak players. I can make guestimates and we can debate back and forth but I really don’t know for sure. That is really hard to evaluate without knowing for sure how often he played with who and against who. This the need for a statistic.

30. Steve says:

But the fact that you’re weighting player performance based on expectations for production depending on who they share the ice with requires you to make determinations about the levels of expected production of those same players. They’re mutually dependent… you can’t isolate an expected production value without taking into account the expectations for everyone on the ice. I just don’t see a way to consider one variable independent of the other. Every player is dependent on every other player, that’s what makes it a team sport more than an individual. Your original point about the hitter vs. pitcher match up in baseball was entirely valid, but unless you plan on dissecting the Shooter vs. Goalie confrontation in a penalty shot/shoot-out situation I don’t see how anything in hockey approaches that same level of one on one confrontation.

31. Steve says:

Oh and one more point about baseball… there’s NO single statistic in baseball that isolates offensive production/defensive production etc. Nobody has found the one stat in that sport to determine the best overall player. If it’s not done in a sport that is that caught up in one on one match ups, why would you have any belief it’s possible to isolate individuals in such an involved team sport as hockey?

32. David Johnson says:

Baseball is a bit different because offense and defense are independent of each other. How well a player hits is independent of how well a player fields.

In baseball there have been attempts to factor out things like ballpark differences and maybe even strength of opponent differences. For pitchers there are things stats like WinShares that attempt to allow people to compare pitchers across eras etc.

Getting back to hockey, there are ways of isolating a players performance. If Player A plays better when he is playing with Player B then Player B might be a better player than the other players that Player A has played with. If Player A plays better against Player C than when he isn’t playing against Player C then maybe Player C is not as good of a player as the rest of the players Player A plays against.

33. “Getting back to hockey, there are ways of isolating a players performance. If Player A plays better when he is playing with Player B then Player B might be a better player than the other players that Player A has played with. If Player A plays better against Player C than when he isn’t playing against Player C then maybe Player C is not as good of a player as the rest of the players Player A plays against.”

Problem is hockey doesn’t work like that it often looks like:
Player A: 1.1xB, 1.2xC, 0.9xD
Player B: 1.1xC, 1.3xD, 0.9xA
Player C: 0.8xA, 0.9xB, 1.1xD
Player D: 1.1xA, 0.7xB, 0.9xC
How good is player D?
Since A is worse than D is A worse than D?
You can assume equal ice time with each skater.
What I’m trying to say is that in hockey it’s rare for a player to be difinitively better than everyone on the team (players who are: Gelinas) get really high scores.

I’ll be creating my own method that doesn’t solve the problem completely, but does a pretty good job. I hope to have some results next week…

34. David Johnson says:

I don’t understand. What do the numbers stand for in your system?

35. Steve says:

Ok so you’re saying you have floating values for every player when they play WITH every other player on their team, and when they play AGAINST every other player in the league?

I think what JavaGeek is trying to say is the variation in the impact of a player on other players might be relatively impossible to eliminate from the process. If one player (A) is having a positive impact on half the players on their team but a negative impact on the other half… and then another player (B) is having a positive impact on the half that (A) negatively impacts but a negative impact on the half that (A) positively impacted, how do you determine which is the better player???

36. David Johnson says:

Yes, I keep track of who everyone plays with and against and how they do in those situations.

As for the situation you describe, I am not sure how frequent that occurs. I do believe that some players probably play better with certain types of other players. For example, we have Playmaker P1 and Playmaker P2 and Sniper S1 and Sniper S2. With these duos we might get circular situations but still I think eventually it all averages out and generally the player I want on my team is the player that performs best with the most other players. A sniper is probably more dependent on the playmaker than the other way around. Dany Heatley scores half of his goals on one-timers from the slot and needs a good playmaker (Spezza) to get him the puck a the right time and location.

37. You’re working with the data and you consider the possibility of a bad player “helping” a good player as the exception not the norm?

Did you know that statistical error (randomness) tells us that teams have a 95% confidence interval of 30 goals. Meaning if the some average team (250 GF/250 GA) played the same teams/seasons/players 20 times you’d expect 19 times out of 20 that that team would be between 220 and 280 goals for and against. This is just the result of scoring being Poisson.

Or an increase in scoring by 12% or a decrease of 12% is perfectly within the realm of expected error (you should expect scoring to vary like this from year to year).

Of course the less ice time you have the error becomes a much larger part of the information (larger percentage). For example the Sedins were together for about 41000 seconds together and 5000 seconds apart (1.5 hrs). The time apart has 100% error (in relation to 95% confidence interval range), meaning any information taken from the time Sedin’s spent apart is worthless (it’s all indistinguishable from error).

Try to explain this data in relation to your previous response:
http://www.sfu.ca/%7Ecboersma/canucks.EV2.htm
This explains this table:
http://hockeynumbers.blogspot.com/2006/07/introduction-shift-analysis.html
Yes there a number of players who always do better, but it certainly would be complicated to figure out the middle players.

38. David Johnson says:

I am aware that there errors exist and that the more data you have the smaller the expected errors will be. But we have to work with what we have. Ideally you would want to work with a 95% confidence interval but sometimes you might have to consider that 90% or 85% is reasonable for identifying general trends most of the time. Just because there is a significant error at the 95% confidence interval doesn’t mean that the analysis is useless and should be abandoned. And while there are pairs like the Sedins who play together most of the time and it becomes difficult to isolate their contributions independent of each other these ‘duos’ are not the norm. At most they would probably be one duo per team amongst the forwards. Many teams would not have and duos at all as some coaches are line jugglers.

But again, this does not make the information or analysis useless.

Yes there a number of players who always do better, but it certainly would be complicated to figure out the middle players.

Yeah, and you know what? Most of those middle players likely have very similar values anyway.

39. Steve says:

If the error is that great with a smaller sample size, wouldn’t it explain how players who over the long run are probably average end up at the top of your rating scale? Perhaps the scale isn’t useful over anything less than a season in length. It might be more useful to determine using data from MULTIPLE seasons, and then give overall rankings that accumulate over time. I know you only have data starting from January of last season, but perhaps rather than figuring out numbers from this year only, you should add this season’s numbers on to last season’s, just to minimize the error. Just a suggestion.