The other day I put up a post on Mike Weaver’s and Bryce’s Salvador’s possible ability to boost their goalies save percentage and I followed it up with a post on the Maple Leafs defensemen where we saw Phaneuf, Gunnarsson, Gleason and Gardiner all seemingly able to do so as well while Robidas had the reverse effect (lowering goalie save percentage). This got some fight back from the analytics community suggesting this is not possible. My question to them is, why not?
Their answer is that if you do year over year analysis of a players on-ice save percentage or a year over year analysis of a players on-ice save percentage relative to their teams you will find almost no correlation. While this is true I claim that this is not sufficient to prove that such a talent does not exist. Here is why.
We Know Players Can and Do Impact Save %
The most compelling argument that players can and do impact save % is that we see it happening all the time and it is fully accepted among the hockey analytics community. It is known as score effects. Score effects are a well entrenched concept in hockey analytics. It is why we often look at 5v5 “close” or 5v5 tied statistics instead of just 5v5 statistics. Generally speaking, the impact score effects have is that the trailing teams usually experiences an increase in shot rate along with a decrease in shooting percentage while the team protecting the lead experiences a decrease in shot rate but an increase in shooting percentage. The following table shows the Boston Bruins shooting and save percentages when tied, leading and trailing over the past 7 seasons combined.
The difference in the Bruins save percentage between leading and trailing is 1.33%. This is the difference between a .923 save percentage goalie and a .910 save percentage goalie which is the difference between an elite goalie and a below average goalie. That is not insignificant. Is this the goalies fault or does it have something to do with the players in front of him? The latter seems most likely. It makes sense that when protecting the lead the players take fewer risks in an attempt to generate offense and in return give up fewer good scoring chances against albeit maybe more chances in total. Conversely, the team playing catch up take more offensive risks so they end up giving up more quality scoring chances against. This is reflected in their teams save and shooting percentages when leading and trailing.
So, now if a team can play a style that boosts the team save percentage when they are protecting a lead, why is it so inconceivable that a player could see the same impact in his on-ice save percentage if that player plays that style of hockey all the time? If Mike Weaver and Bryce Salvador play the same style all the time that teams play when protecting a lead, why can they not boost on-ice save percentage? There is no reason they can’t.
It is Difficult to Detect because Individual Players Don’t Have a lot of Control of Outcomes
The average player’s individual ability to influence of what happens on the ice is actually fairly small as there are also 9 other skaters and 2 goalies on the ice with him. At best you can say the average player has a ~10% impact on outcomes while he is on the ice. That isn’t much. Last week James Mirtle tweeted a link to Connor Brown’s hockeydb.com page as evidence why +/- is a useless statistic. Over the course of three OHL season’s Brown’s +/- went from -72 to -11 to +44. I suggested to Mirtle that if this is the criteria for tossing out stats we can toss out a lot of stats including corsi% because most stats are highly team/linemate dependent. When challenged that this dramatic of reversal is not seen in corsi% I cited David Clarkson as an example. In 2012-13 Clarkson was 4th in CF% but in 2013-14 he was 33rd (of 346) in CF%. From one year to the next he went from 4th best to 14th worst. Why is this? WEll, Clarkson essentially moved from playing with good corsi players on a good corsi team to playing with bad corsi players on a bad corsi team. No matter how much puck possession talent Clarkson has (or hasn’t) his talent doesn’t dominate over the talent level of the 4 team mates he is on the ice with.
Now think about how many players change teams from one year to the next and think about how many players get moved up and down a line up and change line mates from one season to next. It is not an insignificant number. TSN’s UFA tracker currently has 109 UFA’s getting signed starting July 1st, the majority of them changing teams. There are only ~800 NHL players (regulars and depth players) in a season so that is pretty significant turnover. Some teams turn over a quarter to half their line up while others stay largely the same. With that much roster turnover and with so little ability for a single player to drive outcomes it should be expected that the majority of statistics see relatively high “regression”. Regression doesn’t mean lack of individual talent though.
Think of this scenario. We have a player with an average ability to boost on-ice save percentage and he has been playing on a team with a number of players who are good at boosting on-ice save percentage but generally speaking he doesn’t play with those players. Under this scenario it will appear that the player is poor at boosting on-ice save percentage because he is being compared to players who are good at it. Now that player moves to another team who isn’t very good at boosting on-ice save percentage. Now that same average player will look like he is a good player because he has a better on-ice shooting percentage than his teammates. The result is little year over year consistency but that doesn’t mean there aren’t talent differences among players.
Hockey is not like baseball which is a series of one-on-one matchups between pitcher and batter or isolated attempts to make a fielding play on a hit ball. Outcomes in hockey are completely interdependent on up to 12 other players on the ice. QoT is the largest driver of a players statistics in hockey. Only when we factor out QoT completely can we truly be able to identify every players talent level for any metric we measure. This is a kind of like the chicken and an egg problem though because to identify a players talent level we need to know the talent level of their team mates which in turn required knowledge of his own talent level. We can’t just look at year over year regression to isolate talent level.
The “team” aspect in hockey is more significant than any other sport and any particular players statistics are largely driven by the quality of his team mates. Even more than teammates, style of play can be a significant factor in a players statistics. The quality of the players that a particular player plays with is a function of both the team the plays on and the role (offensive first line vs defensive third line) he is playing on the team and this is maybe the greatest driver of a players statistics. This is why David Clarkson can be a Corsi king in New Jersey and a Corsi dud in Toronto. It also accounts for why James Neal can be a 25 goal guy playing on the first line in Dallas to a 40 goal guy in Pittsburgh (and probably back to a 25 guy guy in Nashville next year). This also accounts for why year over year correlation in many stats is not very good despite there being measurable differences in the talent that that stat is measuring. Significant statistical regression is not sufficient, in my opinion, to conclude insignificant controllable talent if no significant attempt to completely isolate individual contribution to team results has been successfully made.
Just for fun, here is a chart of Lidstrom’s on-ice save percentage vs team save percentage. It is pretty outstanding that an offensive defenseman can do this too.