Jul 042014
 

The other day I put up a post on Mike Weaver’s and Bryce’s Salvador’s possible ability to boost their goalies save percentage and I followed it up with a post on the Maple Leafs defensemen where we saw Phaneuf, Gunnarsson, Gleason and Gardiner all seemingly able to do so as well while Robidas had the reverse effect (lowering goalie save percentage). This got some fight back from the analytics community suggesting this is not possible. My question to them is, why not?

Their answer is that if you do year over year analysis of a players on-ice save percentage or a year over year analysis of a players on-ice save percentage relative to their teams you will find almost no correlation. While this is true I claim that this is not sufficient to prove that such a talent does not exist. Here is why.

We Know Players Can and Do Impact Save %

The most compelling argument that players can and do impact save % is that we see it happening all the time and it is fully accepted among the hockey analytics community. It is known as score effects. Score effects are a well entrenched concept in hockey analytics.  It is why we often look at 5v5 “close” or 5v5 tied statistics instead of just 5v5 statistics. Generally speaking, the impact score effects have is that the trailing teams usually experiences an increase in shot rate along with a decrease in shooting percentage while the team protecting the lead experiences a decrease in shot rate but an increase in shooting percentage. The following table shows the Boston Bruins shooting and save percentages when tied, leading and trailing over the past 7 seasons combined.

. Tied Leading Trailing
Shooting% 7.27% 9.14% 7.66%
Save% 93.36% 93.86% 92.53%

The difference in the Bruins save percentage between leading and trailing is 1.33%. This is the difference between a .923 save percentage goalie and a .910 save percentage goalie which is the difference between an elite goalie and a below average goalie. That is not insignificant. Is this the goalies fault or does it have something to do with the players in front of him? The latter seems most likely. It makes sense that when protecting the lead the players take fewer risks in an attempt to generate offense and in return give up fewer good scoring chances against albeit maybe more chances in total. Conversely, the team playing catch up take more offensive risks so they end up giving up more quality scoring chances against. This is reflected in their teams save and shooting percentages when leading and trailing.

So, now if a team can play a style that boosts the team save percentage when they are protecting a lead, why is it so inconceivable that a player could see the same impact in his on-ice save percentage if that player plays that style of hockey all the time? If Mike Weaver and Bryce Salvador play the same style all the time that teams play when protecting a lead, why can they not boost on-ice save percentage? There is no reason they can’t.

It is Difficult to Detect because Individual Players Don’t Have a lot of Control of Outcomes

The average player’s individual ability to influence of what happens on the ice is actually fairly small as there are also 9 other skaters and 2 goalies on the ice with him. At best you can say the average player has a ~10% impact on outcomes while he is on the ice. That isn’t much. Last week James Mirtle tweeted a link to Connor Brown’s hockeydb.com page as evidence why +/- is a useless statistic. Over the course of three OHL season’s Brown’s +/- went from -72 to -11 to +44. I suggested to Mirtle that if this is the criteria for tossing out stats we can toss out a lot of stats including corsi% because most stats are highly team/linemate dependent. When challenged that this dramatic of reversal is not seen in corsi% I cited David Clarkson as an example.  In 2012-13 Clarkson was 4th in CF% but in 2013-14 he was 33rd (of 346) in CF%. From one year to the next he went from 4th best to 14th worst. Why is this? WEll, Clarkson essentially moved from playing with good corsi players on a good corsi team to playing with bad corsi players on a bad corsi team. No matter how much puck possession talent Clarkson has (or hasn’t) his talent doesn’t dominate over the talent level of the 4 team mates he is on the ice with.

Now think about how many players change teams from one year to the next and think about how many players get moved up and down a line up and change line mates from one season to next. It is not an insignificant number. TSN’s UFA tracker currently has 109 UFA’s getting signed starting July 1st, the majority of them changing teams. There are only ~800 NHL players (regulars and depth players) in a season so that is pretty significant turnover. Some teams turn over a quarter to half their line up while others stay largely the same. With that much roster turnover and with so little ability for a single player to drive outcomes it should be expected that the majority of statistics see relatively high “regression”. Regression doesn’t mean lack of individual talent though.

Think of this scenario. We have a player with an average ability to boost on-ice save percentage and he has been playing on a team with a number of players who are good at boosting on-ice save percentage but generally speaking he doesn’t play with those players. Under this scenario it will appear that the player is poor at boosting on-ice save percentage because he is being compared to  players who are good at it. Now that player moves to another team who isn’t very good at boosting on-ice save percentage. Now that same average player will look like he is a good player because he has a better on-ice shooting percentage than his teammates. The result is little year over year consistency but that doesn’t mean there aren’t talent differences among players.

Hockey is not like baseball which is a series of one-on-one matchups between pitcher and batter or isolated attempts to make a fielding play on a hit ball. Outcomes in hockey are completely interdependent on up to 12 other players on the ice. QoT is the largest driver of a players statistics in hockey. Only when we factor out QoT completely can we truly be able to identify every players talent level for any metric we measure. This is a kind of like the chicken and an egg problem though because to identify a players talent level we need to know the talent level of their team mates which in turn required knowledge of his own talent level. We can’t just look at year over year regression to isolate talent level.

Comments

The “team” aspect in hockey is more significant than any other sport and any particular players statistics are largely driven by the quality of his team mates. Even more than teammates, style of play can be a significant factor in a players statistics. The quality of the players that a particular player plays with is a function of both the team the plays on and the role (offensive first line vs defensive third line) he is playing on the team and this is maybe the greatest driver of a players statistics. This is why David Clarkson can be a Corsi king in New Jersey and a Corsi dud in Toronto. It also accounts for why James Neal can be a 25 goal guy playing on the first line in Dallas to a 40 goal guy in Pittsburgh (and probably back to a 25 guy guy in Nashville next year).  This also accounts for why year over year correlation in many stats is not very good despite there being measurable differences in the talent that that stat is measuring. Significant statistical regression is not sufficient, in my opinion, to conclude insignificant controllable talent if no significant attempt to completely isolate individual contribution to team results has been successfully made.

Just for fun, here is a chart of Lidstrom’s on-ice save percentage vs team save percentage. It is pretty outstanding that an offensive defenseman can do this too.

LidstromOnOffSavePct

 

Jul 012014
 

The other day I commented on twitter that I would be happy if the Leafs signed defenseman Mike Weaver because I think he is a defensive defenseman that I think the Leafs could really use. I have thought of Mike Weaver as a premier defensive defenseman for quite some time now. I always seem to get a little flak over it but that’s fine, I can handle it. For example, as a response to my Weaver comment on twitter Eric Tulsky thought it would be prudent to point out a “flaw” in my thought process.

 

And of course, Tyler Dellow never passes up an opportunity to take a jab at me (or anyone who he disagrees with) took the opportunity to re-tweet it.

Now, of course I had thought of responding with a tweet to the effect of “Florida’s save percentage was probably is a bit of a factor in that regression” but I didn’t want to get into a twitter debate at that moment and I was confident I could come up with more concrete evidence. So here is that evidence.

SavePercentageWeaverOnOffIce

The above chart shows the save percentage of Weaver’s team when Weaver is on the ice vs when Weaver is not on the ice including only games in which Weaver has played in (i.e. it is better than just using team save percentage for that season and also allows us to combine his time in Florida and Montreal last season). As you can see, there has only been one season in the last 7 in which his team had a worse save percentage when he is on the ice than not. That is reasonably compelling evidence. It’s difficult to say what happened that season but his main defense partners were a young Dmitry Kulikov and Keaton Ellerby so maybe that was a factor. An investigation of Kulikov’s and Ellerby’s impact on save percentage over the years may help us identify why Weaver slipped that year. It could have been a nagging injury as well. Or, it could just be randomness associated with save percentage.

Regardless of the “reason” for the slide in 2011-12 it is pretty difficult to argue that there has been significant “regression” the past 3 seasons as Tulsky and Dellow so eagerly wanted to point out as the past 2 seasons Weaver has seemingly had a significant positive impact on his teams save percentage. Since I made that statement there has been one seasons of “regression” so to speak and two seasons in support of my claim. I guess that means it is 2-1 in my favour. It continues to appear that Weaver is a good defenseman who can suppress shot quality against.

Another defenseman I have identified as a defenseman who possibly can suppress opposition save percentage is Bryce Salvador. Here is Salvador’s on/off save percentage chart similar to Weaver’s above (2010-11 is missing as Salvador missed the season due to injury).

SavePercentageSalvadorOnOffIce

Salvador’s on-ice save percentage has been better than the teams save percentage every year since 2007-08. Regression? Doesn’t seem to be.

To summarize, there are a lot of instances where if we simply do a correlation of stats from one year to the next or  make observations of future performance relative to past performance we see the appearance of regression. In fact, the raw stats do in fact regress. That doesn’t necessarily mean the talent doesn’t exist, just that we haven’t been able to properly isolate the talent. The talent of the individual player is only a small factor in what outcomes occur when he is on the ice (a single player is just one of 12 players on the ice during typical even strength play) so it is difficult to identify without attempting to account for these other factors (quality of team mates in particular).

Possession and shot generation/suppression is important, but ignore the percentages at your peril. They can matter a lot in player evaluation.

 

Jun 122014
 

The rumour is out there that Sunny Mehta has been hired as Director of Hockey Analytics of the New Jersey Devils (if true, a big congrats to Sunny). This sparked some twitter discussion about the Devils and analytics and Devils defensemen including Bryce Salvador.

I have been a bit of a fan of Salvador, at least statistically, though clearly there are a lot of Devils fans that do not like him and I think it is because of a focus on corsi. One person tweeted me an image of Salvador’s corsi rel % suggesting it was “pretty ugly”. While maybe true the game isn’t about Corsi it is about goals. Here is what I know about Salvador. In 5v5close situations he led the Devils defensemen in on-ice save percentage last season, the season before, and the season before that. He missed 2010-11 due to injury but in 2009-10 he was second best trailing only Andy Greene, his regular defense partner. Either he is extremely lucky (every year) or he is doing something right.

Lets look at this a different way. Over the past 3 seasons Bryce Salvador has had the third best 5v5close save percentage in the league when he is on the ice despite the Devils ranking 23rd in team save percentage. The two players ahead of him play for Boston (Dougie Hamilton) and Los Angeles (Willie Mitchell) who have significantly better goaltending (3rd and 8th best 5v5close save percentages over past 3 seasons) and again, they played in front of far better goaltending.

In February 2012 I wrote an article attempting to quantify a defenders effect on save percentage and in it I identified Salvador as one of the best defensemen at boosting his teams save percentage. In the 2 seasons since he has done nothing but support that claim.

So, what does this all mean? Well, it takes a player who had a team worst 15.9 CA/20 in 5v5close situations this past season to a team best 0.49 GA/20.  Over the past 3 seasons only Dougie Hamilton (Boston), Willie Mitchell (Los Angeles) and Alec Martinez (Los Angeles) have seen goals scored against them at a lower rate than Bryce Salvador.

I know the majority of people are on the corsi bandwagon these days and some will dismiss any argument that runs counter to it but I think the evidence is clearly on Salvador’s side here. All evidence suggest he is really good as suppressing opposition shot quality and in turn suppressing the number of goals scored against the Devils. If I were the new Director of Hockey Analytics for the Devils I wouldn’t be recommending getting rid of Salvador.