Jul 042014
 

The other day I put up a post on Mike Weaver’s and Bryce’s Salvador’s possible ability to boost their goalies save percentage and I followed it up with a post on the Maple Leafs defensemen where we saw Phaneuf, Gunnarsson, Gleason and Gardiner all seemingly able to do so as well while Robidas had the reverse effect (lowering goalie save percentage). This got some fight back from the analytics community suggesting this is not possible. My question to them is, why not?

Their answer is that if you do year over year analysis of a players on-ice save percentage or a year over year analysis of a players on-ice save percentage relative to their teams you will find almost no correlation. While this is true I claim that this is not sufficient to prove that such a talent does not exist. Here is why.

We Know Players Can and Do Impact Save %

The most compelling argument that players can and do impact save % is that we see it happening all the time and it is fully accepted among the hockey analytics community. It is known as score effects. Score effects are a well entrenched concept in hockey analytics.  It is why we often look at 5v5 “close” or 5v5 tied statistics instead of just 5v5 statistics. Generally speaking, the impact score effects have is that the trailing teams usually experiences an increase in shot rate along with a decrease in shooting percentage while the team protecting the lead experiences a decrease in shot rate but an increase in shooting percentage. The following table shows the Boston Bruins shooting and save percentages when tied, leading and trailing over the past 7 seasons combined.

. Tied Leading Trailing
Shooting% 7.27% 9.14% 7.66%
Save% 93.36% 93.86% 92.53%

The difference in the Bruins save percentage between leading and trailing is 1.33%. This is the difference between a .923 save percentage goalie and a .910 save percentage goalie which is the difference between an elite goalie and a below average goalie. That is not insignificant. Is this the goalies fault or does it have something to do with the players in front of him? The latter seems most likely. It makes sense that when protecting the lead the players take fewer risks in an attempt to generate offense and in return give up fewer good scoring chances against albeit maybe more chances in total. Conversely, the team playing catch up take more offensive risks so they end up giving up more quality scoring chances against. This is reflected in their teams save and shooting percentages when leading and trailing.

So, now if a team can play a style that boosts the team save percentage when they are protecting a lead, why is it so inconceivable that a player could see the same impact in his on-ice save percentage if that player plays that style of hockey all the time? If Mike Weaver and Bryce Salvador play the same style all the time that teams play when protecting a lead, why can they not boost on-ice save percentage? There is no reason they can’t.

It is Difficult to Detect because Individual Players Don’t Have a lot of Control of Outcomes

The average player’s individual ability to influence of what happens on the ice is actually fairly small as there are also 9 other skaters and 2 goalies on the ice with him. At best you can say the average player has a ~10% impact on outcomes while he is on the ice. That isn’t much. Last week James Mirtle tweeted a link to Connor Brown’s hockeydb.com page as evidence why +/- is a useless statistic. Over the course of three OHL season’s Brown’s +/- went from -72 to -11 to +44. I suggested to Mirtle that if this is the criteria for tossing out stats we can toss out a lot of stats including corsi% because most stats are highly team/linemate dependent. When challenged that this dramatic of reversal is not seen in corsi% I cited David Clarkson as an example.  In 2012-13 Clarkson was 4th in CF% but in 2013-14 he was 33rd (of 346) in CF%. From one year to the next he went from 4th best to 14th worst. Why is this? WEll, Clarkson essentially moved from playing with good corsi players on a good corsi team to playing with bad corsi players on a bad corsi team. No matter how much puck possession talent Clarkson has (or hasn’t) his talent doesn’t dominate over the talent level of the 4 team mates he is on the ice with.

Now think about how many players change teams from one year to the next and think about how many players get moved up and down a line up and change line mates from one season to next. It is not an insignificant number. TSN’s UFA tracker currently has 109 UFA’s getting signed starting July 1st, the majority of them changing teams. There are only ~800 NHL players (regulars and depth players) in a season so that is pretty significant turnover. Some teams turn over a quarter to half their line up while others stay largely the same. With that much roster turnover and with so little ability for a single player to drive outcomes it should be expected that the majority of statistics see relatively high “regression”. Regression doesn’t mean lack of individual talent though.

Think of this scenario. We have a player with an average ability to boost on-ice save percentage and he has been playing on a team with a number of players who are good at boosting on-ice save percentage but generally speaking he doesn’t play with those players. Under this scenario it will appear that the player is poor at boosting on-ice save percentage because he is being compared to  players who are good at it. Now that player moves to another team who isn’t very good at boosting on-ice save percentage. Now that same average player will look like he is a good player because he has a better on-ice shooting percentage than his teammates. The result is little year over year consistency but that doesn’t mean there aren’t talent differences among players.

Hockey is not like baseball which is a series of one-on-one matchups between pitcher and batter or isolated attempts to make a fielding play on a hit ball. Outcomes in hockey are completely interdependent on up to 12 other players on the ice. QoT is the largest driver of a players statistics in hockey. Only when we factor out QoT completely can we truly be able to identify every players talent level for any metric we measure. This is a kind of like the chicken and an egg problem though because to identify a players talent level we need to know the talent level of their team mates which in turn required knowledge of his own talent level. We can’t just look at year over year regression to isolate talent level.

Comments

The “team” aspect in hockey is more significant than any other sport and any particular players statistics are largely driven by the quality of his team mates. Even more than teammates, style of play can be a significant factor in a players statistics. The quality of the players that a particular player plays with is a function of both the team the plays on and the role (offensive first line vs defensive third line) he is playing on the team and this is maybe the greatest driver of a players statistics. This is why David Clarkson can be a Corsi king in New Jersey and a Corsi dud in Toronto. It also accounts for why James Neal can be a 25 goal guy playing on the first line in Dallas to a 40 goal guy in Pittsburgh (and probably back to a 25 guy guy in Nashville next year).  This also accounts for why year over year correlation in many stats is not very good despite there being measurable differences in the talent that that stat is measuring. Significant statistical regression is not sufficient, in my opinion, to conclude insignificant controllable talent if no significant attempt to completely isolate individual contribution to team results has been successfully made.

Just for fun, here is a chart of Lidstrom’s on-ice save percentage vs team save percentage. It is pretty outstanding that an offensive defenseman can do this too.

LidstromOnOffSavePct

 

Jul 012014
 

The other day I commented on twitter that I would be happy if the Leafs signed defenseman Mike Weaver because I think he is a defensive defenseman that I think the Leafs could really use. I have thought of Mike Weaver as a premier defensive defenseman for quite some time now. I always seem to get a little flak over it but that’s fine, I can handle it. For example, as a response to my Weaver comment on twitter Eric Tulsky thought it would be prudent to point out a “flaw” in my thought process.

 

And of course, Tyler Dellow never passes up an opportunity to take a jab at me (or anyone who he disagrees with) took the opportunity to re-tweet it.

Now, of course I had thought of responding with a tweet to the effect of “Florida’s save percentage was probably is a bit of a factor in that regression” but I didn’t want to get into a twitter debate at that moment and I was confident I could come up with more concrete evidence. So here is that evidence.

SavePercentageWeaverOnOffIce

The above chart shows the save percentage of Weaver’s team when Weaver is on the ice vs when Weaver is not on the ice including only games in which Weaver has played in (i.e. it is better than just using team save percentage for that season and also allows us to combine his time in Florida and Montreal last season). As you can see, there has only been one season in the last 7 in which his team had a worse save percentage when he is on the ice than not. That is reasonably compelling evidence. It’s difficult to say what happened that season but his main defense partners were a young Dmitry Kulikov and Keaton Ellerby so maybe that was a factor. An investigation of Kulikov’s and Ellerby’s impact on save percentage over the years may help us identify why Weaver slipped that year. It could have been a nagging injury as well. Or, it could just be randomness associated with save percentage.

Regardless of the “reason” for the slide in 2011-12 it is pretty difficult to argue that there has been significant “regression” the past 3 seasons as Tulsky and Dellow so eagerly wanted to point out as the past 2 seasons Weaver has seemingly had a significant positive impact on his teams save percentage. Since I made that statement there has been one seasons of “regression” so to speak and two seasons in support of my claim. I guess that means it is 2-1 in my favour. It continues to appear that Weaver is a good defenseman who can suppress shot quality against.

Another defenseman I have identified as a defenseman who possibly can suppress opposition save percentage is Bryce Salvador. Here is Salvador’s on/off save percentage chart similar to Weaver’s above (2010-11 is missing as Salvador missed the season due to injury).

SavePercentageSalvadorOnOffIce

Salvador’s on-ice save percentage has been better than the teams save percentage every year since 2007-08. Regression? Doesn’t seem to be.

To summarize, there are a lot of instances where if we simply do a correlation of stats from one year to the next or  make observations of future performance relative to past performance we see the appearance of regression. In fact, the raw stats do in fact regress. That doesn’t necessarily mean the talent doesn’t exist, just that we haven’t been able to properly isolate the talent. The talent of the individual player is only a small factor in what outcomes occur when he is on the ice (a single player is just one of 12 players on the ice during typical even strength play) so it is difficult to identify without attempting to account for these other factors (quality of team mates in particular).

Possession and shot generation/suppression is important, but ignore the percentages at your peril. They can matter a lot in player evaluation.

 

Sep 152011
 

Over on Pension Plan Puppets there was a brief discussion of some of the top defensive defensemen and I suggested that Mike Weaver has to be considered among the top few defenders in the NHL.  The response was generally along the lines of ‘Mike who?’ and then followed with “he only looks good because he plays in front of Vokoun who may be the best goalie in the NHL.”  My thoughts on Vokoun being over rated aside, the numbers really do support Weaver as being a premiere level defensive defenseman.  Let’s look at some Mike Weaver numbers.

Over the past 4 seasons Mike Weaver played one season in Vancouver, 2 seasons in St. Louis and last season he was in Florida.  During that time there have been 173 defensemen who have played >1500 5v5 close (within 1 goal in first or second period or tied in third period) minutes and of those 173 defensemen Weaver ranks fourth in on ice goals against per 20 minutes of ice time.  He only trails Bryce Salvador, Sean O’Donnell and Paul Martin (3 other under rated defenders IMO).  Ranking 4th is a pretty good argument for why he is a great defender.  So what about the typical excuses for why he might rank so highly?

1.  Goalies make him look good.  Not really.  In the past 4 years he has played 2455:08 minutes of 5v5close ice time, 798:24 (32.5%) in front of Chris Mason, 587:45 (23.9%) in front of Vokoun, 390:58 (15.9%) in front of Luongo, 297:47 (12.1%) in front of Clemmensen, 185:04 (7.5%) in front of Conklin and some time in front of a few other lesser goalies.  At best you can argue he has played 45% of his time behind premiere level goalies (Vokoun and Luongo) with the remaining 55% behind second tier starters (Mason) or third tier starters and backups (Conklin, Clemmensen, etc).  In his year in Vancouver, the Canucks ranked a solid 7th in team goals against average but his 2 years in St. Louis the Blues ranked 12th and 11th and last year the Panthers ranked 14th so while he hasn’t played on any bad defensive teams he hasn’t played on any elite defensive teams either.  It’s difficult to make the case he has had an unusually significant benefit from playing in front of elite goalies or on elite defensive teams.

2. He Plays Easy Minutes.  Not really.  Over the past 4 seasons he ranks 45th of 173 defensemen with 34.1% of his face offs in the defensive zone and last season he started 36.9% of the time in the defensive zone or 21st highest of 157 defensemen with 500 5v5close minutes.  Over the past 4 seasons his opposition goals for per 20 minutes ranks 31st of 173 defensemen so he is seemingly playing against quality offensive forwards.  Last season the forwards he played most against were Ovechkin, Backstrom, Knuble, St. Louis, and Stamkos so yeah, that’s pretty good competition.  Over the past 2 seasons only Chris Phillips and Jay Bouwmeester have played more time on the 4v5 penalty kill than Weaver.  He is trusted playing tough minutes against top competition so the easy minutes argument is not valid.

While we are at it, Mike Weaver is another example why I do not like corsi/fenwick stats.  While Weaver has the 4th best on-ice 5v5close goals against per 20 minutes, he ranks a far less impressive (though still a little above average) 47th in fenwick against per 20 minutes.  The main reason why Weaver is so good defensively is he suppresses shot quality really well.  He ranks 3rd in shooting percentage against (or save percentage) while he is on the ice and he has been consistently above average over the past 4 seasons (6th of 176 in 2007-08, 63rd of 147 in 2008-09, 13th of 154 in 2009-10 and 22nd of 157 in 2010-11).  Three of the past 4 seasons he has been a top 25 defenseman in terms of shooting percentage against and the fourth and worst season he was still in the top half.  Sorry, but there is no ‘regressing to the mean’ there.

Mike Weaver is a premiere, and vastly under rated and under paid ($900,000), defensive defenseman.