The other day I commented on twitter that I would be happy if the Leafs signed defenseman Mike Weaver because I think he is a defensive defenseman that I think the Leafs could really use. I have thought of Mike Weaver as a premier defensive defenseman for quite some time now. I always seem to get a little flak over it but that’s fine, I can handle it. For example, as a response to my Weaver comment on twitter Eric Tulsky thought it would be prudent to point out a “flaw” in my thought process.

@hockeyanalysis @SkinnyPPPhish Below-average on-ice sv% since this was posted. #Regression pic.twitter.com/rhSD65J530

— Eric T. (@BSH_EricT) June 26, 2014

And of course, Tyler Dellow never passes up an opportunity to take a jab at me (or anyone who he disagrees with) took the opportunity to re-tweet it.

Now, of course I had thought of responding with a tweet to the effect of “Florida’s save percentage was probably is a bit of a factor in that regression” but I didn’t want to get into a twitter debate at that moment and I was confident I could come up with more concrete evidence. So here is that evidence.

The above chart shows the save percentage of Weaver’s team when Weaver is on the ice vs when Weaver is not on the ice including only games in which Weaver has played in (i.e. it is better than just using team save percentage for that season and also allows us to combine his time in Florida and Montreal last season). As you can see, there has only been one season in the last 7 in which his team had a worse save percentage when he is on the ice than not. That is reasonably compelling evidence. It’s difficult to say what happened that season but his main defense partners were a young Dmitry Kulikov and Keaton Ellerby so maybe that was a factor. An investigation of Kulikov’s and Ellerby’s impact on save percentage over the years may help us identify why Weaver slipped that year. It could have been a nagging injury as well. Or, it could just be randomness associated with save percentage.

Regardless of the “reason” for the slide in 2011-12 it is pretty difficult to argue that there has been significant “regression” the past 3 seasons as Tulsky and Dellow so eagerly wanted to point out as the past 2 seasons Weaver has seemingly had a significant positive impact on his teams save percentage. Since I made that statement there has been one seasons of “regression” so to speak and two seasons in support of my claim. I guess that means it is 2-1 in my favour. It continues to appear that Weaver is a good defenseman who can suppress shot quality against.

Another defenseman I have identified as a defenseman who possibly can suppress opposition save percentage is Bryce Salvador. Here is Salvador’s on/off save percentage chart similar to Weaver’s above (2010-11 is missing as Salvador missed the season due to injury).

Salvador’s on-ice save percentage has been better than the teams save percentage every year since 2007-08. Regression? Doesn’t seem to be.

To summarize, there are a lot of instances where if we simply do a correlation of stats from one year to the next or make observations of future performance relative to past performance we see the appearance of regression. In fact, the raw stats do in fact regress. That doesn’t necessarily mean the talent doesn’t exist, just that we haven’t been able to properly isolate the talent. The talent of the individual player is only a small factor in what outcomes occur when he is on the ice (a single player is just one of 12 players on the ice during typical even strength play) so it is difficult to identify without attempting to account for these other factors (quality of team mates in particular).

Possession and shot generation/suppression is important, but ignore the percentages at your peril. They can matter a lot in player evaluation.

So Weaver has a four-year stretch where he averages a Sv% rel of something north of +2.5% (just eyeballing here — it looks like roughly +3, +1, +4, +2 in a stretch where he played 55, 58, 77, and 82 games) and you post “no regressing to the mean there” as a sarcastic jab at the well-established notion of regression.

Then he follows that up with a three-year stretch where he averages a Sv% rel of something like +0.5% (again, eyeballed: guessing -0.8, +2, +1.5 in seasons where he played 82, 27, and 72 games) and you put up a self-congratulatory post about the correctness of your position?

Neat.

Looks to me like his performance regressed about 80% of the way toward the mean. I previously suggested that at this sample size, we should project about 85% regression. Yet somehow you think this singular data point supports your “no regressing to the mean” position.

In the article you link above all you do is conduct a regression between on-ice shooting percentage for a 3-year period vs the another 3-year period and then conclude that since there is little to no correlation there is little to no value in on-ice save percentage. You know what? You are right. On-ice save percentage is not valuable in predicting future on-ice save percentage. Where you are wrong is assuming that means that players can’t influence on-ice save percentage.

Your tweet suggested that Weaver went from above average on-ice save percentage when I looked at it to below average on-ice save percentage the past 3 seasons. That is technically true but it doesn’t mean Weaver can’t influence on-ice save percentage.

Now it seems you are trying to claim your statement was right because you can point out that Weaver did regress, albeit not all the way to the mean. So what you are saying is that defensemen can suppress shooting percentage? If that is the case, maybe you should put some effort into quantifying how much they can. You can’t just do a 3 year vs 3 year correlation and assume that that is how much they regress to the mean. You need to isolate true talent from changing circumstances from true randomness.

This is not unlike the shooting percentage debate we had. I tried to claim shooting percentage mattered, you tried to downplay the effect by doing correlations and showing significant year(s) over year(s) regression. Gabe wanted to assume shot quality was practically non-existent. Then when you actually applied the regression you regressed everyone to a different mean based on their TOI which meant regressing to means varying from a low of (if I recall correctly) 6.2% to a high of 9.7%. To me, that is a significant range. When push came to shove you change the equation from regressing to the mean to regressing to some mean (which varies widely across players) more appropriate to the skill level of the player. You essentially proved my point all along that shot quality matters and cannot be ignored.

In your tweet you implied Weaver regressed from well above the mean to below the mean. Now you are moving the goal posts again and suggesting he is regressing towards the mean, not below. Who knows, maybe some day we will have a better handle of how to identify an appropriate mean for a particular “type” of defenseman like we have for forwards and shooting percentage based on TOI. Assuming everyone regresses significantly to the same mean and being closed minded about other possibilities and taking cheap jabs at anyone who disagrees with you is not how we get there though. Why not attempt to understand what clearly seems to be an “anomaly” season rather than just chalk it up to regression or randomness or luck or whatever you think it is?

What I haven’t brought is that Weaver has the 5th highest 5v5close average opposition shooting percentage over the past 3 seasons (only McDonagh, Girardi, Chara and Phaneuf had a higher average opponent shooting percentage) and generally speaking one of the toughest offensive QoC in the league (the fact that his QoC got tougher is a minor factor, but still factor). His QoC has gotten stronger the past 3 seasons which accounts for a small part of any regression we may see. There may be some regression due to the aging process as well and quality of defense partners may be a significant factor too. Some of this is, in theory anyway, quantifiable I find it troubling that so many people are willing to write it up as randomness or unexplained regression without any attempt to understand what is really going on.

I’ll end with these stats.

Over Weaver’s past 7 seasons he has boosted save percentage by an average of 1.67% and saved an average of 7.6 goals per season based on number of shots against he has been on the ice for. That’s good. Over the past 2 seasons that number is 1.58% and 5.8 goals per season (he missed some games in 2012-13 which accounts for the drop to 5.8 goals per season).

Lidstom’s last 5 seasons in the NHL he boosted save percentage by an average of 1.74% or about 10 5v5 goals per season based on his ice time and shot against totals. That is significant. Lidstrom was a significant talent and good at suppressing shot quality against.

Where did you get the off-ice save percentage numbers from, David?

I calculate them myself but I haven’t published them anywhere yet.

Cool. Will they be a part of the summer update?

What are your thoughts on a forward’s ability to impact on-ice SV%? Parkatti’s done some work on this matter and I’d be curious to see what you think of his conclusions.

Basically, I’m looking for ways to defend Tomas Plekanec to my fancy stat buddies. (I realize this is totally the wrong way to go about analysis, but I’m smitten with #14). Good GF percentages, fueled by strong on-ice save percentages, are all I’ve got.

It is an interesting post. I haven’t looked at forwards very much yet but his results make me more inclined to want to. It seems like there could be something there. Two comments though:

1. Players move around a fair bit. Especially middle of the road or worse ones. On one season they could be on a team with several players that are good at suppressing shooting percentage against and have a poor OISv% differential while the next season they are on a team that is bad at it and have a good OISv% Differential. While OISv% Differential is a step forward, it shouldn’t be viewed as a complete isolation of a players ability to control OISv% Differential. Players that are better than the majority of the NHL will always appear good and players that are worse than the majority of the NHL will always appear bad (and probably disappear from the NHL). All the guys in the middle (who are most likely to change teams a lot) might appear better or worse depending on the team they happen to be on. That is something that is extremely difficult to account for.

2. I have issues with the second half of the post where he is attempting to factor out quality of competition based on ice time. The main reason is that I have seen little evidence that QoC has a major impact on a players performance over the course of a season. The differences in QoC for most players is pretty insignificant compared to QoT or any other factor. Furthermore, if players who are good at OISv% Differential typically play on the third line and get third line minutes then the TOI adjustment changes the results from comparing the performance of those third liners to everyone int he league to comparing those third liners to players of similar skill at OISv% Differential.

Where I think the mistake was made was in the analysis of the first table of forwards OISv% Differential.

You could look at it that way. Or you could look at it as “Defensive third liners are good at OISh% Differential and good offensive forwards are poor at it” which is a very different analysis of the table. Put differently, players who play defense-first and don’t take risks in the offensive zone are better OISh% Differential than those who play offense and take more risks in the offensive zone.” The mistake was not believing what the table was telling him and looking to find a way to minimize the extent of the results so it fits more in line with his preexisting views of who is good and who isn’t.

I will dig into this a little further the next few days and try and identify some more players that are either typically good or typically bad at this skill.

As for Plekanec, the past 2 seasons he has shown some ability to be able to boost on ice save percentage but prior to that he was pretty neutral. Not sure what caused that change. It coincides with Therrien so maybe coaching/usage is a factor.