Yesterday I wrote an article for MapleLeafsHotStove.com looking at the Leafs performance so far this season in comparison to previous seasons. In it I showed a chart comparing the Leafs CA/60 rate in comparison with their Save% and it was quite astonishing how they rose and fell in lock-step. Here is that chart:
Very rarely in hockey analytics do you get a chart that looks as “nice” as that one so it is something that really draws my attention. Essentially what this is saying is that the more shot attempts you give up the higher the goalies save percentage will be. If this is true it would imply that more shots does not automatically mean more goals. At least not more goals at the same rate. It would apply that in many cases more shots just means more shots that aren’t difficult for the goalie to save.
I have some theories on this. For one, we know that shots on the rush are more difficult to save. If you are generating a ton of shot attempts it probably means you are spending a lot of time in the offensive zone and if you are in the offensive zone generating shots, they are not the tougher rush shot variety. Thus, if you are generating a lot of shots it probably means they are of lower quality on average.
This is difficult to accept for a lot of people and there have been studies that have shown otherwise. For example, this one at brodeurisafraud.blogspot.com or this one at hockey-graphs.com. This morning twitter user @DTMAboutHeart posted his own chart showing the relationship did not exist. The problem with these studies is they aren’t necessarily looking at the same goalie in different situations. For example, if you plot CA60 vs Save% for all goalies you get some good and bad goalies on both good and bad CA60 teams. Of course the chart will be largely random in that situation.
Chris Boyle of SportsNet did a study that showed that the relationship does exist and higher shot totals leads to higher save percentages but that analysis is also flawed due to selection bias which led to some to rightfully doubt the conclusions. Although I still think there is merit to what Chris Boyle did there is also merit to the claims made by those who doubt his methodology. As such a different analysis really needs to be undertaken which is what I have done here.
In my opinion, the proper way to answer the question of whether shot volume leads to higher save percentages is to look at how individual goalies save percentages have varied from year to year in relation to how their CA60 has varied from year to year. To do this I looked at the past 7 seasons of data and selected all goalie seasons where the goalie played at least 1500 minutes of 5v5 ice time. I then selected all goalies who have had at least 5 such seasons. There were 23 such goalies. I then took their 5-7 years worth of CA60 and save % stats and calculated a correlation between them. Here is what I found.
|Player_Name||Nyrs||CA60 vs Sv% Correlation||StdDev(CA60)||StdDev(Sh%)||One Team|
|Average (CA60 StdDev>2)||0.264|
|Average (CA60 StdDev>3)||0.292|
|Average (One Team)||0.237|
|Average (One Team, CA60 StdDev>2)||0.311|
|Average (One Team, CA60 StdDev>3)||0.474|
The columns are:
- NYrs – Number of seasons goalie played >1500 minutes at 5v5 play
- CA60 vs Sv% Correlation – Correlation between CA60 and Save Percentage
- StdDev(CA60) – The Standard Deviation in CA60
- StdDev(Sh%) – The Standard Deviation in Sh%
- One Team – Flag indicating whether goalie played with a single team (Mostly is single team except for a trade deadline trade in a single season)
So, you can see that there are both positive and negative correlations which puts the claim in some doubt. That said, the overall average correlation is 0.183 so there is some evidence that on average there is a positive correlation.
Now, if CA60 doesn’t vary much in the sample it is difficult to identify a relationship with save %. You just can’t correlate something to a variable if that variable is relatively stable. So, if I restrict the goalies to only those whose standard deviation in CA60 is >2.00 the average correlation between CA60 and save percentage rises to 0.264. If I restrict it further to >3.00 the average correlation between CA60 and save percentage rises to 29.2.
The players playing in front of the goalie and possibly the system the team plays behind may also impact save percentage. If we attempt to minimize this impact by looking at goalies that have only played for one team (or mostly one team) the average correlation between CA60 and save percentage is 0.237. If we restrict that further by looking at goalies with StdDev(CA60)>2 the correlation is 0.311. Restricting it further to goalies with StdDev(CA60)>3 the correlation rises to 0.474.
So, what have we learned?
- There appears to be a correlation between CA60 and save percentage.
- The correlation gets sronger if we restrict to goalies that haven’t changed teams (i.e. relative stability in who is playing in front of them and possibly the system being played).
- If we restrict to only goalies that have had reasonably large variations in CA60 over the years the correlation also gets stronger.
Based on these observations I believe it is reasonable to suggest that there is in fact a positive relationship between CA60 and save percentage though it can be dominated by the impacts of changing teams or significantly changing rosters or playing styles in front of the goalie.Needless to say, this should change how we evaluate goalies as well as evaluate the defensive performance of players.