I have always believed that goaltending is by far the most important position in hockey and have claimed it can make or break any teams season. I have claimed that the main reason that the Leafs have failed to make the playoffs post lockout is because of bad goaltending. Many others have scoffed at this claim blaming everything from bad defense to bad offense (which is mostly not factually true) to poor coaching, to a combination of all of the above. I have seen others claim that goaltending would account for at most four or five games a year. So, I have undertaken a bit of a study to attempt to figure out how important goaltending really is and how many points in the standings poor goaltending can cost you or great goaltending can gain you.
Most goaltending studies I have seen, and done myself, have to do with comparing goalies from one team to the next. The problem with this is people can easily choose to dismiss the study with claims like ‘but team x has such a bad defense you cannot blame the goalie for that’ and to some extent there is some validity in this claim (though I do not believe it to be as much as many do). There have been other studies that attempt to factor out the defense issue by coming up with some sort of shot difficulty rating based on shot type and defense. I believe that this has some merit and improves the validity of the study but people will simply jump in and claim that not all shots from the same distance are equal and teams with bad defense will inherently give up more difficult shots so the shot quality analysis is still far from perfect. Again, there is some merit to this.
So, with all that in mind I set out to study goaltending in a way that eliminates the quality of a team’s defense in a way that most sane people cannot dispute: compare goalies who play on the same team. If we are comparing two goalies who play on the same team we immediately eliminate the ‘but he plays on a bad team’ argument because they are playing behind the same players.
I collected all the goalie statistics from the 5 regular seasons since the NHL lockout of the 2004-05 season. For each team and season I identified each team’s starting goalie (the goalie with the most starts) and then grouped all other goalies who played for that team in that season and merged their statistics into a combined backup goalie statistic. For example, this past season Jonas Hiller was the starter for the Anaheim Ducks and JS Giguere and Curtis McElhinney also played for the Ducks so Giguere and McElhinney’s stats were combined into a single team backup stat. The statistics I am interested in are save percentage and points earned for their team and the number of games started from which we can calculate points earned per start stat for the starter and the combined backup. I next subtracted the backups points per start from the starters points per start and the backups save percentage from the starters save percentage. Here is an example for the Anaheim goalies.
So, from that table we see that Jonas Hiller had a save percentage 0.0618 higher than his backups and produced more points for his team in the standings at a rate of 0.0128 per start. Now since that last stat is pretty meaningless I prorated it to 82 games and over the course of 82 games Hiller would produce 5.07 additional points in the standings over his backups.
I then did the same analysis for all 30 teams for each of the 5 seasons to come up with a 150 point dataset of save percentage and points per start prorated to 82 start differentials (i.e. 2009 Anaheim data point is 0.0618, 5.07). Now, let me mention a few caveats. There were several instances where the backups had a better save percentage than the starter. In this instance instead of subtracting the backups numbers from the starters, I subtracted the starters numbers from the backups. This was done just to keep all the save percentage differential positive so we can study what effect improved goaltending has on a team’s points earned in the standings.
A scatter plot of all this data would look like the following.
There is a lot of variation in that scatter plot but as one might expect there appears to be an upward trend, though it is far from statistically significant. The chart tells us that the greater the difference between a starters save percentage and his backups save percentage, the greater the difference in the points earned, generally speaking. No surprises there.
On average the difference between the starters save percentage and the backups save percentage was 0.0142 and the point differential was on average 12.42. So, that means that the average starter would stop 1.42% more shots than their backups and, if they played all 82 games, would generate 12.42 points in the standings for their team more than if their backups played all 82 games. To simplify matters more if we assume a linear relationship we can conclude that a 0.01 difference in save percentage would result in an 8.76 point change in the standings. That isn’t an insignificant number, especially when you consider the difference between the save percentage of the Boston Bruins was 0.03 higher than that of the Leafs, or potentially the equivalent of 26 points in the standings. If you take 26 points away from the Bruins they are no longer a 91 point team but a 65 point team, just ahead of the Oilers. That isn’t really that farfetched as the Bruins were the worst offensive team in the league and if you gave them the worst goaltending as well, you would expect them to be really bad. Conversely, if you gave the Leafs 26 points they would be a 100 point team and challenging Buffalo for the division lead.
Now, we need to be careful when we theorize in this way because this study does not in any way predict how the Bruins goaltenders would play behind the Leafs set of forwards and defensemen nor is it an exact science (i.e. a 0.01 will not always mean an 8.76 point difference in the standings. The variation is fairly significant, but on average that is what it has been over the past 5 seasons). All we can say is that if the Leafs improved their team save %, either through better goaltending or better defensive play, up to Boston levels we can probably safely assume that the Leafs would make the playoffs, and might even be considered a top team in the eastern conference.
The Buffalo Sabres are an interesting team to look at because they made very few roster changes from last year to this year and their coaching staff remained the same. The most significant change was the loss of Jaroslav Spacek who was replaced by Tyler Myers (Spacek had 8 goals, 45 points last year, Myers had 11 goals, 48 points this year so there may not be much of a net change there either). In the standings though, the Sabres improved 9 points in the standings and I think we can attest much of that to improved goaltending, in part because Ryan Miller continued along his upward trend in his career and partly because he was healthy for the entire season allowing him to play 10 more games this year than last. The end result is the Sabres team save percentage improved by 0.011 points which according to the relationship between save percentage and points developed above would equate to a 9.6 point improvement in the standings, which is almost exactly what happened. Last year the Buffalo Sabres scored 242 goals and that dropped to 231 this year improved offense cannot explain the improvement in the standings.
The Colorado Avalanche improved 26 points in the standings in large part due to the addition of Craig Anderson as their starting goalie. In 2008-09 the Avalanche had a dismal .894 save percentage while this year, thanks in large part to Craig Anderson, the Avalanche had a .913 save percentage, an improvement of 0.019. Using 8.76 points per 0.1 change in save percentage we could conclude that 16.6 of their 26 point improvement came from improved save percentage which would mean only 9.4 points of improvement came from other aspects of their game. Now some of you will claim that they also happened to score 47 more goals in 2009-10 than in 2008-09 and while that is true, they also allowed 10.5% more shots against so the additional offense came at a cost but that cost was absorbed by their improved goaltending.
In future blog posts I’ll look at other teams in more depth let me wrap this up by making this statement: while the above findings will not put an end to the debate on how important a goaltender is to his team’s success I must say that my belief that a goalie is by far the most valuable member of a team has only been enhanced.