Evaluating defence and goaltending revisited
A couple weeks ago I posted an article about the Leafs defence and how it isnâ€™t as bad as many people think. Well, since then I have been working on trying to improve on the methodology by including shot type (slap shot, wrist shot, snap shot, tip in, backhand, wraparound) and in that process I found a few mistakes/issues in what I did previously.
First, I found a bug in my program that caused a number of powerplay goals to be considered even strength goals. When I fixed this the general conclusions of that article remained in tact though the amount of the goals caused by the goalie was reduced for most teams. In general the closer that a teams penalty kill ability was to their even strength ability the more valid the results but where a teams penalty kill ability was seemingly far superior or inferior to their even strength ability the results were skewed. The most notable team was the Philadelphia Flyers who were down right horrible at even strength but for some reason managed to have a pretty good penalty kill. When I fixed the bug the Flyers looked far worse than they did in the previous article.
The second issue I discovered is that not all shot distances and types are created equal and that there is significant (unintentional) bias in how game monitors decide what is a wrist shot vs snap shot as well as the distance a shot is from the goal. It is a bit surprising but it is clear to me now that some game monitors have a real hard time distinguishing between a 10â€™ shot and a 15â€™ shot. Since teams play half their games at the same arena (their home arena) if that arenaâ€™s game monitor couldnâ€™t judge shot type or shot distance very well their shot difficulty ratings would get significantly biased one way or the other.
In an ideal world I would there would be an easy method for factoring out that bias and using all the data but I cannot think of any such easy method. The quick and dirty solution is to just look at shots against while playing on the road. This will eliminate any significant home arena bias and hopefully and biases found in other arenas will get averaged out on their own. For the most part this is likely true but I am still not completely happy with this solution because I think on some level teams play a bit different at home than on the road. More on this later but for now just looking at road shots is the best solution so lets go with that.
Ok, so what I did was group shots into 19 categories based on shot type and distance.
- 0-14′ wrist shot
- 15-29′ wrist shot
- 30-44′ wrist shot
- 45+’ wrist shot
- 0-14′ snap shot
- 15-29′ snap shot
- 30-44′ snap shot
- 45+’ snap shot
- 0-14′ slap shot
- 15-29′ slap shot
- 30-44′ slap shot
- 45+’ slap shot
- 0-9′ tip-in
- 10-25′ tip-in
- 26+’ tip-in
- 0-12′ backhand
- 13-24′ backhand
- 25+’ backhand
I then performed an analysis more or less equivalent to the analysis done in the previous article. By doing that I come up with the following results:
Goalie Impact is the amount of goals per 60 minutes of even strength ice time the goalie is responsible for above or below what an average goalie would allow.
There are a lot of similarities between the table above and the table in my previous article but there are some teams that have moved up or down the list.
Toronto: Torontoâ€™s defence (ExpGA/60m) drops a bit in the ratings influenced partially by factoring in the amount of time the Leafs spend at even strength but more significantly because of removing a small home ice bias that made shots at Air Canada be reported as being at a slightly greater distance than they likely actually were (it should be noted the Air Canada Center bias was much lower than some other arenas). The result is that the Leafs dropped to 11th best road defence from 8th. Not a huge drop but I figured since the article was primarily about the Leafs defence I should mention it. Also, because of these changes it makes Andrew Raycroft look much better than under the previous analysis. Using this current approach the net effect of Leaf goaltending is pretty neutral (i.e. On the Leaf goaltending was about average). And this gets to my problem with just looking at road statistics. The Leafs were generally a worse team at home than on the road despite the fact that it is typical for a team to play better (by about 10%) on home ice as road ice. Raycroft may have been the culprit as he had an .898 save percentage on the road and a .890 save percentage at home but it could also be the Leafs as a team played differently and gave up tougher shots at home. Without reliable statistics we will never know which is true or whether it is some combination of the two.
St. Louis: Wow! How did they become the team with the goalies that saved the most goals? That is hard to believe considering the names of the goalies they have on their roster but it seems to be a function of the number of shots and their difficulty. St. Louis goalies also posted a much better road save % than a home save %.
The next thing I looked at is how individual goalies performed. Based on league wide save percentages for the 19 groupings I calculated how many goals a perfectly average goalie should give up given the shot types that each goalie faced. Here are the results sorted by the number of goals the goalie saved per game.
|SANFORD, CURTIS||St. Louis||30.80||23.00||7.80||0.42|
|BACASHIHUA, JASON||St. Louis||22.43||19.00||3.43||0.33|
|LEGACE, MANNY||St. Louis||35.99||27.00||8.99||0.29|
|BRODEUR, MARTIN||New Jersey||81.80||65.00||16.80||0.28|
|DIPIETRO, RICK||NY Islanders||63.73||52.00||11.73||0.27|
|LUNDQVIST, HENRIK||NY Rangers||59.50||49.00||10.50||0.21|
|BURKE, SEAN||Los Angeles||33.80||31.00||2.80||0.17|
|GARON, MATHIEU||Los Angeles||22.90||22.00||0.90||0.04|
|TOSKALA, VESA||San Jose||30.08||29.00||1.08||0.04|
|NABOKOV, EVGENI||San Jose||39.80||39.00||0.80||0.02|
|HOLMQVIST, JOHAN||Tampa Bay||45.68||48.00||-2.32||-0.07|
|DENIS, MARC||Tampa Bay||37.94||43.00||-5.06||-0.17|
|CLOUTIER, DAN||Los Angeles||17.88||28.00||-10.12||-0.68|
|DUNHAM, MIKE||NY Islanders||18.94||28.00||-9.06||-0.75|
For the most part the table makes perfect sense. It is still surprising to see the St. Louis goalies near the top of the list (I am beginning to think this is an anomaly of some sort) but it is no surprise to see Luongo, Brodeur, Lehtonen, DiPietro, Turco, Miller, Vokoun, Lundqvist, etc. near the top of the list and Esche, Dunham, Cloutier, Toivonen, etc. at the bottom of the list. So for the most part the list passes the smell test as everything seems right.
Here are some more observations:
1. Leaf goalie Andrew Raycroft does OK here as well as he would be somewhere in the middle of the NHL regular starting goalies.
2. It is interesting that Martin Gerber ranks higher than Ray Emery.
3. While the Panthers have seemingly upgraded from Belfour to Vokoun, the same cannot be said for the Leafs as Toskala is ranked well below Raycroft. I should add that while Toskala has a pretty good save percentage it could be attributed to the quality of his opponent as he has started against the Kings 5 times, and Coyotes 4 times, St. Louis, Columbus and the weak offensive Dallas Stars 3 times. That is a pretty easy schedule. Interestingly, like Raycroft, Toskala also seemed to perform much better on the road.
4. Teams might want to consider avoiding trading for Manny Fernandez and his large contract as he ranks quite poorly.
Whatâ€™s left to do?
There are still a couple of things I would like to tackle in this area of analysis. The first would be to see if I can come up with some kind of reliable method for making use of the home stats. The second thing is that while I think the above analysis does a pretty good job of accounting for shot difficulty I think the quality of the shooter is still factor that is not factored in fully and Iâ€™d like to see if I can find some kind of method for factoring that in. Problem is, I am not sure if there is enough data to properly evaluate individual shooters but I might give it a try.