I am taking a bit of a hiatus away from my hiatus on blogging because I read something a few days ago that caught my attention both as a Leaf fan and as a hockey analytics enthusiast and felt compelled to respond.
The Leafs have been a pretty interesting team this year with a significant infusion of youthful talent in Matthews, Marner and Nylander. They may even be good enough to capture a playoff position or at least seriously contend for one. That said, they are still a flawed team. Offensively they are a dramatically improved team and are near the top of the league in 5v5 GF60. Thats the good news. The bad news is they are near the bottom of the league in 5v5 GA60. Keeping pucks out of their own net is their real problem.
A few days ago Dom Luszczyszyn wrote about the Leafs and how the fourth line and third pair defensemen are a problem for the Leafs. Specifically he pointed the finger at Matt Martin, Ben Smith, Roman Polak and Matt Hunwick as the problem. I don’t know exactly how Dom came up with those names but it is interesting to note that as I write this those four players have the worst 5v5 CF% on the Leafs among players with 100+ 5v5 minutes.
Dom suggests that if the Leafs stopped playing Polak, Hunwick and Smith and started playing Marincin, Corrado and Holland they would be a better team (he didn’t believe there was any chance Martin would get benched so didn’t disscuss that option).
With the Leafs current lineup – the one with Smith, Hunwick, Polak – their expected true talent (based on my model which I’ve talked about previously) win percentage is .484, an 89 point team. Sub in Holland, Marincin, and Corrado and the Leafs are a true talent .505 team, or a 93 point team.
So there you have it. Swap out those three players and you will get yourself four additional points over the course of the season based on his model. Although not everyone has placed a point total on it Dom is not alone in the negative views of Polak, Hunwick, Martin and Smith. It isn’t difficult to find detractors of these players.
Here is the problem. Dom has put significant (absolute?) trust in his model and as a result he is certain that making this switch will make the Leafs better. Dom has used his theoretical model to identify a theoretical problem (sub-optimal lineups) with the Leafs and proposed a theoretical solution. In theory it should all work out.
The reality is the four guys Dom identified as being problem players for the Leafs have the best GA60 ratings on the team.
Dom’s model-based proposal to make the Leafs better will in fact will remove the four players that have the best goals against rates on a team that is desperately in need of improving their goals against rates. That seems like the wrong thing to do to me.
Let’s have a look at how the Leafs defenemen have performed since Mike Babcock took over as coach (Polak’s numbers do not include San Jose time).
Is Frank Corrado really going to make the Leafs better? Is Marincin really that much of an upgrade on Hunwick? Will either of these guys provide enough offense to more than offset what appears to be a penalty defensively?
I know goals aren’t the cool thing in hockey analytics these days and I am aware that there are randomness issues involved with them. So yes things may change in the future. It could very well be that Polak, Hunwick, Martin and Smith have just been really lucky in terms of goals against and Corrado and Marincin have been unlucky. It could be just that, or maybe it isn’t.
Maybe Dom’s model is the best player evaluation model in existence. I won’t debate that one way or the other here because it doesn’t matter. What matters is that I am 100% certain that it is far from perfect. It will evaluate some players correctly, maybe even evaluate most players pretty well. At the same time I am certain it will fail for others and in some cases may fail significantly. Even if on the whole Dom’s model is quite good we have no way of telling if it is correct or wrong about Polak, Hunwick, Martin and Smith specifically. We just can’t know for sure.
What I do know is that Polak, Hunwick, Martin and Smith have have not been the problem defensively this year in terms of goals against (which ultimately is what matters). It just doesn’t make sense to get all up in arms about the ice time Polak, Hunwick, Martin and Smith are getting when they clearly can’t be blamed for the Leafs high goals against rates.
If you want to make the Leafs better it would make much more sense to put the focus on a real problem such as trying to improve the defensive performance of Marner, Bozak and van Riemsdyk which has been downright dreadful. That line has been great in the offensive zone but they are giving up 4+ goals per 60 minutes of 5v5 play and that is outrageously bad. You want a real problem to solve, that is a good place to start. If Martin, Smith, Polak and Hunwick become a problem down the road by all means fix it then. For now the real results are there so at minimum ride the wave until it crashes. No sense solving a theoretical problem that in reality may or may not exist.
This post is not to diss Dom or his model or analytics in general. Dom has done some good work and his model probably does a decent job of evaluating players and I am a firm believer that hockey analytics implemented properly can be a huge benefit to an NHL team. It is just important to focus on solving real problems, not theoretical ones that get identified by an imperfect model no matter how good that model may be.