For those that follow hockey analytics you are probably fully aware of Travis Yost’s recent comments on the enhanced stats pages on NHL.com. Today Greg Wyshynski chimed into the debate with a summary of the situation along with more comments from Yost as well as from Chris Foster of the NHL.
The comments from Foster has generated a fair bit of buzz from the hockey analytics community and in particular his comments about “close” stats which Yost railed on earlier.
“Years ago, smart people recognized that simply throwing out data for the sake of correcting for score effects was inefficient. We started using score adjusted stats at the team-level as far back as 2012. It was reaffirmed as a superior approach in terms of repeatability and predictability in 2014. Anyone who has spent 10 minutes on the internet looking up hockey stats is by and large familiar with this work. I don’t know anyone who has cited FenwickClose% or CorsiClose% in years for these very reasons,” said Yost.
Foster argues the jury is still out on “close” stats, which is why the NHL uses them. “It’s one variation. It’s one context. You can use it or you can choose to ignore it. It’s fair to say that some sites are phasing it out, but we’re putting it out there. I think it’s up for debate. I don’t think there’s been anything that definitive,” he said.
I actually think this is a false argument on behalf of the hockey analytics community. Who cares really if the NHL has “close” stats on their site. This doesn’t bother me at all and I in fact still have close stats on my site. There are a few reasons:
- Close stats may have been debunked at the team level as a predictor of future team success but I have yet to see an article that looks at it at the player level. Not everything at the team level can automatically be transferred to the player level.
- I don’t think the NHL’s target audience should necessarily be the hardcore hockey analytics enthusiast. I think the NHL’s target audience is far broader than war-on-ice.com or puckalytics.com would be. As such, I really don’t feel the NHL is obligated to adhere to everything the hockey analytics community believes in.
- Just because Adjusted Fenwick or Adjusted Corsi is deemed better in predicting future success it does not mean the components of it are not of any use. Looking at how teams and players perform when leading or when trailing or in tied or close situations can help identify where problems exist. It is analogous to CF% being useful for predicting team success but the component pieces, CF60 and CA60, an also tell you something important.
- I still have “close” stats on my sites so I can’t complain about the NHL doing the same.
- If you don’t like “close” stats, get over it. Don’t use them. You probably don’t like +/- either but you don’t write articles about how terrible it is that the NHL still has them posted on NHL.com (not to mention TSN.ca, the organization Yost writes for). It’s not a big deal. Get over it.
To me the “close” stats argument is just a silly argument.
The real problem I have with the NHL’s enhanced statistics is that they seemingly have zero quality control. It started from day one when the NHL posted /60 stats along side /20 stats as if they were two different stats. One is just 3 times the other. It really doesn’t take a lot of thought or foresight or planning or quality control to figure out that both aren’t required. Then there have been problems with accuracy such as the short handed save percentage of 1.000 for a large number of goalies when they really meant save percentage when on the power play. This is the kind of thing that can be easily caught by a basic quality control process. Over the summer I noticed mistakes in zone start stats as well. Really basic stuff that should be caught before ever being published.
“The fan feedback has been vital to making improvements to the site. Our goal is a spirit of collaboration,” he [Chris Foster] said.
Accepting feedback from the community and utilizing it to make your product better is a great thing to do. I applaud that. The problem is it seems the NHL is using the community as their quality control process which really just makes the NHL.com enhanced stats pages one massive beta release of a still in-development project. Every now and again I have mistakes happen with my site but there is a huge difference between a relatively low traffic stats site maintained by a single person in his spare time compared to NHL.com with hundreds of thousands (millions?) of visitors daily operated by an organization (NHL) with $4 billion in revenue backed by a company (SAP) with a market cap approaching $100 billion. The level of expectation for quality is dramatically higher than what has been delivered by the NHL.
Taking a bit of a step back, when the NHL first announced that they were venturing into enhanced stats I thought it was a good idea. I thought the NHL would come up with some great stuff to help bring some of the great stuff the hockey analytics community is doing more main stream. I thought it would do for hockey stats what James Mirtle (and others) have done for hockey writing. Mirtle has been great at bringing hockey analytics more main stream without over whelming his readers. I figured the NHL would come up with a really simple, easy to understand set of statistics that are very well presented that would serve as a great basis for the more casual fan. That hasn’t really been the case and it doesn’t appear to have been their goal.
To me it seems the NHL looked at the existing stats sites and chose instead to try and clone them right from day one but without the hockey analytics expertise and understanding in house to actually deliver. They seem to have tried to do too much too quickly with too little experience and not understanding that their target audience is far more casual than war-on-ice.com or stats.hockeyanalysis.com. The result is a stats site that is incomplete and insufficient for the hardcore hockey analytics follower, yet too complex and not well presented for the casual fan, not to mention, of course, the mistakes in the data. All-in-all It just hasn’t worked out well which is utterly unfortunate.