Anecdotes, evidence, and open mindedness

Within a couple of minutes of posting my last article on the Corsi and shooting percentages of Carolina and the Maple Leafs there were a couple of back-handed attacks to the post from Tyler Dellow.

DellowTweets

First, clearly Dellow didn’t fully comprehend the article because I didn’t include any direct commentary on the goal scoring of the Hurricanes and in no way did I imply anything had a correlation of -0.98 with goal scoring at 5v5. Specifically, the correlation was between CF% and shooting percentage and both those statistics include a component that factors into goal scoring. In fact, CF% and shooting percentage could have a -0.98 correlation and goal scoring could go up, but that isn’t the point I was trying to make.

Second, I think this goes a little beyond anecdotes and qualifies as evidence. Let’s summarize what I have found:

  1. I looked at several coaching changes and found there to be a negative correlation between CF% and shooting percentage in the majority of them.
  2. I identified every case where we know analytics has most likely directly influenced on-ice tactics.
  3. I showed that in first after those changes all three instances showed a negative correlation between CF% and shooting percentages.
  4. In the two cases that we know have extended into this season I showed that the correlation continued.

I understand that Dellow and others has staked his reputation and his hockey analytics career on the fact that CF% drives winning and that shooting and save percentages are not nearly as important but it should be clear to anyone that, while not absolute proof, this at the very least rises to some level of evidence above anecdotes. I made some observations and came up with a hypothesis and that hypothesis is holding true as new observations are made.

This is, in a significant way, how analytics people (such as Dellow) sold the public on Corsi. They made observations about Corsi, then made predictions on a number of teams (Wild, Avalanche, Leafs, etc.) and how their won-loss records would regress and then over time those predictions held mostly true and people bought in.

Now I think Dellow has done some good stuff and he has earned what success he has had in hockey analytics but it isn’t like he is infallible. I remember a number of times I debated with Dellow the value of zone starts and the duration of influence of zone starts. At the recent Rochester Institute of Technology Hockey Analytics Conference Micah Blake McCurdy presented what I consider additional strong evidence that zone starts just don’t have a significant impact on a players statistics. Furthermore McCurdy agreed with my work that the impact of a zone start is largely diminished after the first 10 seconds.

At the same Rochester Hockey Analytics Conference Matt Pfeffer, now consulting with the Canadiens, mentioned that some of the stuff he believed a few years ago he was wrong about. “Always remember that you’re probably wrong.” is the piece of advice he gave and it is wise advice. It is great to be confident in what you believe in and it is perfectly acceptable to be a skeptic of new ideas (I certainly am both of those) but in science there is a line you need to not cross before it inhibits your quest for truth and knowledge. The kind of quick, off the cuff comebacks that Dellow tweeted immediately after I posted my last post I believe is an example of crossing that line. To me those tweets were an unfair representation of my post solely in an attempt to discredit it because it doesn’t line up with what he has based his reputation and career on. It is unscientific and closed minded.

What I hate most about the hockey analytics community is when we start talking about things with a great deal of certainty and quickly and abruptly dismiss anything anyone writes that goes counter to what we believe in. It is happening less and less now and there is more diversity in the conversation as the hockey analytics community has grown over the past couple seasons but it still occurs. A few years I would get mocked for even suggesting shot quality exists but there are now a number of projects and research taking place on shot quality. That’s progress but it could have and should have happened sooner if people were more open minded several years ago. Hockey Analytics (and any scientific field for that matter) only progresses with diversity of opinions and coming up new hypotheses to test and investigate. Open mindedness to new or different ideas can go a long way and will serve the community well.

You won’t be right about everything. Get over it.

—————————–

Update: Dellow doubling down:

DellowTweet2

If it was all retrospective, yes, but the trend was predicted to continue and it did which takes it at least up a level from ‘garbage’.

Second, I did have all the adequate qualifiers in there including “It’s only three data points” and “We are dealing with very small sample sizes and only a couple of teams” so you are perfectly allowed to be a skeptic but if you toss all evidence (especially when it was predicted) that runs counter to your prior belief in the trash it might just be you that is exhibiting a bias.

—————————

Update #2: Tripling Down:

So Dellow went on another twitter rant starting with this:

DellowTweet3

Of course it is convenient for him to comment on something that only he can see so I challenged him to repost that stuff so we can all see the discussion. He instead posted a few excerpts including this:

DellowTweet4

I love where he says

“You’ve been adamant that faceoffs aren’t a big deal – this shows pretty clearly that they are and that the impacts lasts for a while.”

 

That clearly runs counter to what Micah Blake McCurdy has found and what I have found as I explained above. A lot of people, and maybe the majority of hockey analytics, now believe that that is the wrong conclusion. Despite Dellow thinking his work “pretty clearly” showed zone starts were a big deal Dellow was almost certainly pretty wrong here.

 

 

This article has 1 Comment

  1. Great post David. I give you credit for standing up to the thick-headed TD. A true scientist knows that everything is always open to being proven wrong. I remember how you stood on an island about shot quality I always intuitively agreed with you very strongly but didn’t have the expertise to proof it. It’s great to see recent work where shot quality improves predictions over score adjusted corsi!

    I also changed my views on ZS because of yr work,, And logic tells us faceoffs are not that important. Why? This is because a faceoff is just 1 battle for possession. And, it’s one that every player is in position as opposed to more dangerous possession battles on the fly. Keep up the thoughtful independent work!

Comments are closed.