Is 4v4 overtime hockey a crap shoot we can or should ignore?

Since the Los Angeles Kings have been eliminated from the playoffs there has been a lot of discussion about why a team with such a good possession game failed to make the playoffs. This included my article from yesterday which generated a fair amount of discussion as well. A lot of the discussion can be summarized by the following tweet by Sunil Agnihotri referencing a comment by Walter Foddis.

The last paragraph is the one that interests me most.

“The substantive reason for LA not making the playoffs is the OT system, which does not reflect team strength. Statistically, OT outcomes have been shown to be a crap shoot. LA was unlucky in OT”

The fact that LA went 1-7 during overtime play does in fact mean that they were unlucky during OT play. They are a better team than that for sure (every team is expected to do better than that). OT results over the course of a single season are extremely random and thus one could consider them a crap shoot. The challenge I have is just because something is highly variable does that mean it is meaningless in our evaluation? Being unlucky in over time does not mean you are unlucky overall.

I’d hazard a guess that outcomes of the first 5 minutes of the second period for games that are played on a Thursday are highly random too. If a team missed the playoffs and had a terrible goal differential during the first 5 minutes of the second period in games that are played on a Thursday can we chalk up missing the playoffs to bad luck during the first 5 minutes of the second period in Thursday games? No, of course not. We don’t get to pick and choose what good luck or what bad luck we can blame results on. Just because we are more aware of bad luck that happens in overtime games doesn’t mean it is more important bad luck worthy of attributing blame to.

The reality of the situation is that unless you can be certain that the Kings OT bad luck is not offset by good luck during the remainder of the game you can’t blame the Kings missing the playoffs on their OT record.  I haven’t seen the complete luck analysis of the Kings season done to claim the Kings were unlucky during regulation and OT play as a whole so I am pretty reluctant to blame the Kings playoff miss on their OT record just yet.

The interesting question for me is whether 4v4 play is indicative of overall talent because if 4v4 hockey requires a completely different skill set then one could conclude that overtime play isn’t representative of true hockey talent. To answer this question I took the correlation between each teams 5v5close GF% over the past 8 seasons (to get large sample sizes though it would reduce the spread in talent) and compared it to their 4v4close GF% over the past 8 seasons (I used close since most 4v4 ice time is in OT and thus in close situations). Here are the results.


And the same for CF%.


Those correlations are good enough for me to consider that 5v5 skills are fairly transferable to 4v4 play and vice versa. Over small samples strange things happen, but to suggest that 4v4 play isn’t indicative of hockey skill and that is why one should ignore OT results is not valid either.

An interesting observation is that the slope on the CF% chart is almost exactly 1.0. The slope on the GF% chart is significantly higher than 1.0 which might indicate that 4v4 play is actually a better indicator of talent than 5v5 play (if you are good at 5v5 play you should be even better at 4v4 play). That said, if I force the intercept to zero the slop drops to 0.9958 or almost exactly even (and r^2 drops to 0.3123 with zero intercept) so maybe 5v5 and 4v4 are on par with each other. Regardless, this should at least alleviate Steve Burtch’s concern that poorer teams are more likely to score first during 4v4 play than during 5v5 play. I don’t believe that to be the case.

Now when we talk about shoot out record I think that it is safe to assume that the shoot out is a lot further from being representative of actual hockey talent than 4v4 play. There is probably not enough shoot out data to actually be able to do a similar analysis with any degree of confidence but I doubt there is much disagreement that the shoot out is a long way from being representative of real hockey.