Rielly and Quality of Competition

The issue of the importance of quality of competition is a hotly debated topic in hockey analytics. On the one hand it is easy to believe that players that play shut down roles against the opposing teams top players are likely to have their statistics negatively impacted. On the other hand, hockey analytics hasn’t been able confirm that is true as most analysis suggests the impact of quality of competition is generally very small.

Recently Tyler Dellow wrote an article on Morgan Rielly defending him as a #1 defenseman. This is counter to what I have been arguing but what I was most interested was Dellow’s comments on quality of competition. Dellow has come up with what he calls ‘Star Percentage’ which he defines as ‘the percentage of time you spend matched up against the other team’s star forward.’ Star percentage suggest that Rielly plays some of the toughest minutes in the league which basically matches with one of my favourite QoC statistics: OppGF60, or the average GF/60 of the opponents he faces. OppGF60 suggests that Rielly has the 9th toughest QoC among defensemen with at least 400 minutes of ice time. In fact most of the players at the top of the OppGF60 list also appear at the top of Dellow’s Star Percentage list.

Dellow went on to say that Rielly’s tough QoC should be considered in any analysis of Rielly.

Over the last two years, Rielly has had the highest star percentage of any defenceman in the NHL.  When evaluating him, that has to be kept in mind.

The problem is there was zero effort to actually quantify what that impact is and yet doing so would be a relatively simple thing to do. So, I will do that here.

I don’t have Dellow’s list of ‘Star’ players so I created my own. I chose the following players, one from each team other than the Leafs.

Team Player
ANA Getzlaf
ARI Domi
BOS Marchand
BUF O’Reilly
CAR J. Staal
CBJ Saad
CGY Monahan
CHI Kane
COL MacKinnon
DAL Seguin
DET Zetterberg
EDM McDavid
FLA Jagr
LA Kopitar
MIN Koivu
MTL Radulov
NJ Hall
NSH Johansen
NYI Tavares
NYR Stepan
OTT Hoffman
PHI Giroux
PIT Crosby
SJ Thornton
STL Tarasenko
TB Kucherov
VAN H. Sedin
WPG Scheifele
WSH Backstrom

Now I know many of you will have your own opinions of who is each teams ‘star’ player but ultimately we are only really interested in a player that represents the top line. So whether you choose Backstrom or Ovechkin it really doesn’t matter.

So now that I have my list, the obvious thing for me to do is look at how Rielly performed against these guys and against everyone else. This should give us an indication of how significant facing star players will impact his results.

Overall vs Stars vs Rest
GF% 40.7 37.0 43.5
CF% 49.8 50.5 49.1
TOI 100% 47% 53%

According to Dellow’s Star Percentage Rielly has played just shy of 50% against star players. Against my list it is 47.4% so there seems to be fairly good agreement.

Rielly has a 37.8 GF% against the star players and 42.9% against everyone else. Just as we all thought, Rielly’s does worse against star players but he is still pretty terrible against everyone else.

What is interesting is that his Corsi for percentage is actually better against the star players than against everyone else. This is a little unexpected and I am not quite sure how to explain it but the different isn’t that big so maybe just on of those statistical oddities.

So who you play does seem affect your statistics as we thought. The question is how much does this really translate when comparing a player that plays tough minutes to a player that plays easy minutes. How much of should we really factor in QoC to justify a players good or poor stats?

To answer this I looked at all forwards who played at least 200 minutes of 5v5 ice time this season and ranked them based on the percentage of their teams 5v5 ice time that they played. The top of the list has Patrick Kane, Connor McDavid, Mark Scheifele and Ryan Getzlaf and the bottom of the list has Jared Boll, Chris Thorburn, Jordin Tootoo and Cody Mcleod.

There are 422 players in total and I called the top 90 of them first liners, the next 90 second liners, the next 90 third liners and everyone else 4th liners. Of the 29 players listed above, 24 were identified as first liners. Jordan Staal (91), Stepan(92), Hoffman(99), Monahan (105) and Domi (108) missed the cut and were classified as second liners. Arizona actually had no one qualify as first liners using this technique.

Here is the average Rel statistics for each group.

GF60 Rel GA60 Rel GF% Rel CF60 Rel CA60 Rel CF% Rel Sh% Rel Sv% Rel
1st liners 0.53 0.12 4.38 4.14 -1.01 2.32 1.19 -0.41
2nd liners 0.24 0.14 0.93 3.03 0.52 1.10 0.36 -0.29
3rd liners -0.18 -0.05 -1.83 0.33 -0.76 0.47 -0.67 0.03
4th liners -0.52 -0.24 -3.89 -5.96 0.56 -3.09 -1.02 0.71

Nothing too surprising though there is an odd relationship with CA60 Rel where first and third liners suppress shots and second and third liners give up more. That said the spread is fairly small and there isn’t much of a difference across all four groups of players.

I will also take this time to point out how Sv% Rel varies down the list. First liners reduce save percentage and 4th liners boost save percentage. Interestingly, I recently wrote an article pointing out that offensive players negatively impact save percentage and these numbers support that claim. Also note that the spread in Sv% Rel is about half that of Sh% Rel which interestingly is what I estimated in a recent article I wrote on Predictive analytics failing hockey analytics.

For a variety of reasons I believe a players ability to impact save percentage is about half of their ability to impact shooting percentage.

This is another reason I can add to the list.

The real question is how facing more first liners and fewer 4th liners affect ones quality of competition. To do this I calculated the average opponent statistic depending on whether a player played tough minutes (more first liners) or weak minutes (more 4th liners). I also looked at a more neutral weighting. The following table shows the percentage of ice time against each group of players.

Tough Neutral Easy
1st liners 50 35 10
2nd liners 25 30 15
3rd liners 20 25 30
4th liners 5 10 45

So a player with tough minutes would play 50% of the time against first liners, 25% of the time against second liners, 20% of the time against third liners and 5% of the time against 4th liners. The numbers are somewhat arbitrary but should represent the extremes fairly well. Next I calculated a weighted average using the percentages in the above table to come up with expected average opponent statistics.

Tough Neutral Easy Tough-Easy
GF60 Rel 0.26 0.16 -0.20 0.46
GA60 Rel 0.07 0.05 -0.09 0.16
GF% Rel 1.86 0.97 -1.72 3.59
CF60 Rel 2.59 1.84 -1.71 4.31
CA60 Rel -0.50 -0.33 0.00 -0.50
CF% Rel 1.37 0.95 -0.86 2.23
Sh% Rel 0.50 0.26 -0.49 0.99
Sv% Rel -0.23 -0.15 0.25 -0.48


The Tough-Easy column is simply the difference between the tough opponents and the easy opponents to give you an idea of the spread in talent.

Let’s start by looking at the GF% Rel line and put it in simple terms. A tough minutes player would have an average opponents GF% of 51.86 and an easy minutes player would have an average opponents GF% of 48.27 for a spread of 3.59.

Now let’s compare this to the actual OppGF% of Toronto Maple Leafs defensemen.

Player Name OppGF%

Morgan Rielly gets the tough minutes and has an OppGF% of 51.2. Marincin gets the easy minutes with an OppGF% of 49.2. This isn’t as large a spread (2.00) as the spread in my calculated estimate for tough/easy minute players (3.59) but not completely out to lunch either.

Looking at CF%Rel we have a difference of 2.23 from tough minute players to weak minute players. Again, looking at Leaf defensemen we see this is a reasonable estimate.

Player Name OppCF%

The spread for Leafs defensemen is 1.4 vs 2.23 for my calculated tough/easy minute players.  This is a similarly sized over estimate as we saw with GF%. It probably means my ice time estimates are too extreme. I also include defensemen in my Opp calculations which probably suppresses the spread somewhat as well. Either way I think my ‘Opp’ statistics are a reasonable estimate for QoC.

So, getting back to Rielly, we know he plays the tough minutes which means the average GF% of his opponents is probably no more than 3.86 higher than those who have the weakest opponents. This suggest that if Rielly had easy minutes at best his GF% would rise 3.86 points. This would take it from 40.7 to 44.3. This is still really bad compared to the rest of the Leafs defensemen. For example, if we take Polak and assume he had easy minutes and gave him tough minutes his 56.2 GF% would drop to 52.6 which is still light years ahead of Rielly.

In no way does quality of opponent change the debate in any significant way. Yeah, extreme players will see a shift in their statistics but if you have terrible statistics to start with they at best will be really bad once you make any QoC adjustment.

Now, before I finish let me address something else that I am sure will come up. That is, I am sure many of you are thinking, yeah, but good players play the tough minutes against good players so there numbers are suppressed because of it. If they had a more neutral opponent their numbers would be higher making the spreads higher making any required QoC adjustment larger. There may be a small sliver of truth in this however remember that Rielly only had a GF% spread of 5.9 between the list of star players and everyone else. That’s basically a 5.9 spread between when he only played against first liners to when he only played against non-first liners. Even if we adjusted his GF% by 5.9 points he doesn’t look good and I think it is a near impossible argument to make that any QoC adjustment should be anywhere close to this much.

In the end QoC matters, just not as much as you may think. For most players it is neglible, for extreme players it matters only a little.

Oh, and Rielly is really struggling this year and the Leafs desperately need him to be better.



This article has 1 Comment

  1. That is a rather interesting analysis. The data from players broken down into buckets based on TOI% is pretty much what I would have expected them to be…

    1st and 3rd liners tend to be more 2-way oriented, with first line being more offensively focused and third line being more defensively focused. In the olden days when standard practice was for your 3rd line to be a defensive shutdown unit it may have been an even more distinct difference, but these days a lot of teams expect their stars to play against top opponents too (and in turn many have switched to dressing a third scoring line rather than the traditional shutdown unit). Of course that would likely effect the TOI% buckets because shutdown units would have to get significant TOI in order to be on the ice against top units (what comes to mind for me is when the Penguins had Staal as a 3C but he had the 3rd highest TOI amongst forwards on the team), which may also mean that any teams that still set their roster that way will have those players in a higher bucket. 2nd liners are pretty much offensive specialists, and 4th liners are not very good at anything (but better defensively).

    I am curious if there would be much difference in the data if we were to look at just Home data, since that is when the coach has last change and can put their players out against the opponents they feel they are best suited against (conversely Away games the opponent has the option of putting your players at a disadvantage). The issue there of course is that would cut our sample size in half, and while we could look at multiple seasons you then get into the issue of players moving to different teams, different coaches may utilize them differently especially as players age (like a rookie skating sheltered minutes may take a bigger role in later years). Which would of course not only effect the player themselves but also the opponents they face may be in different roles from one year to the next.

    However, my general takeaway from QoC is it works best when you look at small sample size players (rookie call-ups and 4th liners) who post incredibly good numbers. You can note that and the lesser QoC and come to the realization that they aren’t necessarily as talented as the top liners who put up similar numbers against top opponents.

Comments are closed.