[Discussion] Analytics in Hockey

[Down By the River]

A thread for the discussion of how analytics are developed, implemented, and used to draw meaning about the value of players. I'll build on this intro post later and provide some resources, but I wanted to get started by going on a bit of a rant in my second post, as I have some thoughts about analytics in hockey but haven't really had a place to voice my thoughts.

[Down By the River]

There is a long history of analytics helping improve the success of professional sports teams. 

 

I'm going to start with my own opinion: Nobody should be placing any value on publicly-available models. 

 

Why? Is it because I'm old school and believe that the game is played on ice and not paper? No. Is it because I hate numbers? No. In fact, I absolutely love stats, and it is because I love stats that I want to debunk the statistical models used in the NHL's analytics community. 

 

 

The models you see from guys like Dom Luszczyszyn, "JFresh", "EvolvingWild", Ryan Bader, and Patrick Bacon are based on flawed data and flawed research methods. Specifically, these authors principally focus on measuring Wins above Replacement (WAR). WAR reflects how much better a player is than a player that would typically be available to replace that player. You regularly see WAR represented in various graphs/player cards. For example:

Here, WAR is reflected as a percentile showing that Calen Addison's WAR value is higher than 78% of his peers (i.e., he's good at hockey). 

 

The idea of WAR comes from baseball, which is a sport that is incredibly different from hockey. Most importantly, a baseball player’s ability to have success at the plate is almost entirely independent of any of their teammates’ performance. Bo Bichette’s ability to hit a 95mph fastball does not depend on George Springer’s ability to hit that pitch. Elias Pettersson’s ability to score a goal or even just generate a shot is dependent on the play of his teammates. Is he playing with teammates who are willing to shoot more? Is he playing with guys who are elite at moving the puck out the zone and preventing shots against? This point about independence is fundamental because the statistical analyses that the NHL analytics community uses make assumptions that cases (i.e., players) are independent of one another.  This means that no two observations in a dataset are related to each other or affect each other in any way. Quite clearly, this assumption is violated.

 

Why does the violation of independence matter? Standard errors of regression coefficients will be underestimated. Smaller standard errors make it easier to identify a significant relationship where no such relationship actually exists. In other words, a danger of the violation of independence is overestimating the predictive utility of your model.

 

Tests of this assumption are possible, such as through the inspection of residual autocorrelation and a Durbin Watson test. I have never seen this done by the NHL analytics community. Instead, the NHL analytics community paints the issue with non-independence as an issue of multicollinearity (i.e., where two variables are so highly correlated that it is difficult to separate the two). There are other methods that can be used, and the analytics community does use them, to try and address multicollinearity, though these methods are imperfect. However, the real issue is that the analytics community is ignoring the larger issue of non-independence and focusing on what is only a secondary issue (multicollinearity).

 

There are ways to analyze data that violate the independence of cases assumption, such as through the use of multilevel modeling. Multilevel models acknowledge, for example, that kids within a particular school are like more similar to one another than they are to kids from an entirely different school. Along this logic, players within one particular team, or who play on one particular line together, are likely more similar to each other than they are to players who play on a different team or a different line. Other analytic strategies, such as network analysis, are used specifically for the purpose of examining relationships among non-independent actors. It is often used as a direct response to violations of independence of cases. To my knowledge, neither multilevel modeling nor network analysis are used in publicly-available NHL analytics models.

 

Why are publicly availably models apparently purposefully/knowingly putting out data that are fundamentally flawed? In my view, they are all running a 10KM race but they all want to start this race 10metres from the finish line. There are some very basic principles of research, one of which is that you need reliable data before you can examine the validity of these data. Put simply, before we try to, for example, identify different stats that predict contribution to WAR (e.g., zone entries/exits, quality scoring chances, etc.) we need to make sure that we can reliably measure these stats. Those who have developed these publicly-available models should be responsible for first showing, for example, that their measure of zone entries is reliable.

 

Interrater reliability is the most common method of doing this. Will two (or more) persons judge a zone entry in the exact same way, and then can they repeatedly judge zone entries the same way across multiple events? This is really the basic building block/tenets of research that the Twitter-based analytics guys gloss over. How quality scoring chances are judged is probably the most concerning in terms of reliability. Many of these models place substantial weight on the contribution of scoring chances to WAR. However, these models don't have the power to go into every single game on every single night and look at each shot from multiple angles to judge if it is "quality" or not. So, they rely on really simple measures of how far from goal a shot occurred. This is why you will see MASSIVE differences between quality chances as measured by publicly-available data versus quality chances as measured by a team's own analytics department (the latter being much more accurate). It all boils down to a very simple phrase: garbage in, garbage out. If unreliable data are used to build your model, your model will be unreliable. 

 

TLDR; Publicly-available analytics models are fundamentally flawed because (a) they violate core assumptions of the statistical analyses being performed and (b) they are based on unreliable data. Sorry for the rant. These ideas have been in my head for years and thought the new forum would be a good place to jot them down. 

Edited September 22, 2023 by Down By the River

[Jess]

Felt like I was in STAT 226 all over again reading that.

 

Good thoughts.

 

I like to think that theoretically, the perfect set of statistical analyses exist out there that can perfectly measure a player's effectiveness. Unfortunately, the complexity of this "perfect analysis" is beyond what is reasonably possible and a drastically more efficient proxy, albeit still imperfect proxy, is simply, the "eye test". I believe this is more or less what proponents of the eye test are claiming when they express their disdain for advanced stats.

 

All that said, I still enjoy dabbling into advanced stats (though I focus more on basic stats) as a complement to my eye test, not a replacement. If my eye test and the statistics mismatch, I've learned over the years to lean more on my eye test than the statistics. It's more likely that there is something flawed with my statistic than my 20 years of eye test experience is flawed.

[Down By the River]

4 minutes ago, -AJ- said:

Felt like I was in STAT 226 all over again reading that.

Good thoughts.

I like to think that theoretically, the perfect set of statistical analyses exist out there that can perfectly measure a player's effectiveness. Unfortunately, the complexity of this "perfect analysis" is beyond what is reasonably possible and a drastically more efficient proxy, albeit still imperfect proxy, is simply, the "eye test". I believe this is more or less what proponents of the eye test are claiming when they express their disdain for advanced stats.

All that said, I still enjoy dabbling into advanced stats (though I focus more of basic stats) as a complement to my eye test, not a replacement. If my eye test and the statistics mismatch, I've learned over the years to lean more on my eye test than the statistics. It's more likely that there is something flawed with my statistic than my 20-years of eye test experience is flawed.

I don't even think that we need a perfect measure of a player's effectiveness, but to grow the idea of analytics as a valuable tool it is important to stop doing things that we know are wrong. Guys in the Twitter analytics community are being incredibly hypocritical claiming that they see value in advanced stats while doing things that directly undermine the value of advanced stats. 

[Jess]

7 hours ago, Down By the River said:

I don't even think that we need a perfect measure of a player's effectiveness, but to grow the idea of analytics as a valuable tool it is important to stop doing things that we know are wrong. Guys in the Twitter analytics community are being incredibly hypocritical claiming that they see value in advanced stats while doing things that directly undermine the value of advanced stats. 

 

Yeah, whenever I see some advanced stat I'm unfamiliar with, my first thought always goes to things like the collection methods etc. My brother would talk to me about high danger scoring chances, but before I consider that stat, it's important to define exactly what a "high danger scoring chance" is and assess the level of objectivity with which these chances can be recorded. How much wiggle room is there in determining whether a chance is high danger or not? Does our definition of a "high danger" chance even actually realistically predict for goal scoring?

 

Feels like there's a progression when it comes to peoples' perceptions of stats in sports from distrust when they first learn about them, to full trust when they begin digging into them, and then back to a bit of skepticism again when you really get into it.

[Down By the River]

25 minutes ago, -AJ- said:

Yeah, whenever I see some advanced stat I'm unfamiliar with, my first thought always goes to things like the collection methods etc. My brother would talk to me about high danger scoring chances, but before I consider that stat, it's important to define exactly what a "high danger scoring chance" is and assess the level of objectivity with which these chances can be recorded. How much wiggle room is there in determining whether a chance is high danger or not? Does our definition of a "high danger" chance even actually realistically predict for goal scoring?

Feels like there's a progression when it comes to peoples' perceptions of stats in sports from distrust when they first learn about them, to full trust when they begin digging into them, and then back to a bit of skepticism again when you really get into it.

My excitement to see analytics brought into hockey was quickly replaced with resentment at how it seemed that people just wanted to dupe NHL teams into giving them jobs as heads of analytic departments using smoke and mirror techniques. The people who are hiring heads of analytic departments probably don't have the background knowledge to determine whether they can trust the metrics being peddled. 

[AatuD2]

You have explained the number one problem in so called advanced stats in hockey.  

 

They saw the nice, neat, one number WAR in baseball and tried to replicate it in hockey. It absolutely sucks balls as a measure. 

 

On the topic of shot quality measurements, I seem to recall Calgary Flames being the analytic darlings last year who were deemed to be ready to break out and "regress back to mean" throughout the whole season, but they never did. Watching them play, you could tell that they dominated games in terms of quantity of shots from everywhere, but rarely had east to west plays to set up those shots.  

 

Kevin Woodley (the goalie analyst) does dip into some private analytics that look at chances more in depth then what's available publicly. 

 

I used to get annoyed when I heard Thomas Drance and Rachel Doerrie talking about advanced stats as they are both just talking out of their ass, but I've learned to just let it go and look at it all with a grain of salt. Expecting someone who studied English Literature and Sports Management to understand what they know or don't know with statistics is foolish. 

[Bounce000]

My analytics are:

Scoring goals = good

Not scoring goals = bad

[Down By the River]

27 minutes ago, Bounce000 said:

My analytics are:

Scoring goals = good

Not scoring goals = bad

 

Dude. We just got a new forum. I don't think it is ready for this type of controversial take. 

[Down By the River]

1 hour ago, AatuD2 said:

You have explained the number one problem in so called advanced stats in hockey.  

 

They saw the nice, neat, one number WAR in baseball and tried to replicate it in hockey. It absolutely sucks balls as a measure. 

 

On the topic of shot quality measurements, I seem to recall Calgary Flames being the analytic darlings last year who were deemed to be ready to break out and "regress back to mean" throughout the whole season, but they never did. Watching them play, you could tell that they dominated games in terms of quantity of shots from everywhere, but rarely had east to west plays to set up those shots.  

 

Kevin Woodley (the goalie analyst) does dip into some private analytics that look at chances more in depth then what's available publicly. 

 

I used to get annoyed when I heard Thomas Drance and Rachel Doerrie talking about advanced stats as they are both just talking out of their ass, but I've learned to just let it go and look at it all with a grain of salt. Expecting someone who studied English Literature and Sports Management to understand what they know or don't know with statistics is foolish. 

 

Everyone should be immediately skeptical whenever Drance say something along the lines of "Team X out-chanced Team Y" because he basically relies on data that are basically just pulling shot location information from NHL.com to determine quality scoring chances. It is insanely frustrating to have him tout analytics while simultaneously misusing the term bell curve on a daily basis. 

 

But you are 100% right that it is far healthier to just let it go, lol. 

[Canuckle]

Great post! And spot on.

 

It always bothered me observing people stormimg down from the proverbial mountain top with a book of 'advanced' stats in their hand acting like they're the word of freaking God. Like... where do you even start, you know?

 

Here is a really good blog post/article I I found awhile back discussing some of the deeper methodological issues on NHL analytics maybe you'd like to read:

 

https://www.neutralzone.net/mens/2023/03/15/a-journey-in-hockey-analytics-and-the-shortcomings-of-the-discipline/

 

 

[Maniwaki Canuck]

Excellent points about independence and lack of standardization in classifying countable events.  On top of that there are misleading proxy stats, like shots as a measure of possession.   Contemporary 3 on 3 overtime and the old soviet teams show how wrong that is.  No doubt some proprietary analytics are more reliable than the lazy slop that dominates public discussion, but as of now, analytics seems more like a good idea than a reality. 

[Down By the River]

PDO, lol. 

 

Why is PDO a bad indicator of team performance?

1. Does not capture quality of scoring chances
2. Score effects. Related to the above point, teams that lead tend to take fewer shots, teams trailing tend to get desperate and take lots of shots from poor angles. 
3. Fails to account for shot attempts
4. If PDO does include 5-4 play, it penalizes teams who (a) have above-average PPs and/or (b) generate a lot of PP opportunities
5. Does not account for better defensive structure that clogs up middle of ice (hallmark of Tocchet's system)
6. Does not reward teams for simply having elite goaltenders/shooters
7. Shooting percentage inflation is not actually that impactful on overall results. The difference between an 11% shooting percentage (Vancouver) and 9% shooting percentage  (league-worst Chicago) amounts to about 2 additional goals every 3 games. When your goal differential is insanely high (Vancouver), an elevated shooting percentage is not the reason why you're winning games. 

Guys like Thomas Drance speak out of both sides of their mouth. They like to preface PDO discussions by saying it is just a 'gut check' measure to indicate whether what they are seeing is in line with performance. However, he uses it to suggest that Canuck performance is not sustainable. It is sustainable if you actually look under the hood of what PDO is measuring and understand why (a) it might poorly measure team performance and (b) mathematically, a decline in PDO for the Canucks would not have impacted their current record that much because they are outscoring opponents by such a wide margin.

 

For the record, I do not think this team is a cup contender. However, there are a bunch of reasons to point to why this is the case, none of which are related to PDO. 

[Bounce000]

[Down By the River]

14 minutes ago, Bounce000 said:

 

The NHL analytics community if they had any self-awareness:

 

 

[Mando27]

7 hours ago, Down By the River said:

PDO, lol. 

 

Why is PDO a bad indicator of team performance?

1. Does not capture quality of scoring chances
2. Score effects. Related to the above point, teams that lead tend to take fewer shots, teams trailing tend to get desperate and take lots of shots from poor angles. 
3. Fails to account for shot attempts
4. If PDO does include 5-4 play, it penalizes teams who (a) have above-average PPs and/or (b) generate a lot of PP opportunities
5. Does not account for better defensive structure that clogs up middle of ice (hallmark of Tocchet's system)
6. Does not reward teams for simply having elite goaltenders/shooters
7. Shooting percentage inflation is not actually that impactful on overall results. The difference between an 11% shooting percentage (Vancouver) and 9% shooting percentage  (league-worst Chicago) amounts to about 2 additional goals every 3 games. When your goal differential is insanely high (Vancouver), an elevated shooting percentage is not the reason why you're winning games. 

Guys like Thomas Drance speak out of both sides of their mouth. They like to preface PDO discussions by saying it is just a 'gut check' measure to indicate whether what they are seeing is in line with performance. However, he uses it to suggest that Canuck performance is not sustainable. It is sustainable if you actually look under the hood of what PDO is measuring and understand why (a) it might poorly measure team performance and (b) mathematically, a decline in PDO for the Canucks would not have impacted their current record that much because they are outscoring opponents by such a wide margin.

 

For the record, I do not think this team is a cup contender. However, there are a bunch of reasons to point to why this is the case, none of which are related to PDO. 

Honestly I never even heard about this PDO stuff before this season. Canucks start playing well and all of a sudden it's all people talk about now.

[Alflives]

5 minutes ago, Mando27 said:

Honestly I never even heard about this PDO stuff before this season. Canucks start playing well and all of a sudden it's all people talk about now.

These fancy stats are for geeks, who couldn’t play, to feel like they know the game. One glaring glance from Zadorov and these fancy stats guys would be running to hide behind skirts. 
Hockey is a tough game that includes intimidating the opponent. The fancy stat guys seem to never accept (because they don’t understand) that nuance. Isn’t it kind of ironic how the players actually value plus/minus but the fancy stat guys don’t? Ask Quinn Hughes about plus/minus. The players value that stat more than those PDo and pretty charts. Plus stats are winners. Minus stats are losers. Very easy to understand and it actually reflects if a player is helping his club win. Fancy stats are for pick suckers. 
Yes, Alf is drunk. Of course! 

[IBatch]

Love hockey and stats are for sure part of it.   Advanced stats have been around for a couple decades now,  two clubs even hired guys who were in love with it to be their GMs (Chakya for sure and Dubas to a lesser degree).    ARI sucked then and well still kind of sucks today, with all those guys now well into their primes, either on that team, traded or retired/didn't work out. 

 

WAR.   Have brought this up before, when the media was making a divide, traditionalist's, folks that watch a ton of hockey, and can take a scouts card and fill it out (shot, skating, compete etc etc etc etc) by watching a game, and on the other side of the spectrum, math geeks who are figuring out stuff using new metrics, and rarely watch a game (there was a real rivalry going on about a player like Bolland, and how this WAR/advanced stat stuff, back then it was mostly about puck could suck so badly with some advanced lens stuff, but be a hero..as in eye test versus math test), .    CHI was in the midst of winning their second cup and a few guys, using Corsi's idea on how to and when to give a number one a break (counting all shot attempts not just shots because, well butterfly and that sort of thing require a goalie to cover the bottom of the net as soon as anyone comes in the zone), and WAR was gaining some traction.  

 

Thing is, WAR came out and said Alex Steen was the most complete, and best player in the NHL.    Alex Steen.  lol.    Better then Crosby, the Sedins, Thornton, Ovi, Kane, Toews, Kopitar,  and a dozen or so guys like Mike Richard's, Carter, Kesler etc etc etc.   

 

I'm not a math genuis, but am pretty good at arithmetic.    Maybe they've come a long way since then, but when I read that and some of their other claims, it just didn't match up the eye test, or the basic stat lines.  

 

Things like zone entries and exits, are kind of neat.    Green was recording this with his staff using pencils way before  it became a thing.   

 

But when it comes time to evaluate players, the basic stats still are king.   Especially TOI.  It shows how much the coach trusted that player, and quality of competition.   To a degree where a coach puts you as well.   It's well known the Sedins got to start with the puck 70% plus of the time in their own zone, after a stoppage.   Guys like Sutter 25%.   Their job is to get the puck back down the ice etc.     You always need to consider the quality of competition.  TOI is a quick way to understand, is he middle six, or top six.   Does he PK.  Is he on the first PP unit etc.    What is his EV production like?   And what is his plus minus, compared to his peer group? 

 

A good example of that, the best line we've had since EP came in, was Hogs EP Podz for pure goals going in or not going in.   They plays 50-60 minutes together, EP was in a funk and demoted to the third line, played 3rd line 5 x 5 against other teams bottom six.   Guess what happened?   They didn't get scored on.    And scored a few goals.   Doesn't make them the best line though...but from that lens, it was our best line Green/Bruce season.  

 

Im with the OP.   Advanced stats, don't always add value.   And some advanced stat darlings, aren't nearly as good on the WAR cards, as they should be on the ice as a result.    Funny how WAR cards all go up once playing with better players right?   Wonder how Myers is doing...So far, 3 - that's 3 different coaches have played him in the top four,  sometimes as the top pairing.   That's because he's better than Stetcher or Burroughs.   Doesn't take a genuis to figure that out.  Kuzmenko makes almost what he does, yet is benched.   I'm sure, Myers also would have been benched, if the coaches had 3 top four RHD's to work with at times too. 

Edited December 21, 2023 by IBatch

[Kevin Biestra]

1 hour ago, IBatch said:

Love hockey and stats are for sure part of it.   Advanced stats have been around for a couple decades now,  two clubs even hired guys who were in love with it to be their GMs (Chakya for sure and Dubas to a lesser degree).    ARI sucked then and well still kind of sucks today, with all those guys now well into their primes, either on that team, traded or retired/didn't work out. 

 

WAR.   Have brought this up before, when the media was making a divide, traditionalist's, folks that watch a ton of hockey, and can take a scouts card and fill it out (shot, skating, compete etc etc etc etc) by watching a game, and on the other side of the spectrum, math geeks who are figuring out stuff using new metrics, and rarely watch a game (there was a real rivalry going on about a player like Bolland, and how this WAR/advanced stat stuff, back then it was mostly about puck could suck so badly with some advanced lens stuff, but be a hero..as in eye test versus math test), .    CHI was in the midst of winning their second cup and a few guys, using Corsi's idea on how to and when to give a number one a break (counting all shot attempts not just shots because, well butterfly and that sort of thing require a goalie to cover the bottom of the net as soon as anyone comes in the zone), and WAR was gaining some traction.  

 

Thing is, WAR came out and said Alex Steen was the most complete, and best player in the NHL.    Alex Steen.  lol.    Better then Crosby, the Sedins, Thornton, Ovi, Kane, Toews, Kopitar,  and a dozen or so guys like Mike Richard's, Carter, Kesler etc etc etc.   

 

I'm not a math genuis, but am pretty good at arithmetic.    Maybe they've come a long way since then, but when I read that and some of their other claims, it just didn't match up the eye test, or the basic stat lines.  

 

Things like zone entries and exits, are kind of neat.    Green was recording this with his staff using pencils way before  it became a thing.   

 

But when it comes time to evaluate players, the basic stats still are king.   Especially TOI.  It shows how much the coach trusted that player, and quality of competition.   To a degree where a coach puts you as well.   It's well known the Sedins got to start with the puck 70% plus of the time in their own zone, after a stoppage.   Guys like Sutter 25%.   Their job is to get the puck back down the ice etc.     You always need to consider the quality of competition.  TOI is a quick way to understand, is he middle six, or top six.   Does he PK.  Is he on the first PP unit etc.    What is his EV production like?   And what is his plus minus, compared to his peer group? 

 

A good example of that, the best line we've had since EP came in, was Hogs EP Podz for pure goals going in or not going in.   They plays 50-60 minutes together, EP was in a funk and demoted to the third line, played 3rd line 5 x 5 against other teams bottom six.   Guess what happened?   They didn't get scored on.    And scored a few goals.   Doesn't make them the best line though...but from that lens, it was our best line Green/Bruce season.  

 

Im with the OP.   Advanced stats, don't always add value.   And some advanced stat darlings, aren't nearly as good on the WAR cards, as they should be on the ice as a result.    Funny how WAR cards all go up once playing with better players right?   Wonder how Myers is doing...So far, 3 - that's 3 different coaches have played him in the top four,  sometimes as the top pairing.   That's because he's better than Stetcher or Burroughs.   Doesn't take a genuis to figure that out.  Kuzmenko makes almost what he does, yet is benched.   I'm sure, Myers also would have been benched, if the coaches had 3 top four RHD's to work with at times too. 

 

 

Yeah but how hot was Alex Steen's girlfriend?

 

 

[IBatch]

13 hours ago, Mando27 said:

Honestly I never even heard about this PDO stuff before this season. Canucks start playing well and all of a sudden it's all people talk about now.

PDO (puck luck as they basically want to call it...and well in real life, just like hockey, you make your own luck!) has been around for quite awhile.    It's not anything to be too concerned about.    Funny how Wayne Gretzky's shooting percentage gets de-bunked by folks who never watched him play.   All he did his entire youth was pick corners, and fake our goalies with his shoulders and fake shots - then opened them up and made them look stupid.     Kuzmenko's however, was way too high given what he accomplished in the KHL.    That's the difference between using stats properly.    A lot of folks expected a regression.    Some fans on the CDC a wanted to lock Kuzmennko up for 6-8 years.   Good thing Allvin is the GM.     To get a baseline you need a sample size.     The "people" talking about PDO, are doing their best to invalidate the teams accomplishments.    The Vancouver media including the goofs that are podcasting or whatever, need to start validating a little more.

 

At least we have a coach that keeps it real.   

[Flipper]

Congratulations, you guys managed to bring back the stats headaches I used to get when I had to take those classes in school.

 

I always found stats interesting but never enough to really understand why some people loved those classes. I was reading the original post and trying to think of a meaningful way to contribute to the discussion but ended up with, better to keep one’s mouth shut and be thought a fool than open it and remove all doubt…

 

I do wonder though if after a significant amount of time wearing the on ice trackers the players are now wearing, say 10 years or more, if the amount of raw data will be available to really analyze some of the things people are discussing. Like will the tracking data that will be available remove some of the observation bias they’ve had to rely on over the years? 
 

And that’s enough stats for me for another 5 years. Back to the eye test…

[Alflives]

11 minutes ago, Flipper said:

Congratulations, you guys managed to bring back the stats headaches I used to get when I had to take those classes in school.

 

I always found stats interesting but never enough to really understand why some people loved those classes. I was reading the original post and trying to think of a meaningful way to contribute to the discussion but ended up with, better to keep one’s mouth shut and be thought a fool than open it and remove all doubt…

 

I do wonder though if after a significant amount of time wearing the on ice trackers the players are now wearing, say 10 years or more, if the amount of raw data will be available to really analyze some of the things people are discussing. Like will the tracking data that will be available remove some of the observation bias they’ve had to rely on over the years? 
 

And that’s enough stats for me for another 5 years. Back to the eye test…

Observation bias? We a fans, so of course there will be bias.

And stats are fine but need to be connected to the actual game by those who played and understand the game. Otherwise stats are just numbers and lack meaningful context. 
 

[Bob Long]

7 minutes ago, Alflives said:

Observation bias? We a fans, so of course there will be bias.

And stats are fine but need to be connected to the actual game by those who played and understand the game. Otherwise stats are just numbers and lack meaningful context. 
 

 

Anything beyond the most basic stats also have bias, its called assumptions. 

[Down By the River]

21 minutes ago, Flipper said:

Congratulations, you guys managed to bring back the stats headaches I used to get when I had to take those classes in school.

 

I always found stats interesting but never enough to really understand why some people loved those classes. I was reading the original post and trying to think of a meaningful way to contribute to the discussion but ended up with, better to keep one’s mouth shut and be thought a fool than open it and remove all doubt…

 

I do wonder though if after a significant amount of time wearing the on ice trackers the players are now wearing, say 10 years or more, if the amount of raw data will be available to really analyze some of the things people are discussing. Like will the tracking data that will be available remove some of the observation bias they’ve had to rely on over the years? 
 

And that’s enough stats for me for another 5 years. Back to the eye test…

 

In other fields, we've known for nearly a century that unstructured judgement leads to poor decision-making. People can talk about the eye test as a way to debunk stats, which is fine/fair, but the eye test alone can also be pretty deceiving, especially if there are no checks/balances. To me, not talking about hockey here, but if you want to be able to get closer to the "truth", quantitative research methods are absolutely necessary. 

[Down By the River]

2 minutes ago, Bob Long said:

 

Anything beyond the most basic stats also have bias, its called assumptions. 

Statistical assumptions are not the same as bias. Quite the opposite. Assumptions of ordinary least squares regression are there to ensure that bias does not occur, or, at the very least, does not impact your analysis in ways that create bias.