Analyzing Luck
Over the past four weeks, I’ve discussed a number of statistics that can help you win your fantasy baseball league. In each article, I’ve mentioned the influence that “luck” plays on the statistic. I realized, though, that I’ve never actually talked about what I mean by the word “luck.”
When we evaluate a player, there are several layers to look at. The most obvious of these layers is the results. This layer consists of stats like batting average, RBIs, ERA, and other categories that typical fantasy leagues use for scoring. Since they are used for scoring, lazy owners – or perhaps uninformed owners – focus solely on them. This, as I’m sure you’ve realized, isn’t especially sound. If we look at the relationship between a player’s batting average from year-to-year, we see that there is an unspectacular 0.37 correlation coefficient and a pretty poor 0.14 R2 (using 2004-2007 data for batters with at least 200 at-bats in both years). Other statistics, like ERA, perform even worse.
What we need to do instead is focus on a player’s skills and indicators. For a category like ERA, skills include the things we mentioned in our discussion on DIPS Theory, things like strikeouts, walks, and ground balls. When we talk about ERA indicators, we’re referring to stats like Left on Base Percentage, Batting Average on Balls in Play, and Home Run per Fly ball.
These indicators, though, tend to fluctuate a good deal, and this fluctuation is often referred to as “luck.” What I mean when I chalk some thing up to luck is, generally speaking, unexplained variation in a statistic. This does not, however, mean that “luck,” in the sense that I am using it, is completely random. It could just be that we don’t have the proper stats at the moment to filter out the noise.
For example, there is a great deal of fluctuation with hitter BABIP. Right now, we don’t really have a great method for predicting BABIP. I may refer to the fluctuation in BABIP as “luck,” but that doesn’t mean that skill isn’t involved in the parts of BABIP that we can’t currently predict. In fact, I have a feeling that within another couple of years we will have made significant strides in predicting BABIP. Greg Rybarczyk’s wonderful program, HitTracker, keeps track of how hard batters hit the ball (Speed Off Bat). Once Greg gets enough help to track all batted balls (not just homers, as HitTracker currently does), I think this Speed Off Bat data will be the key to a hitter’s BABIP. Greg actually penned an article for the 2008 Hardball Times Annual that briefly examines this idea, which definitely shows promise.
In other instances, we might have information but have no real way of quantifying it. While we try to objectify statistics as much as possible, we need to remember that we are studying human beings. And with human beings come unpredictability. Maybe a player’s father died or maybe the player has been diagnosed with a mental disorder, like depression. In July of this year, Matt Wise of the Milwaukee Brewers hit a player in the head with a pitch. He tanked the rest of the year, but how much of this was due to that stray pitch? These types of things can obviously affect a player’s performance, but they affect every player differently, and we therefore have no way of getting real feel for just how much of a player’s performance variation should be attributed to them.
Perhaps even more prevalent than being unable to quantify this information is the absence of the info in the first place. Again, we are dealing with human beings, and just because they are baseball players and are in the media spotlight does not mean that they are required to share every detail of their lives with us. Things may be happening behind the scenes that affect a player’s performance that we are oblivious to. We simply have no way of knowing, so we simply classify it as “luck,” which, as I mentioned, isn’t just random chance. It is unexplained variation in stats, which these personal issues fall squarely into.
Other times, this unexplained variation won’t be due to a lack of available stats or information at all. In some cases, it will be random chance that affects the numbers. In some cases, it will simply be pure, dumb luck. This is true when working with any set of statistics, and while it can’t be accounted for, it does need to be recognized. That is part of what we mean when we talk about a stat like HR/FB regressing to the mean. Stats, especially baseball stats, are subject to unexplained external noise that is not likely to be repeated. If it isn’t repeatable, than it isn’t really a skill, is it? And if it isn’t repeatable, than what possible forecasting value does it have? We need to focus on the stats that we have to work with now (while continuing to try and come up with new and improved ones), utilize the components that we can explain, and expect the pieces that we cannot explain to even themselves out, at least for the time being.
Moving away from my explanation of luck, I wanted to talk about how we should be expecting luck to affect future numbers. Say a pitcher posts a 2% HR/FB rate in the first half of a season. We know that pitchers tend to regress towards the league average of around 11%. Does this mean that the pitcher should post a HR/FB around 20% in the second half to even things out?
I’ve known many players to point to the “law of averages,” as they call it, to try to validate this hypothesis. They say that if a player was expected at the beginning of the year to post an 11% HR/FB rate and it is only 2% in the first-half (which is unsustainable), then there is a good chance he will post a HR/FB significantly higher than 11% in the second-half to make up for it.
In baseball – and anyone who has ever played poker seriously or has some knowledge of games theory knows this – it doesn’t matter how you arrive at a particular point. All that matters is that you are there and that you have a clean slate at every single new moment that arrives. By originally projecting an 11% HR/FB rate, we were expecting this pitcher to be unaffected by luck, or to be affected by neutral luck (however you look at it). Just because he catches a run of really good luck doesn’t mean we should change our opinion of him in this regard. We should still expect him to post luck-neutral stats.
Luck, when examining a large quantity of players over a long period of time, will tend to even itself out. But when you are looking at an individual player over a short period of time, luck should not be expected to correct itself so quickly. It just doesn’t work that way. You should always expect luck to be neutral moving forward, at all times, because as we said before, our definition of luck is “unexplained variation.” If it is unexplained, why would we try to predict it? It is a fool’s errand.
When we evaluate a player, there are several layers to look at. The most obvious of these layers is the results. This layer consists of stats like batting average, RBIs, ERA, and other categories that typical fantasy leagues use for scoring. Since they are used for scoring, lazy owners – or perhaps uninformed owners – focus solely on them. This, as I’m sure you’ve realized, isn’t especially sound. If we look at the relationship between a player’s batting average from year-to-year, we see that there is an unspectacular 0.37 correlation coefficient and a pretty poor 0.14 R2 (using 2004-2007 data for batters with at least 200 at-bats in both years). Other statistics, like ERA, perform even worse.
What we need to do instead is focus on a player’s skills and indicators. For a category like ERA, skills include the things we mentioned in our discussion on DIPS Theory, things like strikeouts, walks, and ground balls. When we talk about ERA indicators, we’re referring to stats like Left on Base Percentage, Batting Average on Balls in Play, and Home Run per Fly ball.
These indicators, though, tend to fluctuate a good deal, and this fluctuation is often referred to as “luck.” What I mean when I chalk some thing up to luck is, generally speaking, unexplained variation in a statistic. This does not, however, mean that “luck,” in the sense that I am using it, is completely random. It could just be that we don’t have the proper stats at the moment to filter out the noise.
For example, there is a great deal of fluctuation with hitter BABIP. Right now, we don’t really have a great method for predicting BABIP. I may refer to the fluctuation in BABIP as “luck,” but that doesn’t mean that skill isn’t involved in the parts of BABIP that we can’t currently predict. In fact, I have a feeling that within another couple of years we will have made significant strides in predicting BABIP. Greg Rybarczyk’s wonderful program, HitTracker, keeps track of how hard batters hit the ball (Speed Off Bat). Once Greg gets enough help to track all batted balls (not just homers, as HitTracker currently does), I think this Speed Off Bat data will be the key to a hitter’s BABIP. Greg actually penned an article for the 2008 Hardball Times Annual that briefly examines this idea, which definitely shows promise.
In other instances, we might have information but have no real way of quantifying it. While we try to objectify statistics as much as possible, we need to remember that we are studying human beings. And with human beings come unpredictability. Maybe a player’s father died or maybe the player has been diagnosed with a mental disorder, like depression. In July of this year, Matt Wise of the Milwaukee Brewers hit a player in the head with a pitch. He tanked the rest of the year, but how much of this was due to that stray pitch? These types of things can obviously affect a player’s performance, but they affect every player differently, and we therefore have no way of getting real feel for just how much of a player’s performance variation should be attributed to them.
Perhaps even more prevalent than being unable to quantify this information is the absence of the info in the first place. Again, we are dealing with human beings, and just because they are baseball players and are in the media spotlight does not mean that they are required to share every detail of their lives with us. Things may be happening behind the scenes that affect a player’s performance that we are oblivious to. We simply have no way of knowing, so we simply classify it as “luck,” which, as I mentioned, isn’t just random chance. It is unexplained variation in stats, which these personal issues fall squarely into.
Other times, this unexplained variation won’t be due to a lack of available stats or information at all. In some cases, it will be random chance that affects the numbers. In some cases, it will simply be pure, dumb luck. This is true when working with any set of statistics, and while it can’t be accounted for, it does need to be recognized. That is part of what we mean when we talk about a stat like HR/FB regressing to the mean. Stats, especially baseball stats, are subject to unexplained external noise that is not likely to be repeated. If it isn’t repeatable, than it isn’t really a skill, is it? And if it isn’t repeatable, than what possible forecasting value does it have? We need to focus on the stats that we have to work with now (while continuing to try and come up with new and improved ones), utilize the components that we can explain, and expect the pieces that we cannot explain to even themselves out, at least for the time being.
Moving away from my explanation of luck, I wanted to talk about how we should be expecting luck to affect future numbers. Say a pitcher posts a 2% HR/FB rate in the first half of a season. We know that pitchers tend to regress towards the league average of around 11%. Does this mean that the pitcher should post a HR/FB around 20% in the second half to even things out?
I’ve known many players to point to the “law of averages,” as they call it, to try to validate this hypothesis. They say that if a player was expected at the beginning of the year to post an 11% HR/FB rate and it is only 2% in the first-half (which is unsustainable), then there is a good chance he will post a HR/FB significantly higher than 11% in the second-half to make up for it.
In baseball – and anyone who has ever played poker seriously or has some knowledge of games theory knows this – it doesn’t matter how you arrive at a particular point. All that matters is that you are there and that you have a clean slate at every single new moment that arrives. By originally projecting an 11% HR/FB rate, we were expecting this pitcher to be unaffected by luck, or to be affected by neutral luck (however you look at it). Just because he catches a run of really good luck doesn’t mean we should change our opinion of him in this regard. We should still expect him to post luck-neutral stats.
Luck, when examining a large quantity of players over a long period of time, will tend to even itself out. But when you are looking at an individual player over a short period of time, luck should not be expected to correct itself so quickly. It just doesn’t work that way. You should always expect luck to be neutral moving forward, at all times, because as we said before, our definition of luck is “unexplained variation.” If it is unexplained, why would we try to predict it? It is a fool’s errand.











