I’m a lifelong Dodgers fan and I waited for 32 years for the team to win another World Series. But during this period of time, the sport has certainly seen much change. With the availability of huge amounts of data, sophisticated computers and advanced analytics, the strategies have become increasingly based on the numbers. It seems that AI (Artificial Intelligence) has dominated the decision making process.
We got an example of this in the crucial Game 6 of the World Series. Tampa Bay Rays manager Kevin Cash took out pitcher Blake Snell from the game, even though he was nearly flawless.
It does look like Cash’s move was based on the numbers—and the decision proved to be a blunder. The Dodgers would rally and win the game (the score was 3-1).
This does show how high-stakes analytics can be. But it also highlights the risks and challenges.
So in light of all this, what are the lessons here? To answer this question, I asked some top people in the AI field and here’s what they had to say:
Sheldon Fernandez, the CEO of Darwin AI:
Hindsight is always 20/20, but Cash’s decision to pull Snell illustrates the myopia that can occur when analytics are elevated above commonsense. In artificial intelligence, we sometimes talk of “confounding variables,” which can limit the effectiveness of decisions driven purely by data because they mask the relationship between influencers. For example, the data may suggest that Snell has a higher expected ERA than his replacement (Anderson) in this scenario, while not considering sample size or other trends in the series.
Going with “the gut” can also have serious drawbacks, of course. Terry Collins ill-fated decision to have Matt Harvey pitch the ninth inning in Game 5 of the 2015 World Series will haunt Mets fans for some time as it gifted Kansas City the championship that night (my beloved Blue Jays at least took them to six games in the preceding ALCS, but that’s another matter).
However, even on its own analytical terms, the decision is hard to understand: Snell was pitching the game of his life and his replacement had struggled in his past five outings. That the Dodgers bench breathed a sigh of relief when Cash went to the bullpen brings to mind the following aphorism: “analytics are meant to inform decisions, not make decisions.”
Kathy Brunner, the Founder of Acumen Analytics:
The batter Corey Seager has hit .218 against lefties all season so you would pull him based on the numbers—a guy who has nine K’s through six innings pitched with 73 pitches thrown. Plot twist the guy they brought in has been the best reliever in all of baseball this season–but has given up six runs in six straight appearances over the past two weeks–so which is more accurate? Neither. They are both equally accurate. Baseball is situational and averages over the season become less significant as more data is amassed. A batting average of .400 in April after two games and five at bats is much less telling than a .400 batting average in September after 170 games and 500+ at Bats. The situation is the same when analyzing any position.
But it looks as though Tampa Bay decided the data from the entire season was more significant in this situation than what had occurred over the past few games. They were wrong. Add that back into the model and see what happens next year.
Tim Baumgartner, VP of Analytics at Laughlin Constable:
Moneyball forever changed baseball. It proved pennywise teams like Tampa Bay can compete with wealthy teams like Los Angeles by harnessing the power of data analysis to drive successful roster-building and in-game strategies.
But the game of baseball is as much art as it is science; the game isn’t played on a spreadsheet. Tampa Bay’s data strategy told them starting pitchers struggle facing batters for the third time in a game. League-wide data supports this and many teams subscribe to this strategy.
However, in game six with a championship on the line, a data-driven team leaned too far into that strategy. They cut a phenomenal pitching performance short adhering to that approach, only to watch their lead–and championship hopes–slip away.
The lesson? Teams that know how and when to use their qualitative and quantitative knowledge together (Los Angeles uses analytics too!) are usually most successful. And nine-figure payrolls don’t hurt, either!
Omri Orgad, the managing director of North America at Luminati Networks:
Like in Back to the Future, we see only one outcome of reality, but we can’t forget that analytics is still a statistical tool. The reason for the move was to increase the number of outcomes where the Rays would win, but as we all know, there is no 100% guarantee. We forget about the other options: (1) Snell staying in and the Rays lose; or (2) Snell being replaced, and the Rays win.
Data and analytics are tools for managers and most likely played a big role in bringing the Rays to the World Series, which is challenging in itself. In the end, the player has to throw the ball and win.
Michael Berthold, CEO and co-founder of KNIME:
The lesson learned here is actually very simple: A decision derived from many, many data points isn’t conducive to predicting the future of an isolated event—especially when it’s about a very finite win or lose type outcome (a World Series, no less!). Those decisions, hopefully, only improve your chances.
Mr. Cash probably had statistics on his side, meaning he improved the chances for his team to win, but that only meant it was a bit more likely to work out positively. Unfortunately for him, the dice fell differently, and things didn’t end well. However, if he keeps following the data, he will win more often than others who go by gut feeling, and over the next decade or so, he will end his career as a successful, winning manager.
Saif Ahmed, who is the Product Owner of Machine Learning at Kinetica:
As we continue to see analytics applied to real-world scenarios, we need to dispel the common misconception that data science is predictive. As with any science, it involves running a repeatable experiment to test a hypothesis, and then generates statistics about likelihoods—not a consistent outcome. If Tampa Bay Rays manager Kevin Cash relied on a data science model for a single decision, even if the factors at play matched his model exactly, the chance of success is still just a statistic—even at 99%, it is not a guarantee. At the end of the day, we love sports because of the unpredictable human element, whether that’s a manager’s ultimate decision to pull a player or a player’s Hail Mary that just works out.
Data science can’t go too far, but our reliance on it can. Any individual model decision can be questioned, but if the model features are reasonable and reflect past instances accurately, you should feel justified in following the model. Transparency is key: we should be able to look at the variables the model takes into account. Following last night’s decision, if an individual model leads to a decision that is questionable and we discover a new dynamic at play (pun intended), it is a good opportunity to revisit the model and introduce more variables to more accurately capture the determinants of success. Data Science is always an iterative process, just like any science.
Joe DosSantos, the Chief Data Officer at Qlik:
It’s hard for a person like me who makes their living in analytics to say, but yes, I think that analytics in sports has gone too far. Cash pulling Blake Snell was just the latest example of some strange decisions in sports. Another recent example was that two teams in the NFL who were down 14 went for two point conversions instead of the extra point. AWS is running commercials about catch probability that hardly anyone can understand.
Moneyball is real, but it works because of odds over 162 games. If you need to pick something to do in this very moment, statistics play a part, but so does human psychology. Snell had the Dodgers mentally beaten. Mookie Betts had struck out twice. You can’t see that in the numbers. The change gave them life. Similarly, while going for a two point conversion over 100 games might pay off, the psychological impact on the team in the moment is lost. A miss causes the team to give up on the game, and potentially even their coach.
Tom (@ttaulli) is an advisor/board member to startups and the author of Artificial Intelligence Basics: A Non-Technical Introduction and The Robotic Process Automation Handbook: A Guide to Implementing RPA Systems. He also has developed various online courses, such as for the COBOL and Python programming languages.