I’ve been doing plenty of taking a look at depth charts this week. All of us FanGraphs writers have – these positional energy rankings don’t write themselves. If you have a look at the majors by means of this lens, you’ll naturally do plenty of occupied with flooring and ceiling. The Yankees are taking part in who at third base? The Brewers are getting how a lot WAR by avoiding weak spots? The Crimson Sox have that many outfielders?
I’ve written some workforce overviews this winter. In them, I make the next declare: “Constructing a workforce that outperforms opponents on the energy of its fifteenth to twenty sixth finest gamers being far superior to their counterparts on different golf equipment may assist in the canine days of August, when everybody’s taking part in their depth guys and cobbling collectively a rotation, however that received’t fly in October.” The converse of that declare – that stars matter disproportionately in October – is an element and parcel of this depth argument. However is that true?
Some may say that the very best time to reply this query is when the playoffs are simply across the nook. I’d counter that these folks haven’t simply spent seven hours observing a pile of acceptable-but-not-overwhelming third base and beginning pitcher choices and making an attempt to jot down one thing about each. So within the spirit of doing something apart from taking a look at energy rankings, I made a decision to check out this assumption.
First issues first: I settled on some definitions. I broke my information up into two elements: 1995-2011, the one-wild-card period, and 2012-present, the many-wild-cards period. I borrowed Dan Szymborski’s technique for evaluating “what issues” in the case of playoff roster configuration.
Dan’s plan is delightfully easy. First, take an inventory of playoff groups, playoff video games, playoff outcomes, and regular-season statistics for these playoff groups. In the event you’re making an attempt to find out whether or not a given regular-season “factor” issues, you simply run two regressions: one which tries to foretell the end result of playoff video games utilizing regular-season Pythagorean file and home-field benefit, and one other that tries to foretell these video games utilizing Pythag, home-field, and your chosen “factor.” If including your factor makes the predictions higher, hooray! If it doesn’t, it’s again to the drafting board. He used an ROC curve, which is ideal for these functions: It’s good at testing whether or not an impact exists extra so than how good your mannequin is.
Dan used his technique to look into plenty of components you’ll hear about on playoff broadcasts: bullpen energy, house run reliance, contact price, second-half file, September file, playoff expertise, and so forth. He discovered an entire lot of nothing. I repeated his research with a couple of further years of information and in addition discovered an entire lot of nothing. Then I began including new stuff – and splitting by period, which deserves its personal rationalization.
You Aren’t a FanGraphs Member
It appears such as you aren’t but a FanGraphs Member (or aren’t logged in). We aren’t mad, simply upset.
We get it. You need to learn this text. However earlier than we allow you to get again to it, we would prefer to level out a couple of of the great the reason why you must develop into a Member.
1. Advert Free viewing! We can’t bug you with this advert, or another.
2. Limitless articles! Non-Members solely get to learn 10 free articles a month. Members by no means get reduce off.
3. Darkish mode and Basic mode!
4. Customized participant web page dashboards! Select the participant playing cards you need, within the order you need them.
5. One-click information exports! Export our projections and leaderboards on your private tasks.
6. Take away the pictures on the house web page! (Truthfully, this does not sound so nice to us, however some folks needed it, and we like to offer our Members what they need.)
7. Much more Steamer projections! Now we have handedness, percentile, and context impartial projections accessible for Members solely.
8. Get FanGraphs Stroll-Off, a personalized yr finish evaluation! Discover out precisely the way you used FanGraphs this yr, and the way that compares to different Members. Do not be a sufferer of FOMO.
9. A weekly mailbag column, completely for Members.
10. Assist help FanGraphs and our complete employees! Our Members present us with vital sources to enhance the location and ship new options!
We hope you may contemplate a Membership at the moment, for your self or as a present! And we understand this has been an awfully lengthy gross sales pitch, so we have additionally eliminated all the opposite adverts on this article. We did not need to overdo it.
Beginning in 2012, MLB expanded the variety of groups that certified for the playoffs, first to 10 and ultimately to 12. In my estimation, the composition of the postseason subject has modified meaningfully within the expanded playoff period. For one factor, it’s simpler to make it. There are simply extra spots now, which lowers the bar essential to get in. A workforce with common gamers throughout the board was far much less more likely to make it into the playoffs in 1998 versus in 2025. It’s simply math; much more groups make the playoffs at the moment.
On the identical time, groups don’t distribute taking part in time the way in which they used to. It’s most notable on the pitching aspect, the place an more and more broad group of pitchers every takes a decreasingly giant innings share. It’s true on the hitting aspect to a lesser diploma; extra relaxation days, extra platoons, extra deal with load administration and surviving the season-long grind. In the meantime, groups are treating the playoffs as their very own paradigm: faster hooks on marginal starters, higher-leverage bullpen utilization, and the very best place gamers within the lineup each sport.
I made a decision that the appropriate strategy to deal with this dataset was by splitting it in two. Conveniently, there are an analogous variety of seasons on both sides: 17 seasons with eight groups within the playoffs, and 13 (after excluding 2020) with 10 or extra groups reaching postseason play. I’m open to the argument that the change from 10 to 12 playoff groups ought to additional stratify the information, however that might be slicing issues too thinly, with solely 4 seasons below the newest guidelines.
Dan’s base technique – regular-season Pythagorean file plus home-field benefit – was far more practical at predicting the winner of video games within the 1995-2011 period. Right here’s the ROC curve for utilizing Pythagorean and home-field to foretell playoff winners in these years:
Dan picked these graphs as a result of they’re simple to intuit visually. The extra space under the road, the higher: You need it to go up and to the left of the diagonal, basically. This isn’t an important match, however the fact of the matter is, predicting the winner of baseball video games is an inexact science.
That was 1995-2011. Right here’s 2012-2025:
In different phrases, utilizing Pythagorean expectation to foretell the winner of playoff video games isn’t working in addition to it used to. This is smart to me, although I can’t level to a smoking gun within the information. Groups deal with the common season in a different way than the playoffs. There’s no explicit purpose that the components that turns regular-season run scoring into an estimation of workforce expertise has to work completely for out-of-sample video games. It’s a regression that assumes each run scored by a workforce is equally indicative of its expertise. We all know that’s not fairly true – it’s simply an abstraction of actuality we’re keen to just accept. It’s not stunning that Pythag is likely to be an imperfect reflection of baseball, as a result of it was actually designed as an imperfect reflection of baseball. It nonetheless works pretty properly, and it nonetheless captures so much about what’s occurring, however the linkages don’t appear to be as clear anymore.
Now we get to the fascinating half. I gave the mannequin a brand new piece of knowledge – the WAR accrued by every workforce’s high 5 gamers through the common season. I requested it to foretell the end result of every sport with that new info in hand after which in contrast it to the Pythag-and-home-field “baseline” model. I’ve drawn the baseline curve in grey and the curve that makes use of data about every workforce’s finest 5 gamers in inexperienced:
Within the 1995-2011 period, the baseline mannequin carried out properly sufficient that including a further characteristic didn’t assist a lot. The ROC curves are mainly the identical – the AUC worth you see reported there may be the world below the curve. However the identical hasn’t been true within the expanded playoff period:
My plain-English interpretation of the information is that whereas figuring out what number of runs a workforce scored and allowed explains a few of how that workforce will carry out within the playoffs, it’s explaining lower than ever, and the way good its finest gamers are appears so as to add some related info. That info has by no means harm; it’s simply extra crucial than it was earlier than in the case of predicting how a workforce will do within the playoffs.
Truthfully, this is smart in case you cease and give it some thought. I ran a standard sense check to substantiate what I feel is occurring. I took the share of taking part in time accrued by every participant within the playoffs after which compelled regular-season taking part in time to reflect these precise fractions. In different phrases, if a workforce’s rotation shortened from 5 to 4 pitchers within the playoffs, I redid its regular-season numbers assuming that taking part in time distribution. I saved per-plate look and per-inning statistics fixed and allotted 100% of the workforce’s full regular-season taking part in time.
Within the 2025 common season, the Brewers racked up a powerful 46.8 WAR. The Dodgers had been hardly higher, checking in at 49.8 WAR. However when these two groups entered the playoffs, one thing modified. The Brewers performed a squad that, over 162 video games, would have produced an estimated 53.1 WAR. They reduce out a couple of part-time backups, redid the bullpen, and usually improved across the margins. The Dodgers performed a squad that might have gathered 67.5 WAR over a full season. They remade their rotation nearly utterly and performed their finest hitters extra. The Dodgers and Brewers had very comparable run differentials within the common season. However these squads weren’t those who performed within the postseason, and the Dodgers had one other gear accessible.
Is that precisely the identical as measuring the WAR of solely the highest 5 gamers on a workforce? No, nevertheless it’s actually correlated. And as a supporting impact, I attempted some less-severe slices: WAR of the highest 10 gamers on a squad, WAR of the highest 15 gamers on a squad, an HHI coefficient, and varied different measures of workforce composition. The extra gamers I threw into the combo, the weaker the impact acquired. Utilizing the WAR of the highest 10 gamers nonetheless gave me a statistically vital consequence, however solely barely so. Shifting to the highest 15 meant no significance. Measuring solely the sixth to fifteenth gamers added no predictive energy in any respect. In different phrases, figuring out one thing about the very best gamers on a workforce has been useful for making playoff predictions within the double-digit-playoff-team period.
It’s value mentioning that figuring out this impact exists isn’t the identical as having a mannequin that captures that impact properly. The AUC/ROC technique measures how properly a mannequin ranks outcomes ordinally, not how confidently it predicts these outcomes. It doesn’t check calibration, merely the power to tell apart broadly no matter calibration. And whereas these values are clearly higher than likelihood, they’re nowhere close to offering actionable single-game predictions. There’s a structural sample throughout 500 or so video games within the a number of wild-card period that implies that figuring out a workforce’s finest gamers provides you extra invaluable details about that workforce’s probabilities, and that groups with higher stars outperform within the playoffs relative to what you’d anticipate in case you solely checked out run differential within the common season. I can’t say greater than that – however I feel that’s a robust conclusion nonetheless.