The projection system OOPSY made its main league debut this yr. So how did it do?
OOPSY’s methodology mirrors that of the opposite FanGraphs projection programs, with just a few twists — most notably, the inclusion of bat velocity for hitters and Stuff+ for pitchers. Projection programs are comprised of many alternative elements, nevertheless, together with growing older curves, main league equivalencies to account for minor league and international league efficiency, recency weights, regression to the imply, league run atmosphere, and park elements. There are lots of methods for projection programs to face out or lag behind their friends past simply the inclusion of a selected variable like bat velocity. Projection programs are comprised of tons of of small methodological selections. Given the sum whole of the selections that went into OOPSY, did it maintain its personal in 2025 relative to its extra established friends?
To assessment the projections, this text follows trade greatest practices as outlined by Tom Tango, MLBAM’s senior knowledge architect. I’ve performed this assessment course of for pitchers earlier than, as my pitching projections have been featured by Eno Sarris in The Athletic since 2023, however this was my first yr publishing a full set of hitting projections. This assessment focuses on wOBA for hitters and wOBA in opposition to for pitchers (a substitute for ERA, additional outlined under). These metrics are sometimes the main focus of projection system critiques, a very powerful hitting and pitching charge statistics for projection programs to get proper from a “real-life” perspective. Each are catch-all charge statistics that measure, respectively, a participant’s offensive and pitching worth. The varied element projections, e.g., Ok% and BB%, feed into these catch-all metrics.
As a information for anybody aiming to duplicate the projections assessment course of, I’ll stroll you thru the steps I took, borrowing from my 2023 assessment.
First, I grabbed all of the 2025 statistics from the FanGraphs leaderboards. I eliminated the hitters who pitched from the pitching knowledge, and vice-versa (although I saved Shohei Ohtani in each knowledge units). I additionally downloaded all the preseason projections and merged them with the 2025 statistics leaderboard. I left any lacking projections clean, returning to them in a while.
Subsequent, I rescaled all the projections in order that they assumed the identical league run atmosphere. Different outstanding projection critiques could generally skip over this, however sabermetricians view it as a non-negotiable step. In the event you fail to do that, the efficiency of your projections could also be dominated by the assumptions you make concerning the league run atmosphere, that are impacted by selections that fall exterior the standard scope of a projection system, e.g., selections about how the baseball is manufactured. If you’re nonetheless interested in league run atmosphere, e.g., you wish to see which projections greatest accounted for the consequences of a brand new rule change, it’s greatest to order it for a separate train.
For demonstration functions, right here’s how I rescaled OOPSY’s wOBA projections. I checked out all the 2025 main league hitters with an OOPSY projection. I took the typical of their wOBA projections weighted by their precise variety of plate appearances in 2025; for OOPSY, this was .316. Subsequent, I rescaled all the projections to imagine the identical .313 2025 league common wOBA. To do that rescaling for OOPSY, I subtracted .316 from every wOBA projection, then added .313. To transform ERA to wOBA in opposition to, Tango’s most popular catch-all metric for pitchers, I divided every pitcher’s ERA projection by their projection system’s league common ERA, took the sq. root, after which multiplied by the 2025 league common wOBA, .313.
Subsequent I had to decide on a technique for addressing the lacking gamers. Some projection programs cowl a broader inhabitants of gamers (e.g., ZiPS), whereas others concentrate on comparatively extra established gamers (e.g., THE BAT X). You possibly can mission all the lacking gamers to be league common, the MARCEL method, or barely worse than league common, which is often a bit extra correct. I selected a mixture of those methods, which I element under.
Lastly, I calculated the foundation imply sq. errors (RMSE) for every statistic from every projection system. I might have additionally calculated the imply absolute error, however RMSE is extra typical. RMSE represents the usual deviation of the residuals, a measure of the standard distance between a forecast and an precise outcome. I weighted the RMSE by 2025 plate appearances for bats and whole batters confronted for arms. Following directions from an alternate with Tango, I additionally calculated the distinction between every projection and a naive projection, with the naive projection assuming everybody has the identical projection. To calculate the distinction between a projection and a naive projection, I squared each, subtracted the smaller quantity from the bigger quantity, after which took the sq. root.
Following these steps, the desk under exhibits the RMSE of the preseason forecasts for the complete pattern of hitters and pitchers in 2025 (a decrease RMSE is healthier). The naive forecast assumes everybody has a league common projection. The composite forecast is the typical of the 4 standalone programs (OOPSY, THE BAT X, Steamer, and ZiPS). For lacking gamers, I imputed a projection that was both league common, 5% worse than league common, or 10% worse than league common. I selected whichever possibility resulted within the lowest error for every system. Hitters with out a Steamer got a league common projection. For the opposite programs, hitters with lacking projections got a projection 10% worse than league common. Pitchers with out a THE BAT or ATC projection got a league common projection. For the opposite programs, lacking pitchers got a projection 10% worse than league common.
Desk 1. Forecast accuracy for full pattern.
Throughout the complete pattern, Steamer was probably the most correct standalone projection system for wOBA in opposition to for pitchers — because it typically is. ZiPS was the clear greatest system for wOBA for hitters. Steamer’s pitching projections have been .0216 wOBA higher than the naive projection. ZiPS’ hitting projections have been .0238 wOBA higher than the naive projection. OOPSY held its personal, rating second of the standalone programs for each pitchers and hitters. Notably, all the programs have been grouped intently for each hitters and pitchers. The composite projection, a mean of the 4 standalone programs, was the runaway winner total for each arms and bats. That is no shock, as averaging the “specialists” is sometimes a profitable method when it comes to forecast accuracy.
Subsequent, let’s check out how the projections did for rookies. Word that for rookies, the naive forecast provides everybody a projection 5% worse than league common.
Desk 2. Forecast accuracy for rookies.
Of the standalone programs, Steamer led the way in which for projecting rookies in 2025, adopted by OOPSY. (OOPSY’s most up-to-date high prospects record, based mostly on long-term projections, might be discovered right here.) Steamer doesn’t publish long-term projections, however I’d like to see them in the event that they ever determine to. As soon as once more, the composite projection is the clear runaway winner total.
Subsequent, we’ll exclude rookies and simply have a look at veterans.
Desk 3. Forecast accuracy for veterans.
For pitchers, the outcomes are just like the complete pattern. For hitters, ZiPS nonetheless finishes first of the standalone programs, however THE BAT X jumps to second place. ATC additionally improves when focusing solely on veterans. THE BAT X was rather more correct for veterans than for rookies, maybe an indication that it’s making good use of Statcast knowledge that’s extra available for main leaguers than it’s for minor leaguers. THE BAT X was additionally best-in-class final yr for hitters total. In my early explorations, it seems that THE BAT X could embrace variables like candy spot and dash velocity that OOPSY doesn’t but embrace.
Lastly, as a robustness test, the desk under appears at OOPSY versus every system, however excludes lacking gamers — the RMSEs under solely concentrate on gamers who had a projection from each programs in every comparability. The outcomes are just like Desk 1.
Desk 4. Forecast accuracy for full pattern, various method.
Total, I might have appreciated to have the most effective forecasts when it comes to accuracy, nevertheless it wasn’t a disappointing debut season for OOPSY. I’m glad to slot in with the opposite FanGraphs projection programs. I’ve had the privilege of speaking with every of those forecasters a bit through the years, and I’ve already realized an excellent deal from them. I proceed to depend on their programs closely. There may be at all times extra to be realized within the pursuit of slight enhancements to forecast accuracy. I nonetheless must look into the accuracy of my element forecasts to see the place the most important enhancements might be made. I already plan to contemplate further Statcast metrics to assist mission hitters, doubtlessly candy spot, dash velocity, assault angle, and horizontal spray tendency. For pitchers, there may be much less low hanging fruit, as my projections there have an extended historical past, however I could tinker with the recency weights and incorporate pitch-level metrics into the projections. I’m not the one one making tweaks every offseason, nevertheless. Regardless of the enhancements that anyone of us makes, customers of projection programs are very more likely to be greatest served by merely averaging the forecasts from the completely different programs.