All work

Forecasting system

Pitch sequencing

A system that follows a live baseball game and predicts the next pitch: its type, its outcome, and roughly where it crosses the plate.

Built with
PyTorch, LightGBM, FastAPI, Tauri and React, Google Cloud Storage
AI tooling
  • Google
  • Open source
  • Anthropic
Status
In use. Code is private.
Illustration: a strike zone with a shaded estimate of where the next pitch will cross it.

The pain

Predicting what happens next, live, from millions of past events, and knowing honestly how good the prediction is. The business version is forecasting.

Pitchers mix their pitches on purpose so hitters cannot guess, which puts a ceiling on how predictable they are. Gains are measured in single points.

Some pitch types are rare, the tracking technology and the plate-location measurement both changed over the period, and every input fed to the model during a live game has to match what it saw in training.

It is easy for information from the future to leak into training data and inflate the results.

What was built

  • Data

    Pitch-by-pitch tracking data for 2015 to 2026, about 8.5 million pitches, with weekly profiles for pitchers, batters and catchers.

  • Models

    Gradient-boosted baselines first, then a multi-task transformer that predicts pitch type, outcome, location, run value and more: 11 outputs in the current version.

  • A finding worth telling

    A simple lookup table keyed on pitcher, count, previous pitch and batter handedness beat the transformer, 39.9% to 37.1% on an early-2026 test set. Production blends the model with the lookup, and now uses a lookup that refreshes every day.

  • A model of the whole game

    A 26-million-parameter transformer over every event in a game, paired with a deterministic rules engine, trained on my own GPU machine. It is the most accurate model so far and is not serving live yet.

  • Product

    A desktop app with a live view and a scouting view for a pitcher’s arsenal. A browser version adds a broadcast assistant that suggests stat angles for the current situation: a language model only selects and phrases them, and any percentage or speed that is not in the source facts is removed.

Results

  • 46.50%

    top-1 for the newest model, which is not serving live yet, against 43.34% for the live system: a lead of 3.16 points

    2026 regular season, 710,006 pitches

  • +1.00

    points of top-1 for the current production model over its predecessor, with calibration error down from 2.88 to 1.44

    Live replay of 100 games, 29,600 pitches

  • 99.992%

    of 9.43 million recorded game transitions reproduced by the rules engine

    Full history

  • 98.2%

    agreement between live and offline predictions on the model’s top pick

    The same 29,600 pitches

How it was built

  • Every production change can be undone with one setting, with checksummed backups.
  • Model comparisons use paired, game-level bootstrap confidence intervals.
  • An independent leak audit rebuilt the inputs live-style at 14 cut dates and found no differences.
  • The daily lookup refresh is tested to prove it never sees the game it is predicting.
  • Two data bugs were found, written up openly and fixed before the current model replaced its predecessor.

What it means for a business

The same discipline applies to any forecast a business depends on: test simple baselines first, measure honestly, guard against data leaks, and make every change reversible.

Tell me what’s painful in your organization.

I’ll tell you plainly whether AI can fix it and what it would take.

Work with me
Portrait of Alek Lichucki