Model deprecation: why pinning an LLM snapshot sets a deadline

On December 11, 2026, OpenAI is scheduled to shut down gpt-5-2025-08-07 and o3-2025-04-16 in its API. Those are pinned snapshots, the exact model IDs careful teams chose so their products would behave the same every day.

That's the trade pinning makes. You get a fixed version, and the vendor keeps control of its retirement date.

Pinning buys stability and starts a clock

A dated snapshot locks a model version so its behavior is more reproducible. OpenAI's model documentation says snapshots let you lock in a specific version so performance and behavior stay consistent.

An alias doesn't offer the same stability. Google's -latest aliases, for example, point to the newest release and can move to a different model over time.

So teams pin, which is the right call. But a pin isn't permanent. OpenAI notified developers on June 11, 2026 that older GPT-5 and o3 snapshots would be shut down six months later.

After that date, the pinned ID stops working. The stability you chose turns into a deadline.

Notice periods differ by vendor

How much warning you get depends on the vendor and the kind of model. Each vendor publishes its own minimum.

OpenAI states at least 6 months for generally available models and at least 3 months for specialized variants. Preview models may be retired with much shorter notice, such as 2 weeks.

Anthropic gives at least 60 days before retiring a publicly released model.

Google's Gemini API gives at least 2 weeks for preview models, plus a 2-week email notice before the model behind a -latest alias changes.

If you route across providers, your real window is the shortest notice among the models you actually use.

A pinned ID doesn't guarantee frozen behavior

Pinning reduces drift, but it doesn't remove it entirely. Anthropic's versioning docs say a dated model ID always refers to the same model. They also note that changes to serving infrastructure can still cause minor differences in observable behavior.

Drift between versions can be large. In a 2023 study, researchers at Stanford and UC Berkeley found GPT-4's accuracy at identifying prime numbers fell from 84.0% in March 2023 to 51.1% in June 2023. The service was still called GPT-4. Its behavior had changed.

That's an old measurement of an old model. The lesson still holds: a prompt tuned against one version is a guess about the next one until you test it.

The test harness can be retired too

Testing is where this gets harder. OpenAI announced on June 3, 2026 that its hosted Evals platform is being deprecated.

Existing evals become read-only on October 31, 2026. The Evals dashboard and API shut down on November 30, 2026. The transition also covers the graders documented for eval workflows.

These are separate deprecations. But put the dates next to each other. A team that pinned gpt-5-2025-08-07 and built its regression suite on OpenAI's hosted Evals can't add new evals after October 31, loses the harness on November 30, and loses the model on December 11.

It isn't only evals. Agent Builder and reusable prompt objects were deprecated the same day, and both shut down November 30, 2026. The Assistants API was already shut down on August 26, 2026.

Vendor-hosted tooling runs on the vendor's product roadmap, not your release schedule.

OpenAI's migration path points to a company it's acquiring

OpenAI's deprecation page links a migration guide titled Moving from OpenAI Evals to Promptfoo. On March 9, 2026, OpenAI announced it would acquire Promptfoo, subject to customary closing conditions.

Both companies say the open-source Promptfoo CLI and library will continue. Promptfoo's own announcement also says it will keep supporting a wide range of providers and models.

So this isn't about what the tool can do. It's about independence. The recommended way off a retired vendor tool now leads to a project that same vendor has agreed to buy.

If you want your eval layer to stay independent of any one model vendor, that's worth weighing.

Keep your evals, routing, and prompts in your own code

You can't avoid migrations. What you can control is whether the things you need to test a replacement belong to you. That comes down to three decisions.

Keep eval cases in your own repository, and run them with a harness that isn't tied to one model vendor. Open options include Inspect, created by the UK AI Security Institute, and EleutherAI's lm-evaluation-harness.

The suite doesn't need to be big. It needs to test the behavior your product depends on.

For most AI features, that means three kinds of cases. Does the output parse against the schema your code expects? Does the model decline requests outside the feature's scope? Does it get the answers right on inputs where you already know the correct result?

Store the cases as files in the same repository as the code. Run them on every prompt change, not only when the model changes.

Then save the results from the model you're pinned to today. A replacement can only be judged against a baseline, and the baseline is easiest to capture while the current model still works.

Put provider routing behind a layer you control. A self-hosted gateway such as LiteLLM, or a thin internal interface, keeps provider details out of application code and makes it easier to run one eval suite against candidate models.

It won't hide every difference. Tool calling, structured output, and reasoning parameters still vary by provider.

In practice, this is a short chain:

application code
    -> internal model interface
        -> provider adapter (OpenAI, Anthropic, Google)

Application code calls the interface and never names a model. The model ID lives in deployment config. Each adapter translates the interface into one provider's API.

With that in place, testing a candidate is a config change in a test environment. The same eval cases run against the candidate, and the results sit next to the baseline.

Promotion is also a config change, and so is rollback. Keep the old ID available until its shutdown date, so you have somewhere to go back to if production shows something the evals missed.

Keep prompts in version control next to the code that depends on them. OpenAI's own migration guidance for reusable prompts says to move prompt content into application code. That's where it should have been from the start.

The December 11 calendar runs backward

For a team pinned to gpt-5-2025-08-07 with evals on OpenAI's hosted platform, every step depends on the one before it.

Before October 31, copy eval cases and grading logic into your own repository. After that date the hosted evals are read-only, so this is the last point where they can still be changed in place.

Before November 30, run the moved suite against the current model and compare it with the hosted results. If they match, the move didn't change what you're measuring. After that date there's nothing left to compare against.

Before December 11, run the same suite against one or two candidates, promote the one that holds the baseline, and keep a rollback path. A team that starts after the harness is gone has eleven days to do all of it.

The through-line

A pinned model is a dependency with an expiration date. The tools for testing its replacement can expire too, sometimes a few days earlier.

Pin the model. Keep everything you'd need to replace it.

Written by Synthetixis, an AI-native product studio. More on what most AI software gets wrong.