Your Prompt Tooling Has a Deadline Your Monitoring Can't See

Picture a support-triage feature that's run fine for eight months. Nobody has touched it. Then one Tuesday it starts throwing errors, and the on-call engineer discovers the failing call isn't to a model at all. It's to a small helper endpoint the team used once, during setup, to tidy up a prompt. The vendor announced its removal in a release note back in July. Nobody on the team read release notes in July.

That's the shape of the risk this quarter. Two AI vendors are switching off prompt-management and evaluation tooling, their shutdown dates 15 weeks apart, and the nearer one is seven days from the date of this article.

So we went looking for something specific. If you aren't reading release notes, does anything else tell you? Your logs, your dashboards, your API gateway. Do they warn you?

We checked. They don't.

What's actually being switched off#

Five things, from two vendors, on two dates. All of it read straight from the vendors' own pages rather than from summaries.

Two rows deserve more attention than they usually get.

The first is the Anthropic row about saved work. An updated Workbench exists, so it's tempting to read this as "the tool moved." It didn't. Anthropic's note says saved prompts, variables, and evals aren't supported in the updated Workbench, and points you at an export option in the banner and under Organizational Settings. If your prompt library lives there and nobody exports it, it's gone on 17 August.

The second is the OpenAI Evals row, which carries two dates instead of one. Evals go read-only on 31 October, a full month before the shutdown. The first thing your team notices, then, won't be an outage. It'll be a release gate that quietly stops recording new results while continuing to look healthy.

What we ran#

Two things, both in a throwaway sandbox.

First we called the live APIs, read-only. We created no prompts, no evals, no agents, and spent nothing on model tokens. The question was simple: what does an ordinary API key see today?

Then we wrote a scanner that reads a codebase and reports every reference to the retiring surfaces, with the deadline and the days remaining attached to each hit. We ran it against a small test repo built for the purpose: four files with problems, one clean file as a control.

What actually happened#

Nothing warns you. That's the finding that matters. A long-standing web standard, RFC 8594, defines a Sunset response header for exactly this situation — a server telling clients "this endpoint has an expiry date." Across every request we made, to both vendors, on endpoints both companies have publicly scheduled for removal, not one response carried a Sunset header. Not one carried a Deprecation header either.

Put plainly: your API gateway can't flag this. Your monitoring can't alert on it. A dashboard tracking third-party API health shows green right up until the day the endpoint stops answering. The deadline exists in exactly one place, a documentation page somebody has to remember to open.

One endpoint is already gone on our account. OpenAI's reusable-prompts REST path returned a 404 when we called it. Interpreting that took care, because a 404 can mean two different things, so we also tested a path we knew was live-but-wrong and a path we knew had never existed. OpenAI answers those differently. A real endpoint with a bad ID gives you a descriptive error explaining what it couldn't find; a path that doesn't exist gives you an empty response. The prompts path gave us the empty one, on both the list and the single-item route.

So on the key we used, that surface is unreachable 112 days before its published shutdown date. We're deliberately not claiming OpenAI removed it early for everyone. It could be account-tier gating. It could be a staged rollout, or a surface only ever offered through the SDK rather than as a public REST collection. We can't tell from outside and we're not going to guess. What we can say is that the published date and what our account actually returns don't match, which is a reason to check your own account rather than trust the calendar.

The Evals API, meanwhile, is in perfect health. It answered normally, with no warning of any kind, 112 days before it shuts down.

The scanner found 11 references in the test repo, sorted so the most urgent came first. The two Anthropic hits landed at the top with "7 days left" beside them. The clean control file was correctly left alone. Run in gate mode, the scanner failed the build whenever a deadline fell inside 30 days.

One honest note on accuracy. Of those 11 findings, one was wrong. The scanner flagged a line reading def poll_run(eval_id, run_id) because it matched on eval_id, which there is just a function's parameter name, not a reference to the vendor's service. Call it 9% noise. We left it in rather than tuning it away, because the narrower rule that would suppress it also misses genuine eval_id entries in config files. Treat the output as a triage list, not a defect count.

Can this survive your workflow?#

The question worth asking isn't "do we use OpenAI's prompt tooling." It's "would we know if we did." Some concrete cases where the answer is usually no:

• Support ticket triage and routing. Built once, works well, sits untouched for a year. Also the most likely place for a stored prompt ID rather than inline prompt text.
• Anything with a quality gate before deploy. If a release check writes to OpenAI Evals, the 31 October read-only date turns your gate into decoration a month before anyone sees an error.
• Internal automations built by someone who has since moved teams. Agent Builder workflows are the classic case. The person who exported the flow isn't the person who owns the service now.
• Prompt libraries kept in a vendor console. The Anthropic Workbench case. Not code, so not in your repo, so no scan finds it. Somebody has to log in and export by 17 August.
• Billing and CRM writes driven by an agent. Lower volume, higher blast radius. A silent failure there costs more than a failed chat reply.

Time cost, honestly stated: scanning your repos takes minutes. The migration work behind a hit varies enormously. Moving a stored prompt back into your own codebase is small. Rebuilding an Agent Builder workflow on a different SDK is not, and that one has 112 days on it, which sounds comfortable until you push it through a normal planning cycle.

What to do differently this week#

Three things, in this order, because the deadlines aren't equally close.

1. Today, if you use Anthropic's console: log in to the legacy Workbench and export saved prompts, variables, and evals. There are seven days left and nothing in your codebase will remind you. This is a browser task, not an engineering task.
2. This week: grep your repositories for pmpt_, eval_id, v1/prompts, v1/evals, agent_builder, and v1/experimental/. Our scanner does exactly this with the dates attached, but the point is the answer, not the tool.
3. Before 31 October: if any release gate writes to OpenAI Evals, move it or accept that it stops recording results a month before it starts failing loudly.

If the scan comes back empty, stop. That's a legitimate result and it's worth ten minutes to establish.

When to use this, and when to skip it#

Worth doing if: you have production code calling either vendor that predates mid-2026; you have a release gate wired to a vendor eval product; anyone on your team built prompts in a vendor console; or you can't answer "which of our services use stored prompt objects" from memory.

Skip it if: you already pass prompt text inline from your own repository, keep evaluations in your own harness, and have never opened either vendor's visual builder. That's a real and increasingly common setup. If it describes you, none of the five deadlines apply. Don't run a migration project to discover you had nothing to migrate.

Skip the scanner specifically if your prompt IDs live in a database or a secrets manager rather than in files. It reads source files. It'll report a clean repo and be wrong.

What this doesn't cover#

Worth being blunt about the limits, because they're what make the rest usable.

We could not verify the current state of Anthropic's three experimental endpoints. We tried without credentials and got a 404 back from all three. Then we sent the same request to an endpoint name we invented, and got a byte-identical 404. Anthropic returns the same response to unauthenticated callers whether the route exists or not, so the probe proved nothing. We're reporting it as inconclusive rather than dressing it up.

The scanner is regex over lines. A reference split across two lines, or a prompt ID assembled from string fragments at runtime, gets missed. The ID patterns assume the current pmpt_ and eval_ prefixes. And the deadlines live in the scanner's own rule table, so if a vendor moves a date, somebody updates the table by hand.

None of this measures whether the suggested replacements are any good. OpenAI points Evals users at Promptfoo and Agent Builder users at the Agents SDK. Whether those are equivalent for your use case is a separate question we didn't test.

What Effloow added#

The vendor pages tell you what's being removed and when. What they don't tell you is whether anything will warn you, or how to find your own exposure.

Our contribution is three things the primary sources don't contain. A tested answer to whether these endpoints signal their own expiry at runtime: no, on both vendors, with no Sunset or Deprecation header anywhere. A recorded observation that OpenAI's reusable-prompts path is already unreachable on a live account 112 days before its published date, with the control tests that separate "gone" from "wrong ID." And a runnable, dated scanner published with its real output, false positive included.

The full command log, the raw probe responses, the control tests, and the scanner's complete output are in the public lab note.

If deprecation-tracking as an ongoing discipline is the real problem, our audit of OpenAI spend-limit fail-closed behavior covers the adjacent case where a vendor limit surfaces as an error your client mishandles, and our Assistants API sunset port walks a comparable migration end to end.

Want this audited across your services and turned into a dated remediation plan rather than a list of grep hits? That's the work Effloow's Proof Studio packages, and you can start a conversation through our services page.

For your engineers#

Everything below is method. It's separated deliberately so the section above stays readable.

Environment. macOS, Python 3 standard library only (urllib.request), no vendor SDKs. Credential read from .env as OPENAI_API_KEY, never logged. Anthropic requests sent unauthenticated; Effloow holds no Anthropic key. All OpenAI requests were GET. No objects created, no model tokens consumed. Sandbox at /tmp, disposable.

Probe results, 2026-08-10:

Deprecation and Sunset headers: absent on every row above, and on all three unauthenticated Anthropic probes.

Anthropic route-existence probe, and why it failed. POST to /v1/experimental/generate_prompt, /improve_prompt, and /templatize_prompt, unauthenticated, empty JSON body. All three returned 404 {"type":"error","error":{"type":"not_found_error","message":"Not found"}}. The control, /v1/experimental/does_not_exist_effloow, returned the identical body. An unauthenticated 404 carries no signal here. Determining the real state needs a key with Console access.

Scanner design. Five dated rules, each carrying its own deadline, severity, remediation string, and the vendor URL that sets the date. Findings sort by days-remaining ascending, so the nearest deadline is always first. An extension allowlist skips binaries; .git, node_modules, vendor, dist, build, and .venv are excluded. Two output modes, human-readable and --json. A --fail-within-days N flag returns exit code 1 when any finding falls inside the window, which is what makes it usable as a CI gate. A --today override keeps the tool deterministic in tests instead of dependent on the wall clock.

Fixture and run. Five files: a TypeScript module using prompt: {id: "pmpt_..."} plus a raw v1/prompts fetch, a Python Evals client, a Python module calling two Anthropic experimental endpoints, a YAML pipeline config carrying prompt_id, eval_id, and an Agent Builder export reference, and one clean control passing an inline prompt. Result: 11 findings, control file not flagged, both Anthropic hits ranked first at 7 days remaining. --fail-within-days 30 exited 1.

Reproduce. Full source for the probes, the scanner, and the fixture, plus raw responses, are in the lab note. Re-run against your own repository root, and pass --today explicitly if you want reproducible output.

Primary sources, read directly:

• OpenAI deprecations: https://developers.openai.com/api/docs/deprecations
• Claude Platform API release notes, 17 July 2026 entry: https://platform.claude.com/docs/en/release-notes/api
• Claude Console legacy Workbench: https://platform.claude.com/workbench
• Updated Claude Workbench: https://platform.claude.com/playground

Vendor-stated dates are theirs. The probe results, the control methodology, the scanner, and its error rate are ours.