When an AI model outage hits, treat it as a product incident, not a mystery bug. Confirm the provider is failing (status page + shared error codes), switch on a preplanned degradation mode, tell users what still works in plain language, and keep expensive free paths from burning your bill while the model is sick.
OpenAI, Anthropic, Google, Azure, and the rest will go down or degrade. Early AI products feel it first, because almost every "Generate" click is a live call. This playbook is for founders who ship on EarlyHunt-style early products: how to tell a provider failure from your own bug, how to degrade without looking broken, what to say in-product and by email, when to pause free or expensive calls, and a 14-day prep sprint so the next AI model outage is boring instead of existential.
When an LLM provider outage actually matters
Not every red chart needs a war room. It matters when users cannot finish the job they came for, when paying accounts are blocked mid-flow, or when a free path starts retrying so hard it burns your credits while nothing useful comes back.
It matters less when a background batch job can wait an hour, or when a single slow response is still within your normal p95. The goal is not zero downtime. The goal is: users still trust you after OpenAI is down, or Anthropic is overloaded, for a few hours.
Callquill is an illustrative, made-up product, and every number about it below is made up too. The setup: a solo founder builds a tool that turns sales call recordings into CRM-ready notes and follow-up emails. Production calls one primary chat model. On a Tuesday afternoon the generate button starts returning empty errors for everyone. Support Slack fills with "is it just me?" The founder has no banner, no fallback, and no email ready. That is the failure mode this post exists to prevent.
Detect provider failure vs your own bug
Before you rewrite prompts or restart the app, spend two minutes proving where the fault is.
Check the provider status page first. OpenAI publishes live component status on OpenAI's system status page. Anthropic's status hub redirects to Claude's system status page. Google Cloud and Azure have their own status hubs. If they show an active incident on the exact API you use, stop debugging your code as the root cause.
Read the error code, not only the toast. OpenAI's API error codes guide documents 500 as a server-side failure (retry after a brief wait, check the status page) and 503 with
server_is_overloadedwhen the requested model is temporarily overloaded. Anthropic's HTTP errors docs list 500api_error, 504timeout_error, and 529overloaded_errorfor temporary overload across users. Shared codes across many tenants point at the provider. A single 401 on your key does not.Compare one request outside your app. Hit the same model from a tiny script or the provider playground with a fixed prompt. If that fails too, it is not your React form.
Look for sameness across accounts. If every workspace fails at the same second, and your own /health still returns 200 for non-AI routes, you almost certainly have an LLM provider outage, not a crashed web process. One sentence of provider-status checking on /health is enough; this is not a monitoring deep dive.
Your bug still happens: bad deploy, broken auth header, wrong model name after a rename, a client that truncates the body. Those usually fail for one path, one environment, or one API key. Provider outages fail broadly and match the vendor's status and error vocabulary.
Pick a degradation mode before you need it
Graceful degradation is a product decision you make on a calm day. When OpenAI is down, you should already know which mode flips on.
Mode | What the user can do | When to use it | Trust risk if you skip explaining it |
|---|---|---|---|
Queue and retry | Submit work; get a "we'll finish this" receipt | Short overload; jobs can wait minutes | People mash Generate and think nothing happened |
Cached / last-good results | Read recent outputs; cannot generate new ones | Read-heavy products; prior runs are still useful | Users think the product "forgot" how to generate |
Fallback model | Generate continues on a second provider or smaller model | You have tested quality and cost on that path | Surprise quality drop looks like a silent product change |
Read-only / manual path | Edit, export, copy prior notes; AI buttons disabled with reason | Long outage; no safe fallback | Disabled buttons with no copy feel like a broken app |
Pause free / expensive paths | Paying jobs still try; free or high-cost routes wait | Retries would burn the monthly budget for nothing | Free users feel abandoned if you never say why |
For Callquill (made-up), the default plan is: queue for up to 15 minutes, then flip to read-only with a banner if the primary model is still failing. Fallback to a second provider only for paying seats, and only after a quality spot-check on a fixed set of five sample calls. That rule is a policy choice, not an industry standard.
In-product messaging that does not sound like a crash
Users forgive a model outage. They do not forgive a blank error toast. Write the copy before the incident.
Banner (global): "Our AI provider is having trouble right now. You can still open past notes and edit them. New generation is queued / paused until service recovers. Last update: 3:12pm IST."
Button state: Disable Generate, keep the label honest: "Generate (temporarily unavailable)" rather than a spinner that never ends.
Inline detail: One short paragraph under the button: what still works, what does not, and where to check status.
Do not invent confidence. Avoid "should be fixed soon" unless you have a vendor ETA. "We are watching the provider status page and will turn generation back on when calls succeed again" is enough.
Callquill's made-up banner for a Tuesday outage: "Call summaries are delayed because our model provider is overloaded. Your recording is saved. We will run the summary when the provider recovers, or you can write the CRM note by hand in the meantime."
Status page, email, and the quiet channels
Ship a tiny public status page before your first real outage. If you do not have one yet, use this Hashnode guide to ship a status page before your first real outage (Notion page, subdomain template, or pinned incidents section is enough). Link it from the footer and from the in-app error state.
Email only the people who hit the broken flow, plus anyone who asked. Subject lines that work: "Callquill generation delayed (provider outage)" not "Exciting update." Body: what broke, what you saved, what they should do (wait / retry later / use manual path), and a link to status. One follow-up when you close the incident. Do not send a marketing newsletter dressed as an incident note.
After you recover, a short dated note in your changelog helps silent users who never open email. The same habit as writing a changelog that brings silent users back: short, human, and specific about what changed.
Fallback model rules (so quality does not silently rot)
A second model is not free insurance. It is a product branch with its own failure modes.
Name the tradeoff in the UI when you switch: "Using backup model; summaries may be shorter." Surprise quality drops feel like a bug even when the primary provider is healthy again.
Gate fallback behind a flag you can flip without a deploy. Test the flag on a staging key monthly.
Keep prompts and max tokens conservative on the backup path. Do not retry the expensive primary prompt five times and then also run the backup five times.
Record which model served each job in your own logs. You will need that for the postmortem and for any user who asks why Tuesday's notes look different.
OpenAI's rate limit guide separates slow_down (429) from server_is_overloaded (503): ramp-rate and overload are different animals. Your fallback should not fire on a single 429 with a Retry-After of a few seconds. Honor Retry-After when present, use exponential backoff with jitter when it is missing, and only escalate to fallback after sustained failure or a confirmed status incident. Anthropic's docs note the official SDK retries transient failures (connection errors, rate limits, 5xx) with exponential backoff by default. That is helpful, but retries without a product-level circuit breaker can still stampede your bill during a long outage.
Pause expensive free paths without looking hostile
During an AI model outage, free tiers and "try it" buttons are often the noisiest and least valuable traffic. Pausing them is a cost and fairness choice, not a punishment.
Keep saved work for free users. Do not delete drafts because generation is paused.
Say the pause is temporary and tied to the provider incident, not a plan change.
Prefer queue-with-email over silent drop. "We'll email you when your free run completes" beats a dead button with no text.
If paying users still get a fallback model, say so. Hidden preferential treatment that leaks on Twitter hurts more than an honest line in the banner.
In the Callquill example (made-up), free accounts pause new summaries after three consecutive provider 503s in five minutes. Paying accounts keep the queue. The founder posts that rule on the status page so nobody has to guess.
Measure impact while n is still small
You do not need a full observability suite. You need a few incident numbers you can write in a note afterward.
First failed AI call timestamp vs first public status note (goal under 15 minutes while you are awake; rule of thumb, not a SLA).
Share of generate clicks that ended in error vs queued vs succeeded on fallback.
Support threads that say "is it just me?" vs threads that already saw your banner.
Paying seats who churn or ask for refunds in the 72 hours after the outage (count stories, not only percentages, while you have few customers).
If you are still choosing which product metrics to track before 100 users, fold these into the light sheet from tracking the product metrics that matter before your first 100 users. During an outage, activation funnels lie; incident counts tell the truth.
A 14-day prep sprint (before the next outage)
Do this once on a quiet week. Then rehearse the banner copy every month.
Day 1-2: List every user-facing action that calls a model. Mark each as must-work, can-queue, or can-pause.
Day 3: Write banner, button, and email copy for three states: investigating, degraded (fallback on), recovered. Store them in a doc your phone can open.
Day 4: Add a feature flag for "AI degraded" that disables or queues generate without a deploy.
Day 5: Wire a single health check that records provider error rates (or a canary prompt every few minutes). Alert yourself; do not build a full on-call stack yet.
Day 6-7: Stand up the tiny status page and link it from footer + error state.
Day 8-9: If you want a fallback model, run five fixed sample jobs on it and write pass/fail notes. No fallback ships untested.
Day 10: Decide free-path pause rules and document them in the status template.
Day 11: Dry-run: flip the flag on staging, send yourself the email, update status, close the incident.
Day 12-13: Add a one-page postmortem template: timeline, customer impact, what you told users, what you will change in 7 days.
Day 14: Put provider status URLs, runbook link, and copy doc in the same bookmark folder. Share with anyone who has admin access.
Traps that make an AI outage worse
Silent retries in a tight loop. You turn a 20-minute provider blip into a maxed credit card. Cap retries, honor
Retry-After, and stop when the status page confirms an incident.Blaming the user. "Try a shorter prompt" while the whole API is 503 teaches people the product is flaky.
Fake "all systems normal." Partial outages deserve a degraded state. Lying green trains people to ignore you.
Shipping fallback without saying so. Quality cliffs look like regressions.
Writing a root-cause essay mid-fire. Two honest sentences beat a speculative RCA while calls are still failing.
Only posting after someone screenshots Twitter. If the first update is an hour late, the status page already failed.
Treating provider uptime as your SLA marketing. Your commitment is how you behave when the model layer fails, not a number you invent.
FAQ
Should I refund everyone when OpenAI is down?
Not automatically. Refund or credit when a paying customer lost a time-sensitive job and your degradation path could not help (no queue completion, no manual path, long silence). For short incidents where work was queued and finished, a clear status note usually beats a blanket refund. Decide the rule before the outage so you are not negotiating under stress.
Is a second provider worth it before product-market fit?
Only if you have already felt a real outage or your paying users cannot wait. Dual-provider adds key management, prompt drift, and billing complexity. Many early teams do better with queue + honest read-only mode first, then add fallback after the first painful incident. Label that as a sequencing choice, not a universal rule.
How long should I wait before posting publicly?
As soon as you know users are blocked and you have confirmed it is not a single-account bug. Under about 15 minutes while you are awake is a useful personal target. Waiting for a perfect RCA helps nobody staring at a spinner.
Build the habit before the next model blip
AI model outages are normal. Losing early users over silence is optional. Detect provider failure fast, degrade on purpose, speak plainly, pause the paths that only burn money, and write the five-line postmortem while the timeline is fresh.
If you are still shipping the first version of an AI product and want early adopters who expect rough edges handled honestly, you can launch your AI product on EarlyHunt when you are ready. Ship the outage runbook before you need it.