The Simplest AI Fallback Pattern You're Not Using

Your AI provider will go down. Not "might" — will. OpenAI had 4 major outages in 2025. If your app does nothing when that happens, your users get errors and you get angry emails.

The fix is dead simple. Here's the pattern.

The Problem

Most apps call one AI provider:

response = openai_client.chat(model: "gpt-4o-mini", messages: messages)

When OpenAI returns a 500, your app returns a 500. When OpenAI times out, your app times out. Your availability is now OpenAI's availability.

Congratulations, you've outsourced your uptime.

The Pattern: Adapter + Automatic Fallback

The idea is simple:

1. One entry point for all LLM calls — regardless of provider
2. Route to the right provider based on the model name
3. On transient failure, automatically retry with a different provider on different infrastructure

Here's the shape of it:

def call
  primary_provider.call(**params)
rescue RetryableError => e
  log_warning("Primary failed: #{e.message}, falling back")
  fallback_provider.call(**params.merge(model: FALLBACK_MODEL))
end

def primary_provider
  case @model
  when /\Agemini/   then google_provider
  when /\Adeepseek/ then deepseek_provider
  else                   openai_provider
  end
end

That's the whole pattern. A case statement and a rescue block. Let's break down why each decision matters.

Model Name = Routing Key

No config files. No provider registry. No dependency injection framework. The model name is the routing decision.

Pass "gpt-4o-mini" → OpenAI. Pass "gemini-2.0-flash" → Google. Pass "deepseek-chat" → DeepSeek.

Want to add Anthropic tomorrow? One more when clause. That's it.

This works because model names are already namespaced by convention. OpenAI models start with gpt- or o1-. Gemini models start with gemini-. DeepSeek models start with deepseek-. There's no ambiguity.

Unified Interface

Every provider must implement the same contract:

Input:  system_prompt, context, tools, model, temperature, max_tokens
Output: { tool_arguments, input_tokens, output_tokens, model_used, latency_ms }

Same in, same out. The caller doesn't know which provider ran. This is why the fallback is a one-liner — the interfaces are identical.

Building a new provider adapter? Implement the contract, handle the API-specific quirks (OpenAI and DeepSeek use the same format, Gemini needs translation), normalize the output. Done.

Not All Failures Deserve a Fallback

We don't catch all errors. We catch RetryableError — a specific class for transient failures:

• 500, 502, 503 — the provider is having a bad day
• Timeout — network issues, slow response
• 429 — rate limited (the fallback provider has its own quota)

Things we do NOT retry:

• 401 — bad API key. Fallback won't fix this.
• 400 — malformed request. Your code is broken, not the provider.
• Schema errors — your tool definitions are wrong. Fix them.

Only fall back on "the infrastructure is broken" errors. Everything else should fail fast and loud so you can fix the actual bug.

Different Infrastructure, Not Different Model

This is the most important decision: the fallback provider must be on different infrastructure.

We don't fall back to another OpenAI model. If OpenAI is returning 500s, sending the same request to a different OpenAI model will also return a 500. The whole point is independent failure domains.

Our fallback is DeepSeek because:

• Different cloud, different region — when OpenAI goes down, DeepSeek is (usually) fine
• Cheap — fallback traffic doesn't blow your budget
• Compatible — supports function calling with the same tool schema
• Good enough — slightly different output quality, but a slightly worse response is infinitely better than no response

What About Quality Differences?

Yes, providers produce different outputs. The fallback might be slightly less accurate. But here's the thing:

When your primary is down, your choices are:

1. Return an error to the user
2. Return a slightly-different-but-still-useful response

Option 2 wins every time. Your users don't care which model answered. They care that they got an answer.

For critical paths, we add a second layer of resilience on top of the adapter. Our message router, for example, fails open if both providers are down — it routes to all agents and lets each one decide. Three layers:

1. Primary provider (OpenAI)
   ↓ fails
2. Fallback provider (DeepSeek)
   ↓ fails
3. Fail-open default (route to everyone)

A customer message would need all three layers to fail simultaneously. Not impossible, but extremely unlikely.

Common Objections

"The fallback adds latency."

Only when the primary fails. Happy path = zero overhead. The fallback adds ~1-2s only during outages. During normal operation, it's a no-op.

"What if both providers are down?"

That's what the application-level fallback is for (fail-open, safe defaults, etc.). The adapter handles provider failures. The caller handles adapter failures. Each layer has its own strategy.

"Should I retry the primary first?"

No. If OpenAI returned a 500, sending the same request 100ms later will probably also return a 500. Switch providers immediately. Fast fallback > slow retry.

"What about streaming?"

This pattern works great for non-streaming calls (structured tool output is our main use case). For streaming, you'd need to detect failure mid-stream and restart with the fallback. Same concept, more plumbing.

The Pattern, Generalized

You don't need Rails or Ruby for this. The pattern is language-agnostic:

1. Define a uniform interface for all providers
2. Route to primary based on model name
3. Catch transient errors only (not all errors)
4. Fallback to a provider on different infrastructure
5. Let the caller handle total failure with its own strategy

No fancy libraries. No retry frameworks. No circuit breakers (though you could add one). Just a conditional and a try/catch.

The best infrastructure code is the code that's too simple to break.

This Powers WhatsRB Cloud

Every LLM call in WhatsRB Cloud — from message routing to agent execution — goes through this adapter. When OpenAI hiccups, your WhatsApp agents keep running. No configuration needed.

We built this because our users process customer messages on WhatsApp. A dropped message isn't a minor inconvenience — it's a frustrated customer who thinks they're being ignored. The fallback pattern means that doesn't happen.

Get started → | API docs →

Built with Ruby, Rails, and the assumption that everything will eventually fail.