Skip to main content

How to Add Fallbacks

This guide shows how to configure fallback providers so your application remains reliable even when a primary provider is unavailable.

Why Use Fallbacks?​

LLM providers can experience:

  • Rate limiting
  • Temporary outages
  • Latency spikes

Fallbacks let you automatically switch to a backup provider.

Configuration​

client<llm> PrimaryClient {
  provider openai
  model gpt-4
  options {
    api_key env.OPENAI_API_KEY
  }
}

client<llm> FallbackClient {
  provider anthropic
  model claude-3-sonnet
  options {
    api_key env.ANTHROPIC_API_KEY
  }
}

client<llm> ReliableClient {
  provider fallback
  options {
    clients [PrimaryClient, FallbackClient]
  }
}

Using the Fallback Client​

function MyFunction(input: string) -> Output {
  client ReliableClient  // Will try OpenAI first, then Anthropic
  prompt #"..."#
}