Skip to main content

Client Configuration Reference

This reference covers all options for configuring LLM clients in BAML.

Basic Structure​

client<llm> ClientName {
  provider ProviderName
  model ModelName
  options {
    // Provider-specific options
  }
}

Supported Providers​

ProviderValueModels
OpenAIopenaigpt-4, gpt-3.5-turbo, etc.
Anthropicanthropicclaude-3-opus, claude-3-sonnet, etc.
Googlegoogle-aigemini-pro, etc.
Azure OpenAIazure-openaiDeployed model names

Common Options​

client<llm> MyClient {
  provider openai
  model gpt-4
  options {
    api_key env.OPENAI_API_KEY
    temperature 0.7
    max_tokens 1000
    timeout_ms 30000
  }
}

Environment Variables​

Use env.VARIABLE_NAME to reference environment variables:

client<llm> MyClient {
  provider env.LLM_PROVIDER
  model env.LLM_MODEL
  options {
    api_key env.LLM_API_KEY
  }
}

Fallback Configuration​

client<llm> ReliableClient {
  provider fallback
  options {
    clients [PrimaryClient, BackupClient]
    retry_policy {
      max_retries 3
      backoff_ms 1000
    }
  }
}

See How to Add Fallbacks for usage examples.