Skip to main content

Ollama

Ollama supports the OpenAI-compatible API, allowing you to use local models.

tip

Use Ollama's OpenAI-compatible /v1 endpoint. See Ollama's OpenAI compatibility documentation.

Basic Example​

client<llm> MyClient {
    provider "openai-generic"
    options {
        base_url "http://localhost:11434/v1"
        model llama3
    }
}

BAML-Specific Options​

OptionTypeDefaultDescription
base_urlstringhttp://localhost:11434/v1Ollama API endpoint
modelstring-Model name
headersobject-Additional headers
note

Include /v1 at the end of the URL for OpenAI compatibility.

Custom Headers​

client<llm> MyClient {
    provider "openai-generic"
    options {
        base_url "http://localhost:11434/v1"
        model "llama3"
        headers {
            "X-My-Header" "my-value"
        }
    }
}

Available Models​

ModelDescription
llama4Meta Llama 4: Latest with enhanced reasoning
llama3.3Meta Llama 3.3: Improved performance
llama3Meta Llama 3: Highly capable open LLM
qwen2Alibaba's large language models
phi3Microsoft's lightweight models (3B/14B)
mistralMistral AI 7B model v0.3
gemmaGoogle DeepMind lightweight models
mixtralMistral MoE models (8x7b, 8x22b)

See Ollama Model Library for the complete list.

tip

Use specific versions: "mixtral:8x22b"

PromptFiddle Integration​

Try BAML with Ollama at promptfiddle.com:

OLLAMA_ORIGINS='*' ollama serve

Learn more in the Ollama structured output guide.

See Also​