Skip to main content

Vertex AI

The vertex-ai provider supports Google Vertex AI services, including Gemini and Anthropic models.

tip

As of BAML 0.85.0, vertex-ai supports Anthropic models.

Quick Start (Express Mode)​

client<llm> MyClient {
    provider vertex-ai
    options {
        model gemini-2.5-pro
        location us-central1
        project_id my-project-id
        query_params {
            key env.VERTEX_API_KEY
        }
    }
}

Authentication​

Vertex API Key (Express Mode)​

The quickest way to get started:

client<llm> VertexApiKeyClient {
    provider vertex-ai
    options {
        model gemini-2.5-pro
        location us-central1
        project_id my-project-id
        query_params {
            key env.VERTEX_API_KEY
        }
    }
}

See Google's Express Mode guide for setup.

Google Application Credentials​

For service account authentication:

client<llm> MyClient {
    provider vertex-ai
    options {
        model gemini-2.5-pro
        location us-central1
        project_id my-project-id
        credentials env.MY_APPLICATION_CREDENTIALS_CONTENT
    }
}

BAML tries authentication in this order:

  1. GOOGLE_APPLICATION_CREDENTIALS environment variable
  2. gcloud auth application-default login credentials
  3. GCP metadata server (when running in GCP)
  4. gcloud auth print-access-token command

BAML-Specific Options​

OptionTypeRequiredDefaultDescription
modelstringYes-Model identifier
locationstringYes-GCP region
project_idstringNoFrom credentialsGCP project ID
base_urlstringNoAuto-generatedCustom API URL
credentialsstring/objectNoAuto-detectedService account credentials
headersobjectNo-Additional headers
query_paramsobjectNo-Query parameters

Available Models​

ModelInputOptimized For
gemini-2.5-proAudio, images, video, textComplex reasoning, code generation
gemini-2.5-flashAudio, images, video, textFast, versatile performance
gemini-1.0-proTextMulti-turn chat, code generation

Configuration Examples​

With Safety Settings​

client<llm> MyClient {
    provider vertex-ai
    options {
        model gemini-2.5-pro
        project_id my-project-id
        location us-central1
        safetySettings {
            category HARM_CATEGORY_HATE_SPEECH
            threshold BLOCK_LOW_AND_ABOVE
            method SEVERITY
        }
    }
}

With Generation Config​

client<llm> MyClient {
    provider vertex-ai
    options {
        model gemini-2.5-pro
        project_id my-project-id
        location us-central1
        generationConfig {
            maxOutputTokens 100
            temperature 1
        }
    }
}

Anthropic Models on Vertex​

client<llm> VertexClaudeSonnet {
    provider vertex-ai
    options {
        model "claude-sonnet-4"
        anthropic_version "${ANTHROPIC_VERSION}"
        base_url "https://${LOCATION}-aiplatform.googleapis.com/v1/projects/${PROJECT_ID}/locations/${LOCATION}/publishers/anthropic/models"
    }
}

Troubleshooting​

Model not found errors

Try setting location: global:

client<llm> MyClient {
    provider vertex-ai
    options {
        model gemini-2.5-pro
        location global
        project_id my-project-id
    }
}

Some models are only available in specific regions or through the global endpoint.

Authentication issues

Set BAML_INTERNAL_LOG=debug to see detailed authentication logs.

For local development:

gcloud auth application-default login

Set GOOGLE_CLOUD_PROJECT in the playground "API Keys" dialog.

See Also​