Anthropic Vertex AI Object Structure

LibreChat supports running Anthropic Claude models through Google Cloud Vertex AI. This allows you to use Claude models with your existing Google Cloud infrastructure, billing, and credentials.

For quick setup using environment variables, see the Anthropic configuration guide

Benefits

• Unified Billing: Use your existing Google Cloud billing account
• Enterprise Features: Access Google Cloud's enterprise security and compliance features
• Regional Compliance: Deploy in specific regions to meet data residency requirements
• Existing Infrastructure: Leverage your current GCP service accounts and IAM policies

Prerequisites

Before configuring Anthropic Vertex AI, ensure you have:

1. Google Cloud Project with the Vertex AI API enabled
2. Service Account with the Vertex AI User role (roles/aiplatform.user)
3. Claude models enabled in your Vertex AI Model Garden
4. Service Account Key (JSON file) downloaded and accessible to LibreChat

Example Configuration

endpoints:
  anthropic:
    streamRate: 20
    titleModel: "claude-3.5-haiku"  # Use the visible model name (key from models config)

    vertex:
      region: "global"
      # serviceKeyFile: "/path/to/service-account.json"  # Optional, defaults to api/data/auth.json
      # projectId: "${VERTEX_PROJECT_ID}"  # Optional, auto-detected from service key

      # Model mapping: visible name -> Vertex AI deployment name
      models:
        claude-fable-5-1:
          deploymentName: claude-fable-5-1
        claude-opus-5:
          deploymentName: claude-opus-5
        claude-sonnet-5:
          deploymentName: claude-sonnet-5
        claude-3.7-sonnet:
          deploymentName: claude-3-7-sonnet-20250219
        claude-3.5-sonnet:
          deploymentName: claude-3-5-sonnet-v2@20241022
        claude-3.5-haiku:
          deploymentName: claude-3-5-haiku@20241022

Note: Anthropic endpoint supports all Shared Endpoint Settings, including streamRate, titleModel, titleMethod, titlePrompt, titlePromptTemplate, and titleEndpoint.

vertex

The vertex object contains all Vertex AI-specific configuration options.

region

Key:

Default: us-east5

Available Regions:

• global
• us
• eu
• us-east5
• us-central1
• europe-west1
• europe-west4
• asia-southeast1

Important: Claude Opus 4.7 and newer, Opus 5, Sonnet 5, and Fable/Mythos 5 or 5.1 require global, us, or eu. Specific regions such as us-east5 serve Claude Sonnet 4.6 and earlier and return a model-not-found response for newer models. Use global for the broadest availability and no regional pricing premium, or us/eu when you need multi-region data residency.

Example:

region: "global"

projectId

Key:

Default: Auto-detected from service key file

Example:

projectId: "${GOOGLE_PROJECT_ID}"

serviceKeyFile

Key:

Default: api/data/auth.json (or GOOGLE_SERVICE_KEY_FILE environment variable)

Example:

serviceKeyFile: "/etc/secrets/gcp-service-account.json"

models

The models field defines the available Claude models and maps user-friendly names to Vertex AI deployment IDs. This works similarly to Azure OpenAI model mapping.

When models is omitted, LibreChat uses its built-in Vertex catalog and removes models incompatible with the configured region. An explicit models list or map is never filtered, so every deployment ID must be available in the selected location.

Format Options

You can configure models in three ways:

Option 1: Simple Array

Use the actual Vertex AI model IDs directly. These will be shown as-is in the UI:

models:
  - "claude-fable-5-1"
  - "claude-opus-5"
  - "claude-sonnet-5"
  - "claude-sonnet-4-20250514"
  - "claude-3-7-sonnet-20250219"
  - "claude-3-5-haiku@20241022"

Option 2: Object with Custom Names (Recommended)

Map user-friendly names to Vertex AI deployment names:

models:
  claude-fable-5-1:        # Visible in UI
    deploymentName: claude-fable-5-1  # Actual Vertex AI model ID
  claude-opus-5:           # Visible in UI
    deploymentName: claude-opus-5  # Actual Vertex AI model ID
  claude-sonnet-5:
    deploymentName: claude-sonnet-5
  claude-3.5-haiku:
    deploymentName: claude-3-5-haiku@20241022

Option 3: Mixed Format with Default

Set a default deployment name and use boolean values for models that inherit it:

deploymentName: claude-sonnet-4-20250514  # Default deployment
models:
  claude-sonnet-4: true  # Uses default deploymentName
  claude-3.5-haiku:
    deploymentName: claude-3-5-haiku@20241022  # Override for this model

Model Object Properties

Example:

models:
  claude-sonnet-4:
    deploymentName: claude-sonnet-4-20250514

Environment Variable Alternative

For simpler setups, you can configure Vertex AI using environment variables instead of YAML:

# Enable Vertex AI mode
ANTHROPIC_USE_VERTEX=true

# Vertex AI region (optional, defaults to us-east5)
ANTHROPIC_VERTEX_REGION=global

# Path to service account key (optional, defaults to api/data/auth.json)
GOOGLE_SERVICE_KEY_FILE=/path/to/service-account.json

Note: When using environment variables, model mapping is not available and LibreChat includes its standard Anthropic model catalog. Set ANTHROPIC_VERTEX_REGION to global, us, or eu before selecting a modern model that is unavailable from a specific region.

Complete Examples

Basic Setup

Minimal configuration using defaults (Vertex AI is enabled by the presence of the vertex section):

endpoints:
  anthropic:
    vertex:
      region: global

This uses:

• Region: global
• Service key: api/data/auth.json (or GOOGLE_SERVICE_KEY_FILE env var)
• Project ID: Auto-detected from service key
• Models: LibreChat's built-in Vertex catalog for the selected location

Production Setup with Model Mapping

Full configuration with custom model names and titles:

endpoints:
  anthropic:
    streamRate: 20
    titleModel: "haiku"
    titleMethod: "completion"

    vertex:
      region: "global"
      serviceKeyFile: "${GOOGLE_SERVICE_KEY_FILE}"

      models:
        fable:
          deploymentName: claude-fable-5-1
        opus:
          deploymentName: claude-opus-5
        sonnet:
          deploymentName: claude-sonnet-5
        haiku:
          deploymentName: claude-3-5-haiku@20241022

Multi-Region Setup

You can only configure one region per deployment. For multi-region needs, consider using separate LibreChat instances or custom endpoints.

Troubleshooting

Common Errors

"Could not load the default credentials"

• Ensure the service account key file exists at the specified path
• Check file permissions (must be readable by the LibreChat process)
• Verify the JSON file is valid and not corrupted

"Permission denied" or "403 Forbidden"

• Verify the service account has the Vertex AI User role
• Ensure Claude models are enabled in your Vertex AI Model Garden
• Check that the service account belongs to the correct project

"Model not found"

• Check that the model ID in deploymentName is correct
• Verify the model is available in your selected region
• Use global, us, or eu for Opus 4.7+, Opus 5, Sonnet 5, and Fable/Mythos 5
• Ensure the model is enabled in your Vertex AI Model Garden

Region Issues

"Invalid region" or "Region not supported"

• Use one of the supported regions listed above
• Try using global region which provides automatic routing
• Check Google Cloud's documentation for the latest list of regions where Claude is available

"Model not available in region"

• Not all Claude models are available in all regions
• Switch to global, us, or eu for models newer than Sonnet 4.6
• Check the Vertex AI Model Garden to see which models are available in your region

Latency issues

• If you're experiencing high latency, try using a region geographically closer to your users
• The global region automatically routes to the nearest available region
• For production workloads with strict latency requirements, test different regions and choose the one with best performance for your use case

Verifying Setup

1. Ensure your service account key is valid:

   gcloud auth activate-service-account --key-file=/path/to/key.json
   gcloud auth list

2. Test Vertex AI access:

   gcloud ai models list --region=us-east5

3. Verify Claude model access:

   curl -X POST \
     -H "Authorization: Bearer $(gcloud auth print-access-token)" \
     -H "Content-Type: application/json" \
     "https://us-east5-aiplatform.googleapis.com/v1/projects/YOUR_PROJECT/locations/us-east5/publishers/anthropic/models/claude-3-5-haiku@20241022:rawPredict" \
     -d '{"anthropic_version": "vertex-2023-10-16", "max_tokens": 100, "messages": [{"role": "user", "content": "Hello"}]}'

Notes

• Vertex AI and direct Anthropic API are mutually exclusive. When a vertex configuration section is present, the ANTHROPIC_API_KEY environment variable is ignored.
• Web search functionality is fully supported with Vertex AI.
• Prompt caching is supported via automatic header filtering for Vertex AI compatibility.
• Function calling and tool use work the same as with the direct Anthropic API.