Databricks

Databricks serves foundation models and your own fine-tuned models through Mosaic AI Model Serving, which exposes an OpenAI-compatible serving endpoint.

Get an API key

Sign up for Databricks and generate a personal access token from your workspace. Add it to your .env file:

DATABRICKS_API_KEY=your-api-key

Configuration

Add the endpoint under endpoints.custom in your librechat.yaml:

    - name: 'Databricks'
      apiKey: '${DATABRICKS_API_KEY}'
      baseURL: 'https://your_databricks_serving_endpoint_url_here_ending_with/invocations'
      models:
        default: [
          "databricks-meta-llama-3-70b-instruct",
        ]
        fetch: false
      titleConvo: true
      titleModel: 'current_model'
      directEndpoint: true # required
      titleMessageRole: 'user' # required

Notes

• Databricks exposes a full completions endpoint ending in invocations for serving-endpoints, so directEndpoint is required.
• Set titleMessageRole to user for title generation. A standalone system message is not supported.