Docs
Configuration
librechat.yaml
Object Structure
Azure OpenAI

Azure OpenAI Object Structure

Integrating Azure OpenAI Service with your application allows you to seamlessly utilize multiple deployments and region models hosted by Azure OpenAI. This section details how to configure the Azure OpenAI endpoint for your needs.

For a detailed guide on setting up Azure OpenAI configurations, click here

Example Configuration

Example Azure OpenAI Object Structure
endpoints:
  azureOpenAI:
    titleModel: "gpt-4-turbo"
    plugins: true
    groups:
      - group: "my-westus" # arbitrary name
        apiKey: "${WESTUS_API_KEY}"
        instanceName: "actual-instance-name" # name of the resource group or instance
        version: "2023-12-01-preview"
        # baseURL: https://prod.example.com
        # additionalHeaders:
        #   X-Custom-Header: value
        models:
          gpt-4-vision-preview:
            deploymentName: gpt-4-vision-preview
            version: "2024-02-15-preview"
          gpt-3.5-turbo:
            deploymentName: gpt-35-turbo
          gpt-3.5-turbo-1106:
            deploymentName: gpt-35-turbo-1106
          gpt-4:
            deploymentName: gpt-4
          gpt-4-1106-preview:
            deploymentName: gpt-4-1106-preview
      - group: "my-eastus"
        apiKey: "${EASTUS_API_KEY}"
        instanceName: "actual-eastus-instance-name"
        deploymentName: gpt-4-turbo
        version: "2024-02-15-preview"
        baseURL: "https://gateway.ai.cloudflare.com/v1/cloudflareId/azure/azure-openai/${INSTANCE_NAME}/${DEPLOYMENT_NAME}" # uses env variables
        additionalHeaders:
          X-Custom-Header: value
        models:
          gpt-4-turbo: true

plugins

Key:

KeyTypeDescriptionExample
pluginsBooleanEnables or disables plugins for the Azure OpenAI endpoint. When set to `true`, it activates any plugins associated with this endpoint, allowing the system to use additional features or functionality provided by the plugins.Choose one, either the official OpenAI API or Azure OpenAI API for plugins, not both.

Default: Not specified

Example:

plugins
plugins: true

assistants

Key:

KeyTypeDescriptionExample
assistantsBooleanEnables or disables assistants for the Azure OpenAI endpoint. When set to `true`, activates assistants associated with this endpoint.Choose one, either the official OpenAI API or Azure OpenAI API for assistants, not both.

Default: Not specified

Example:

endpoints / azureOpenAI / assistants
assistants: true

groups

Key:

KeyTypeDescriptionExample
groupsArrayConfiguration for groups of models by geographic location or purpose. Each item in the `groups` array configures a set of models under a certain grouping, often by geographic region or distinct configuration.

Default: Not specified

Note: See example above.

Group Object Structure

Each item under groups is part of a list of records, each with the following fields:

group

Key:

KeyTypeDescriptionExample
groupStringIdentifier for a group of models.

Required: yes

Example:

endpoints / azureOpenAI / groups / {group_item} / group
"group": "my-westus"

apiKey

Key:

KeyTypeDescriptionExample
apiKeyStringThe API key for accessing the Azure OpenAI Service.It's highly recommended to use a custom env. variable reference for this field, i.e. `${YOUR_VARIABLE}`

Required: yes

Example:

endpoints / azureOpenAI / groups / {group_item} / apiKey
apiKey: "${WESTUS_API_KEY}"

instanceName

Key:

KeyTypeDescriptionExample
instanceNameStringName of the Azure instance.It's recommended to use a custom env. variable reference for this field, i.e. `${YOUR_VARIABLE}`

Required: yes

Example:

endpoints / azureOpenAI / groups / {group_item} / instanceName
instanceName: "my-westus"

version

Key:

KeyTypeDescriptionExample
versionStringAPI version.It's recommended to use a custom env. variable reference for this field, i.e. `${YOUR_VARIABLE}`

Default: Not specified

Example:

endpoints / azureOpenAI / groups / {group_item} / version
version: "2023-12-01-preview"

baseURL

Key:

KeyTypeDescriptionExample
baseURLStringThe base URL for the Azure OpenAI Service.It's recommended to use a custom env. variable reference for this field, i.e. `${YOUR_VARIABLE}`

Default: Not specified

Example:

endpoints / azureOpenAI / groups / {group_item} / baseURL
baseURL: "https://prod.example.com"

additionalHeaders

Key:

KeyTypeDescriptionExample
additionalHeadersDictionaryAdditional headers for API requests.It's recommended to use a custom env. variable reference for the values of field, as shown in the example. `api-key` header value is sent on every request.

Default: Not specified

Example:

endpoints / azureOpenAI / groups / {group_item} / additionalHeaders
additionalHeaders:
  X-Custom-Header: ${YOUR_SECRET_CUSTOM_VARIABLE}

serverless

Key:

KeyTypeDescriptionExample
serverlessBooleanIndicates the use of a serverless inference endpoint for Azure OpenAI chat completions. When set to `true`, specifies that the group is configured to use serverless inference endpoints as an Azure "Models as a Service" model.More info [here](./azure_openai.md#serverless-inference-endpoints)

Default: Not specified

Example:

endpoints / azureOpenAI / groups / {group_item} / serverless
serverless: true

addParams

Key:

KeyTypeDescriptionExample
addParamsObject/DictionaryAdds additional parameters to requests. Useful for specifying API-specific options.

Default: Not specified

Example:

endpoints / azureOpenAI / groups / {group_item} / addParams
addParams:
  safe_prompt: true

dropParams

Key:

KeyTypeDescriptionExample
dropParamsArray/List of StringsRemoves [default parameters](#default-parameters) from requests. Excludes specified [default parameters](#default-parameters).For a list of default parameters sent with every request, see the ["Default Parameters"](#default-parameters) Section below.

Default: Not specified

Example:

endpoints / azureOpenAI / groups / {group_item} / dropParams
dropParams: ["stop", "user", "frequency_penalty", "presence_penalty"]

forcePrompt

Key:

KeyTypeDescriptionExample
forcePromptBooleanIf `true`, sends a `prompt` parameter instead of `messages`. This combines all messages into a single text payload, following OpenAI format, and uses the `/completions` endpoint of your baseURL rather than `/chat/completions`.

Default: Not specified

Example:

endpoints / azureOpenAI / groups / {group_item} / forcePrompt
forcePrompt: false

models

Key:

KeyTypeDescriptionExample
modelsConfiguration for individual models within a group. Configures settings for each model, including deployment name and version.Model configurations can adopt the group's deployment name and/or version when configured as a boolean (set to `true`) or an object for detailed settings of either of those fields.

Default: Not specified

Example:

endpoints / azureOpenAI / groups / {group_item} / models
models:
  gpt-4-vision-preview: 
    deploymentName: "arbitrary-deployment-name"
    version: "2024-02-15-preview"