Azure OpenAI Object Structure

Integrating Azure OpenAI Service with your application allows you to seamlessly utilize multiple deployments and region models hosted by Azure OpenAI. This section details how to configure the Azure OpenAI endpoint for your needs.

For a detailed guide on setting up Azure OpenAI configurations, click here

Example Configuration

Example Azure OpenAI Object Structure

endpoints:
  azureOpenAI:
    titleModel: "gpt-4-turbo"
    plugins: true
    groups:
      - group: "my-westus" # arbitrary name
        apiKey: "${WESTUS_API_KEY}"
        instanceName: "actual-instance-name" # name of the resource group or instance
        version: "2023-12-01-preview"
        # baseURL: https://prod.example.com
        # additionalHeaders:
        #   X-Custom-Header: value
        models:
          gpt-4-vision-preview:
            deploymentName: gpt-4-vision-preview
            version: "2024-02-15-preview"
          gpt-3.5-turbo:
            deploymentName: gpt-35-turbo
          gpt-3.5-turbo-1106:
            deploymentName: gpt-35-turbo-1106
          gpt-4:
            deploymentName: gpt-4
          gpt-4-1106-preview:
            deploymentName: gpt-4-1106-preview
      - group: "my-eastus"
        apiKey: "${EASTUS_API_KEY}"
        instanceName: "actual-eastus-instance-name"
        deploymentName: gpt-4-turbo
        version: "2024-02-15-preview"
        baseURL: "https://gateway.ai.cloudflare.com/v1/cloudflareId/azure/azure-openai/${INSTANCE_NAME}/${DEPLOYMENT_NAME}" # uses env variables
        additionalHeaders:
          X-Custom-Header: value
        models:
          gpt-4-turbo: true

plugins

Key:

Key	Type	Description	Example
plugins	Boolean	Enables or disables plugins for the Azure OpenAI endpoint. When set to `true`, it activates any plugins associated with this endpoint, allowing the system to use additional features or functionality provided by the plugins.	Choose one, either the official OpenAI API or Azure OpenAI API for plugins, not both.

Default: Not specified

Example:

plugins

plugins: true

assistants

Key:

Key	Type	Description	Example
assistants	Boolean	Enables or disables assistants for the Azure OpenAI endpoint. When set to `true`, activates assistants associated with this endpoint.	Choose one, either the official OpenAI API or Azure OpenAI API for assistants, not both.

Default: Not specified

Example:

endpoints / azureOpenAI / assistants

assistants: true

groups

Key:

Key	Type	Description	Example
groups	Array	Configuration for groups of models by geographic location or purpose. Each item in the `groups` array configures a set of models under a certain grouping, often by geographic region or distinct configuration.

Default: Not specified

Note: See example above.

Group Object Structure

Each item under groups is part of a list of records, each with the following fields:

group

Key:

Key	Type	Description	Example
group	String	Identifier for a group of models.

Required: yes

Example:

endpoints / azureOpenAI / groups / {group_item} / group

"group": "my-westus"

apiKey

Key:

Key	Type	Description	Example
apiKey	String	The API key for accessing the Azure OpenAI Service.	It's highly recommended to use a custom env. variable reference for this field, i.e. `${YOUR_VARIABLE}`

Required: yes

Example:

endpoints / azureOpenAI / groups / {group_item} / apiKey

apiKey: "${WESTUS_API_KEY}"

instanceName

Key:

Key	Type	Description	Example
instanceName	String	Name of the Azure instance.	It's recommended to use a custom env. variable reference for this field, i.e. `${YOUR_VARIABLE}`

Required: yes

Example:

endpoints / azureOpenAI / groups / {group_item} / instanceName

instanceName: "my-westus"

version

Key:

Key	Type	Description	Example
version	String	API version.	It's recommended to use a custom env. variable reference for this field, i.e. `${YOUR_VARIABLE}`

Default: Not specified

Example:

endpoints / azureOpenAI / groups / {group_item} / version

version: "2023-12-01-preview"

baseURL

Key:

Key	Type	Description	Example
baseURL	String	The base URL for the Azure OpenAI Service.	It's recommended to use a custom env. variable reference for this field, i.e. `${YOUR_VARIABLE}`

Default: Not specified

Example:

endpoints / azureOpenAI / groups / {group_item} / baseURL

baseURL: "https://prod.example.com"

additionalHeaders

Key:

Key	Type	Description	Example
additionalHeaders	Dictionary	Additional headers for API requests.	It's recommended to use a custom env. variable reference for the values of field, as shown in the example. `api-key` header value is sent on every request.

Default: Not specified

Example:

endpoints / azureOpenAI / groups / {group_item} / additionalHeaders

additionalHeaders:
  X-Custom-Header: ${YOUR_SECRET_CUSTOM_VARIABLE}

serverless

Key:

Key	Type	Description	Example
serverless	Boolean	Indicates the use of a serverless inference endpoint for Azure OpenAI chat completions. When set to `true`, specifies that the group is configured to use serverless inference endpoints as an Azure "Models as a Service" model.	More info [here](./azure_openai.md#serverless-inference-endpoints)

Default: Not specified

Example:

endpoints / azureOpenAI / groups / {group_item} / serverless

serverless: true

addParams

Key:

Key	Type	Description	Example
addParams	Object/Dictionary	Adds additional parameters to requests. Useful for specifying API-specific options.

Default: Not specified

Example:

endpoints / azureOpenAI / groups / {group_item} / addParams

addParams:
  safe_prompt: true

dropParams

Key:

Key	Type	Description	Example
dropParams	Array/List of Strings	Removes [default parameters](#default-parameters) from requests. Excludes specified [default parameters](#default-parameters).	For a list of default parameters sent with every request, see the ["Default Parameters"](#default-parameters) Section below.

Default: Not specified

Example:

endpoints / azureOpenAI / groups / {group_item} / dropParams

dropParams: ["stop", "user", "frequency_penalty", "presence_penalty"]

forcePrompt

Key:

Key	Type	Description	Example
forcePrompt	Boolean	If `true`, sends a `prompt` parameter instead of `messages`. This combines all messages into a single text payload, following OpenAI format, and uses the `/completions` endpoint of your baseURL rather than `/chat/completions`.

Default: Not specified

Example:

endpoints / azureOpenAI / groups / {group_item} / forcePrompt

forcePrompt: false

models

Key:

Key	Type	Description	Example
models		Configuration for individual models within a group. Configures settings for each model, including deployment name and version.	Model configurations can adopt the group's deployment name and/or version when configured as a boolean (set to `true`) or an object for detailed settings of either of those fields.

Default: Not specified

Example:

endpoints / azureOpenAI / groups / {group_item} / models

models:
  gpt-4-vision-preview: 
    deploymentName: "arbitrary-deployment-name"
    version: "2024-02-15-preview"

Custom Endpoint Azure Model Config