Docs
Configuration
librechat.yaml
Object Structure
Model Specs

Model Specs Object Structure

Overview

The modelSpecs object helps you provide a simpler UI experience for AI models within your application.

There are 3 main fields under modelSpecs:

  • enforce (optional; default: false)
  • prioritize (optional; default: true)
  • list (required)

Notes:

  • If enforce is set to true, model specifications can potentially conflict with other interface settings such as endpointsMenu, modelSelect, presets, and parameters.
  • The list array contains detailed configurations for each model, including presets that dictate specific behaviors, appearances, and capabilities.
  • If interface fields are not specified in the configuration, having a list of model specs will disable the following interface elements:
    • endpointsMenu
    • modelSelect
    • parameters
    • presets
  • If you would like to enable these interface elements along with model specs, you can set them to true in the interface object.

Example

modelSpecs
modelSpecs:
  enforce: true
  prioritize: true
  list:
    - name: "meeting-notes-gpt4"
      label: "Meeting Notes Assistant (GPT4)"
      description: "Generate meeting notes by simply pasting in the transcript from a Teams recording."
      iconURL: "https://example.com/icon.png"
      preset:
        default: true
        endpoint: "azureOpenAI"
        model: "gpt-4-turbo-1106-preview"
        maxContextTokens: 128000 # Maximum context tokens
        max_tokens: 4096 # Maximum output tokens
        temperature: 0.2
        modelLabel: "Meeting Summarizer"
        greeting: |
          This assistant creates meeting notes based on transcripts of Teams recordings.
          To start, simply paste the transcript into the chat box.
        promptPrefix: |
          Based on the transcript, create coherent meeting minutes for a business meeting. Include the following sections:
          - Date and Attendees
          - Agenda
          - Minutes
          - Action Items
 
          Focus on what items were discussed and/or resolved. List any open action items.
          The format should be a bulleted list of high level topics in chronological order, and then one or more concise sentences explaining the details.
          Each high level topic should have at least two sub topics listed, but add as many as necessary to support the high level topic. 
 
          - Do not start items with the same opening words.
 
          Take a deep breath and be sure to think step by step.

enforce

Key:

KeyTypeDescriptionExample
enforceBooleanDetermines whether the model specifications should strictly override other configuration settings.Setting this to `true` can lead to conflicts with interface options if not managed carefully.

Default: false

Example:

modelSpecs / enforce
modelSpecs:
  enforce: true

prioritize

Key:

KeyTypeDescriptionExample
prioritizeBooleanSpecifies if model specifications should take priority over the default configuration when both are applicable.When set to `true`, it ensures that a modelSpec is always selected in the UI. Doing this may prevent users from selecting different endpoints for the selected spec.

Default: true

Example:

modelSpecs / prioritize
modelSpecs:
  prioritize: false

list

Key:

KeyTypeDescriptionExample
listArray of ObjectsContains a list of individual model specifications detailing various configurations and behaviors.Each object in the list details the configuration for a specific model, including its behaviors, appearance, and capabilities related to the application's functionality.

Each spec object in the list can have the following settings:

Overview

  • name
    • Unique identifier for the model.
  • label
    • A user-friendly name or label for the model, shown in the header dropdown.
  • description
    • A brief description of the model and its intended use or role, shown in the header dropdown menu.
  • iconURL
    • URL or a predefined endpoint name for the model’s icon.
  • default
    • Specifies if this model spec is the default selection, to be auto-selected on every new chat.
  • showIconInMenu
    • Controls whether the model’s icon appears in the header dropdown menu.
  • showIconInHeader
    • Controls whether the model’s icon appears in the header dropdown button, left of its name.
  • preset
    • Detailed preset configurations that define the behavior and capabilities of the model (see preset object structure section below for more details).

Preset Object Structure

The preset field for a modelSpec list item is made up of a comprehensive configuration blueprint for AI models within the system. It is designed to specify the operational settings of AI models, tailoring their behavior, outputs, and interactions with other system components and endpoints.

endpoint

Key:

KeyTypeDescriptionExample
endpointEnum (EModelEndpoint) or String (nullable)Specifies the endpoint the model communicates with to execute operations. This setting determines the external or internal service that the model interfaces with.

Default: None

Example:

modelSpecs / list / {spec_item} / preset / endpoint
preset:
  endpoint: "openAI"

model

Key:

KeyTypeDescriptionExample
modelString (nullable)The model to use for the preset. This has to correspond to a model configured under endpoints.None

Default: None

Example:

modelSpecs / list / {spec_item} / preset / model
preset:
  model: "gpt-4-turbo"

modelLabel

Key:

KeyTypeDescriptionExample
modelLabelString (nullable, optional)The label used to identify the model in user interfaces or logs. It provides a human-readable name for the model, which is displayed in the UI, as well as made aware to the AI.None

Default: None

Example:

modelSpecs / list / {spec_item} / preset / modelLabel
preset:
  modelLabel: "Customer Support Bot"

greeting

Key:

KeyTypeDescriptionExample
greetingString (optional)A predefined message that is visible in the UI before a new chat is started. This is a good way to provide instructions to the user, or to make the interface seem more friendly and accessible.

Default: None

Example:

modelSpecs / list / {spec_item} / preset / greeting
preset:
  greeting: "This assistant creates meeting notes based on transcripts of Teams recordings. To start, simply paste the transcript into the chat box."

promptPrefix

Key:

KeyTypeDescriptionExample
promptPrefixString (nullable, optional)A static text prepended to every prompt sent to the model, setting a consistent context for responses.When using "assistants" as the endpoint, this becomes the OpenAI field `additional_instructions`.

Default: None

Example 1:

modelSpecs / list / {spec_item} / preset / promptPrefix
preset:
  promptPrefix: "As a financial advisor, ..."

Example 2:

modelSpecs / list / {spec_item} / preset / promptPrefix
preset:
  promptPrefix: |
          Based on the transcript, create coherent meeting minutes for a business meeting. Include the following sections:
          - Date and Attendees
          - Agenda
          - Minutes
          - Action Items
 
          Focus on what items were discussed and/or resolved. List any open action items.
          The format should be a bulleted list of high level topics in chronological order, and then one or more concise sentences explaining the details.
          Each high level topic should have at least two sub topics listed, but add as many as necessary to support the high level topic. 
 
          - Do not start items with the same opening words.
 
          Take a deep breath and be sure to think step by step.

model_options

These settings control the stochastic nature and behavior of model responses, affecting creativity, relevance, and variability. Additionally it is possible to specify the number of tokens for the context and output windows.

Key:

KeyTypeDescriptionExample
temperatureNumber (optional)Model response temperature.
top_pNumber (optional)Value of top_p (nucleus sampling).
top_kNumber (optional)Value of top_k (k-sampling).
frequency_penaltyNumber (optional)Penalty for repetition in model responses.
presence_penaltyNumber (optional)Penalty for repetition in model responses.
stopArray of Strings (optional)Stop tokens for model responses.
maxContextTokensNumber (optional)The maximum number of context tokens to provide to the model.This is useful when configuring custom or non-standard models, or when you want to limit the maximum context for this preset.
max_tokensNumber (optional)The maximum number of output tokens to request from the model.This is useful when configuring custom or non-standard models, e.g. phi-3.

Example:

modelSpecs / list / {spec_item} / preset / model_options
preset:
  temperature: 0.7
  top_p: 0.9
  maxContextTokens: 4096

resendFiles

Key:

KeyTypeDescriptionExample
resendFilesBoolean (optional)Indicates whether files should be resent in scenarios where persistent sessions are not maintained.

Default: false

Example:

modelSpecs / list / {spec_item} / preset / resendFiles
preset:
  resendFiles: true

imageDetail

Key:

KeyTypeDescriptionExample
imageDetaileImageDetailSchema (optional)Specifies the level of detail required in image analysis tasks, applicable to models with vision capabilities (OpenAI spec).

Example:

modelSpecs / list / {spec_item} / preset / imageDetail
preset:
  imageDetail: "high"

agentOptions

Key:

KeyTypeDescriptionExample
agentOptionsRecord/Object (optional)Specific to `gptPlugins` endpoint. Can be omitted either partially or completely for default settings
agentString (optional)Type of agent (either "functions" or "classic"; default: "functions")
skipCompletionBoolean (optional)Whether to skip automatic completion suggestions (default: true)
modelString (optional)Model version or identifier (default: "gpt-4-turbo")
temperatureNumber (optional)Randomness in the model's responses (default: 0)

Example:

modelSpecs / list / {spec_item} / preset / agentOptions
preset:
  agentOptions:
    agent: "functions"
    skipCompletion: false
    model: "gpt-4-turbo"
    temperature: 0.5

tools

Key:

KeyTypeDescriptionExample
toolsArray of Strings (optional)Specific to `gptPlugins` endpoint. List of tool/plugin names.

Example:

modelSpecs / list / {spec_item} / preset / tools
preset:
  tools: ["dalle", "tavily_search_results_json", "azure-ai-search", "traversaal_search"]

Notes:

  • At the moment, only tools that have credentials provided for them via .env file can be used with modelSpecs, unless the user already had the tool installed.
  • You can find the names of the tools to filter in api/app/clients/tools/manifest.json
    • Use the pluginKey value
  • Also, any listed under the “.well-known” directory api/app/clients/tools/.well-known
    • Use the name_for_model value

assistant_options

Key:

KeyTypeDescriptionExample
assistant_optionsConfigurations specific to assistants, such as identifying an assistant, overriding the assistant's instructions.
assistant_idString (optional)Identification of an assistant.
instructionsString (optional)Overrides the assistant's default instructions.

Example:

modelSpecs / list / {spec_item} / preset / assistant_options
preset:
  assistant_id: "asst_98765"
  instructions: "Please handle customer queries regarding order status."