Model Specs Object Structure

Overview

The modelSpecs object helps you provide a simpler UI experience for AI models within your application.

There are 3 main fields under modelSpecs:

enforce (optional; default: false)
prioritize (optional; default: true)
list (required)

Notes:

If enforce is set to true, model specifications can potentially conflict with other interface settings such as endpointsMenu, modelSelect, presets, and parameters.
The list array contains detailed configurations for each model, including presets that dictate specific behaviors, appearances, and capabilities.
If interface fields are not specified in the configuration, having a list of model specs will disable the following interface elements:
- endpointsMenu
- modelSelect
- parameters
- presets
If you would like to enable these interface elements along with model specs, you can set them to true in the interface object.

Example

modelSpecs

modelSpecs:
  enforce: true
  prioritize: true
  list:
    - name: "meeting-notes-gpt4"
      label: "Meeting Notes Assistant (GPT4)"
      description: "Generate meeting notes by simply pasting in the transcript from a Teams recording."
      iconURL: "https://example.com/icon.png"
      preset:
        default: true
        endpoint: "azureOpenAI"
        model: "gpt-4-turbo-1106-preview"
        maxContextTokens: 128000 # Maximum context tokens
        max_tokens: 4096 # Maximum output tokens
        temperature: 0.2
        modelLabel: "Meeting Summarizer"
        greeting: |
          This assistant creates meeting notes based on transcripts of Teams recordings.
          To start, simply paste the transcript into the chat box.
        promptPrefix: |
          Based on the transcript, create coherent meeting minutes for a business meeting. Include the following sections:
          - Date and Attendees
          - Agenda
          - Minutes
          - Action Items
 
          Focus on what items were discussed and/or resolved. List any open action items.
          The format should be a bulleted list of high level topics in chronological order, and then one or more concise sentences explaining the details.
          Each high level topic should have at least two sub topics listed, but add as many as necessary to support the high level topic. 
 
          - Do not start items with the same opening words.
 
          Take a deep breath and be sure to think step by step.

enforce

Key:

Key	Type	Description	Example
enforce	Boolean	Determines whether the model specifications should strictly override other configuration settings.	Setting this to `true` can lead to conflicts with interface options if not managed carefully.

Default: false

Example:

modelSpecs / enforce

modelSpecs:
  enforce: true

prioritize

Key:

Key	Type	Description	Example
prioritize	Boolean	Specifies if model specifications should take priority over the default configuration when both are applicable.	When set to `true`, it ensures that a modelSpec is always selected in the UI. Doing this may prevent users from selecting different endpoints for the selected spec.

Default: true

Example:

modelSpecs / prioritize

modelSpecs:
  prioritize: false

list

Key:

Key	Type	Description	Example
list	Array of Objects	Contains a list of individual model specifications detailing various configurations and behaviors.	Each object in the list details the configuration for a specific model, including its behaviors, appearance, and capabilities related to the application's functionality.

Each spec object in the list can have the following settings:

Overview

name
- Unique identifier for the model.
label
- A user-friendly name or label for the model, shown in the header dropdown.
description
- A brief description of the model and its intended use or role, shown in the header dropdown menu.
iconURL
- URL or a predefined endpoint name for the model’s icon.
default
- Specifies if this model spec is the default selection, to be auto-selected on every new chat.
showIconInMenu
- Controls whether the model’s icon appears in the header dropdown menu.
showIconInHeader
- Controls whether the model’s icon appears in the header dropdown button, left of its name.
preset
- Detailed preset configurations that define the behavior and capabilities of the model (see preset object structure section below for more details).

Preset Object Structure

The preset field for a modelSpec list item is made up of a comprehensive configuration blueprint for AI models within the system. It is designed to specify the operational settings of AI models, tailoring their behavior, outputs, and interactions with other system components and endpoints.

endpoint

Key:

Key	Type	Description	Example
endpoint	Enum (EModelEndpoint) or String (nullable)	Specifies the endpoint the model communicates with to execute operations. This setting determines the external or internal service that the model interfaces with.

Default: None

Example:

modelSpecs / list / {spec_item} / preset / endpoint

preset:
  endpoint: "openAI"

model

Key:

Key	Type	Description	Example
model	String (nullable)	The model to use for the preset. This has to correspond to a model configured under endpoints.	None

Default: None

Example:

modelSpecs / list / {spec_item} / preset / model

preset:
  model: "gpt-4-turbo"

modelLabel

Key:

Key	Type	Description	Example
modelLabel	String (nullable, optional)	The label used to identify the model in user interfaces or logs. It provides a human-readable name for the model, which is displayed in the UI, as well as made aware to the AI.	None

Default: None

Example:

modelSpecs / list / {spec_item} / preset / modelLabel

preset:
  modelLabel: "Customer Support Bot"

greeting

Key:

Key	Type	Description	Example
greeting	String (optional)	A predefined message that is visible in the UI before a new chat is started. This is a good way to provide instructions to the user, or to make the interface seem more friendly and accessible.

Default: None

Example:

modelSpecs / list / {spec_item} / preset / greeting

preset:
  greeting: "This assistant creates meeting notes based on transcripts of Teams recordings. To start, simply paste the transcript into the chat box."

promptPrefix

Key:

Key	Type	Description	Example
promptPrefix	String (nullable, optional)	A static text prepended to every prompt sent to the model, setting a consistent context for responses.	When using "assistants" as the endpoint, this becomes the OpenAI field `additional_instructions`.

Default: None

Example 1:

modelSpecs / list / {spec_item} / preset / promptPrefix

preset:
  promptPrefix: "As a financial advisor, ..."

Example 2:

modelSpecs / list / {spec_item} / preset / promptPrefix

preset:
  promptPrefix: |
          Based on the transcript, create coherent meeting minutes for a business meeting. Include the following sections:
          - Date and Attendees
          - Agenda
          - Minutes
          - Action Items
 
          Focus on what items were discussed and/or resolved. List any open action items.
          The format should be a bulleted list of high level topics in chronological order, and then one or more concise sentences explaining the details.
          Each high level topic should have at least two sub topics listed, but add as many as necessary to support the high level topic. 
 
          - Do not start items with the same opening words.
 
          Take a deep breath and be sure to think step by step.

model_options

These settings control the stochastic nature and behavior of model responses, affecting creativity, relevance, and variability. Additionally it is possible to specify the number of tokens for the context and output windows.

Key:

Key	Type	Description	Example
temperature	Number (optional)	Model response temperature.
top_p	Number (optional)	Value of top_p (nucleus sampling).
top_k	Number (optional)	Value of top_k (k-sampling).
frequency_penalty	Number (optional)	Penalty for repetition in model responses.
presence_penalty	Number (optional)	Penalty for repetition in model responses.
stop	Array of Strings (optional)	Stop tokens for model responses.
maxContextTokens	Number (optional)	The maximum number of context tokens to provide to the model.	This is useful when configuring custom or non-standard models, or when you want to limit the maximum context for this preset.
max_tokens	Number (optional)	The maximum number of output tokens to request from the model.	This is useful when configuring custom or non-standard models, e.g. phi-3.

Example:

modelSpecs / list / {spec_item} / preset / model_options

preset:
  temperature: 0.7
  top_p: 0.9
  maxContextTokens: 4096

resendFiles

Key:

Key	Type	Description	Example
resendFiles	Boolean (optional)	Indicates whether files should be resent in scenarios where persistent sessions are not maintained.

Default: false

Example:

modelSpecs / list / {spec_item} / preset / resendFiles

preset:
  resendFiles: true

imageDetail

Key:

Key	Type	Description	Example
imageDetail	eImageDetailSchema (optional)	Specifies the level of detail required in image analysis tasks, applicable to models with vision capabilities (OpenAI spec).

Example:

modelSpecs / list / {spec_item} / preset / imageDetail

preset:
  imageDetail: "high"

agentOptions

Key:

Key	Type	Description
agentOptions	Record/Object (optional)	Specific to `gptPlugins` endpoint. Can be omitted either partially or completely for default settings
agent	String (optional)	Type of agent (either "functions" or "classic"; default: "functions")
skipCompletion	Boolean (optional)	Whether to skip automatic completion suggestions (default: true)
model	String (optional)	Model version or identifier (default: "gpt-4-turbo")
temperature	Number (optional)	Randomness in the model's responses (default: 0)

Example:

modelSpecs / list / {spec_item} / preset / agentOptions

preset:
  agentOptions:
    agent: "functions"
    skipCompletion: false
    model: "gpt-4-turbo"
    temperature: 0.5

tools

Key:

Key	Type	Description	Example
tools	Array of Strings (optional)	Specific to `gptPlugins` endpoint. List of tool/plugin names.

Example:

modelSpecs / list / {spec_item} / preset / tools

preset:
  tools: ["dalle", "tavily_search_results_json", "azure-ai-search", "traversaal_search"]

Notes:

At the moment, only tools that have credentials provided for them via .env file can be used with modelSpecs, unless the user already had the tool installed.
You can find the names of the tools to filter in api/app/clients/tools/manifest.json
- Use the pluginKey value
Also, any listed under the “.well-known” directory api/app/clients/tools/.well-known
- Use the name_for_model value

assistant_options

Key:

Key	Type	Description
assistant_options		Configurations specific to assistants, such as identifying an assistant, overriding the assistant's instructions.
assistant_id	String (optional)	Identification of an assistant.
instructions	String (optional)	Overrides the assistant's default instructions.

Example:

modelSpecs / list / {spec_item} / preset / assistant_options

preset:
  assistant_id: "asst_98765"
  instructions: "Please handle customer queries regarding order status."

Interface (UI)Registration