OCR Config Object Structure

Overview

The ocr object allows you to configure Optical Character Recognition (OCR) settings for the application, enabling the extraction of text from images. This section provides a detailed breakdown of the ocr object structure.

There are 4 main fields under ocr:

mistralModel
apiKey
baseURL
strategy

Notes:

If using the Mistral OCR API, you don’t need to edit your librechat.yaml file.
- You only need the following environment variables to get started: OCR_API_KEY and OCR_BASEURL.
OCR functionality allows the application to extract text from images, which can then be processed by AI models.
The default strategy is mistral_ocr, which uses Mistral’s OCR capabilities.
You can also configure a custom OCR service by setting the strategy to custom_ocr.
Azure-deployed Mistral OCR models can be used by setting the strategy to azure_mistral_ocr.
Google Vertex AI-deployed Mistral OCR models can be used by setting the strategy to vertexai_mistral_ocr.
- Requires the GOOGLE_SERVICE_KEY_FILE environment variable to be set with service account credentials
- The service key can be provided as: file path, URL, base64 encoded JSON, or raw JSON string
- Project ID and location are automatically extracted from the service account credentials
If using the default Mistral OCR, you may optionally specify a specific Mistral model to use.
Environment variable parsing is supported for apiKey, baseURL, and mistralModel parameters.
A user_provided strategy option is planned for future releases but is not yet implemented.

Example

ocr

ocr:
  mistralModel: "mistral-ocr-latest"
  apiKey: "your-mistral-api-key"
  strategy: "mistral_ocr"

Example with custom OCR:

ocr with custom OCR

ocr:
  apiKey: "your-custom-ocr-api-key"
  baseURL: "https://your-custom-ocr-service.com/api"
  strategy: "custom_ocr"

Example with Azure Mistral OCR:

ocr with Azure Mistral OCR

ocr:
  mistralModel: "deployed-mistral-ocr-2503" # should match deployment name on Azure
  apiKey: "${AZURE_MISTRAL_OCR_API_KEY}" # arbitrary .env var reference
  baseURL: "https://your-deployed-endpoint.models.ai.azure.com/v1" # hardcoded, can also be .env var reference
  strategy: "azure_mistral_ocr"

Example with Google Vertex AI Mistral OCR:

ocr with Google Vertex AI Mistral OCR

ocr:
  mistralModel: "mistral-ocr-2505" # model name as deployed in Vertex AI
  strategy: "vertexai_mistral_ocr"

mistralModel

Key	Type	Description	Example
mistralModel	String	The Mistral model to use for OCR processing. For Azure deployments, this should match your deployment name. For Google Vertex AI, this should match the model name in your deployment.	Optional. Specifies which Mistral model should be used when the strategy is set to mistral_ocr, azure_mistral_ocr, or vertexai_mistral_ocr.

ocr / mistralModel

ocr:
  mistralModel: "mistral-ocr-latest"

For Azure deployments:

ocr / mistralModel (Azure)

ocr:
  mistralModel: "deployed-mistral-ocr-2503" # Your Azure deployment name

For Google Vertex AI deployments:

ocr / mistralModel (Google Vertex AI)

ocr:
  mistralModel: "mistral-ocr-2505" # Your Vertex AI model name

apiKey

Key	Type	Description	Example
apiKey	String	The API key for the OCR service. Not used for Google Vertex AI (uses service account authentication via GOOGLE_SERVICE_KEY_FILE).	Optional. Defaults to the environment variable OCR_API_KEY if not specified.

ocr / apiKey

ocr:
  apiKey: "your-ocr-api-key"

baseURL

Key	Type	Description	Example
baseURL	String	The base URL for the OCR service API. For Google Vertex AI, this is automatically constructed from the service account credentials.	Optional. Defaults to the environment variable OCR_BASEURL if not specified.

ocr / baseURL

ocr:
  baseURL: "https://your-ocr-service.com/api"

strategy

Key	Type	Description	Example
strategy	String	The OCR strategy to use.	Determines which OCR service to use. Options are "mistral_ocr", "azure_mistral_ocr", "vertexai_mistral_ocr", or "custom_ocr". Defaults to "mistral_ocr".

ocr / strategy

ocr:
  strategy: "custom_ocr"

Available Strategies:

mistral_ocr: Uses Mistral’s OCR capabilities via the standard Mistral API.
azure_mistral_ocr: Uses Mistral OCR models deployed on Azure AI Foundry.
vertexai_mistral_ocr: Uses Mistral OCR models deployed on Google Cloud Vertex AI.
custom_ocr: Uses a custom OCR service specified by the baseURL (not yet implemented).

MCP Servers Memory