LiteLLM

LiteLLM Proxy is a self-hosted gateway that exposes models from many providers through a single OpenAI-compatible endpoint, so you can point LibreChat at your own proxy.

Configuration

The API key matches the value set in your LiteLLM proxy config rather than a provider key, so use the placeholder below or your own. Point baseURL at your running proxy. Add the endpoint to the custom list under endpoints in your librechat.yaml. If your file already has an endpoints.custom list, paste only the - name: entry into it, keeping the same indentation:

endpoints:
  custom:
    - name: "LiteLLM"
      apiKey: "sk-from-config-file"
      baseURL: "http://localhost:8000/v1"
      # if using LiteLLM example in docker-compose.override.yml.example, use "http://litellm:8000/v1"
      models:
        default: ["gpt-3.5-turbo"]
        fetch: true
      titleConvo: true
      titleModel: "gpt-3.5-turbo"
      modelDisplayLabel: "LiteLLM"

Notes

• See Using LibreChat with LiteLLM Proxy for a full walkthrough of setting up the proxy.
• With fetch: true, LibreChat loads the full list of models configured in your proxy, so default is only the initial selection.
• If you run the bundled proxy from docker-compose.override.yml.example, set baseURL to http://litellm:8000/v1 so the containers reach each other by service name.