LiteLLM

LiteLLM Proxy is a self-hosted gateway that exposes models from many providers through a single OpenAI-compatible endpoint, so you can point LibreChat at your own proxy.

Configuration

The API key matches the value set in your LiteLLM proxy config rather than a provider key, so use the placeholder below or your own. Point baseURL at your running proxy. Add the endpoint under endpoints.custom in your librechat.yaml:

    - name: "LiteLLM"
      apiKey: "sk-from-config-file"
      baseURL: "http://localhost:8000/v1"
      # if using LiteLLM example in docker-compose.override.yml.example, use "http://litellm:8000/v1"
      models:
        default: ["gpt-3.5-turbo"]
        fetch: true
      titleConvo: true
      titleModel: "gpt-3.5-turbo"
      summarize: false
      summaryModel: "gpt-3.5-turbo"
      modelDisplayLabel: "LiteLLM"

Notes

• See Using LibreChat with LiteLLM Proxy for a full walkthrough of setting up the proxy.
• With fetch: true, LibreChat loads the full list of models configured in your proxy, so default is only the initial selection.
• If you run the bundled proxy from docker-compose.override.yml.example, set baseURL to http://litellm:8000/v1 so the containers reach each other by service name.