Portkey AI

Portkey is an AI gateway that fronts 250+ models through an OpenAI-compatible endpoint, adding observability, 50+ guardrails, caching, and conditional routing with fallbacks and retries. See the full provider list in the Portkey docs.

Get an API key

Create a key at app.portkey.ai and add it to your .env file. You will also need a gateway URL, which Portkey provides for self-hosted or hosted gateways:

PORTKEY_API_KEY=your-api-key
PORTKEY_GATEWAY_URL=your-gateway-url

Configuration

LibreChat requires the apiKey field, but Portkey does not use it, so pass the string dummy. Authentication happens through the x-portkey-* headers instead. There are two ways to connect, depending on whether you route by Virtual Keys or Configs. Add one of the following under endpoints.custom in your librechat.yaml.

Virtual Keys

    - name: "Portkey"
      apiKey: "dummy"
      baseURL: ${PORTKEY_GATEWAY_URL}
      headers:
        x-portkey-api-key: "${PORTKEY_API_KEY}"
        x-portkey-virtual-key: "PORTKEY_OPENAI_VIRTUAL_KEY"
      models:
        default: ["gpt-4o-mini"]
        fetch: true
      titleConvo: true
      titleModel: "current_model"
      summarize: false
      summaryModel: "current_model"
      modelDisplayLabel: "Portkey:OpenAI"
      iconURL: https://images.crunchbase.com/image/upload/c_pad,f_auto,q_auto:eco,dpr_1/rjqy7ghvjoiu4cd1xjbf

Configs

    - name: "Portkey"
      apiKey: "dummy"
      baseURL: ${PORTKEY_GATEWAY_URL}
      headers:
        x-portkey-api-key: "${PORTKEY_API_KEY}"
        x-portkey-config: "pc-libre-xxx"
      models:
        default: ["llama-3.2"]
        fetch: true
      titleConvo: true
      titleModel: "current_model"
      summarize: false
      summaryModel: "current_model"
      modelDisplayLabel: "Portkey:Llama"
      iconURL: https://images.crunchbase.com/image/upload/c_pad,f_auto,q_auto:eco,dpr_1/rjqy7ghvjoiu4cd1xjbf

Notes

Configs let you set model-specific parameters like top_p and max_tokens on the Portkey side. See the Configs docs.
Replace PORTKEY_OPENAI_VIRTUAL_KEY and pc-libre-xxx with the virtual key or config ID from your Portkey dashboard.