Portkey AI

Portkey is an AI gateway that fronts 250+ models through an OpenAI-compatible endpoint, adding observability, 50+ guardrails, caching, and conditional routing with fallbacks and retries. See the full provider list in the Portkey docs.

Get an API key

Create a key at app.portkey.ai and add it to your .env file. You will also need a gateway URL, which Portkey provides for self-hosted or hosted gateways:

PORTKEY_API_KEY=your-api-key
PORTKEY_GATEWAY_URL=your-gateway-url

Configuration

LibreChat requires the apiKey field, but Portkey does not use it, so pass the string dummy. Authentication happens through the x-portkey-* headers instead. There are two ways to connect, depending on whether you route by Virtual Keys or Configs. Add one of the following to the custom list under endpoints in your librechat.yaml. If your file already has an endpoints.custom list, paste only the - name: entry into it, keeping the same indentation.

Virtual Keys

endpoints:
  custom:
    - name: "Portkey"
      apiKey: "dummy"
      baseURL: ${PORTKEY_GATEWAY_URL}
      headers:
        x-portkey-api-key: "${PORTKEY_API_KEY}"
        x-portkey-virtual-key: "PORTKEY_OPENAI_VIRTUAL_KEY"
      models:
        default: ["gpt-4o-mini"]
        fetch: true
      titleConvo: true
      titleModel: "current_model"
      modelDisplayLabel: "Portkey:OpenAI"
      iconURL: https://images.crunchbase.com/image/upload/c_pad,f_auto,q_auto:eco,dpr_1/rjqy7ghvjoiu4cd1xjbf

Configs

endpoints:
  custom:
    - name: "Portkey"
      apiKey: "dummy"
      baseURL: ${PORTKEY_GATEWAY_URL}
      headers:
        x-portkey-api-key: "${PORTKEY_API_KEY}"
        x-portkey-config: "pc-libre-xxx"
      models:
        default: ["llama-3.2"]
        fetch: true
      titleConvo: true
      titleModel: "current_model"
      modelDisplayLabel: "Portkey:Llama"
      iconURL: https://images.crunchbase.com/image/upload/c_pad,f_auto,q_auto:eco,dpr_1/rjqy7ghvjoiu4cd1xjbf

Notes

Configs let you set model-specific parameters like top_p and max_tokens on the Portkey side. See the Configs docs.
Replace PORTKEY_OPENAI_VIRTUAL_KEY and pc-libre-xxx with the virtual key or config ID from your Portkey dashboard.