TrueFoundry AI Gateway

TrueFoundry AI Gateway is an enterprise proxy layer between your applications and LLM providers, giving access to 1000+ models through a unified OpenAI-compatible interface with built-in observability and governance.

Get an API key

Authenticate with a Personal Access Token. Generate one by following the PAT guide, then add it along with your gateway URL to your .env file:

TRUEFOUNDRY_API_KEY=your-api-key
TRUEFOUNDRY_GATEWAY_URL=your-gateway-url

Get the gateway URL and model names from the unified code snippet in the TrueFoundry console. Use the same model names you added to the gateway. The quick start guide walks through setup.

Configuration

Add the endpoint to the custom list under endpoints in your librechat.yaml. If your file already has an endpoints.custom list, paste only the - name: entry into it, keeping the same indentation:

endpoints:
  custom:
    - name: "TrueFoundry"
      apiKey: "${TRUEFOUNDRY_API_KEY}"
      baseURL: "${TRUEFOUNDRY_GATEWAY_URL}"
      models:
        default: ["openai-main/gpt-4o-mini", "openai-main/gpt-4o"]
        fetch: true
      titleConvo: true
      titleModel: "current_model"
      modelDisplayLabel: "TrueFoundry:OpenAI"

Notes

• Model names are namespaced by the integration you configured in the gateway, such as openai-main/gpt-4o. Match them to the names shown in your TrueFoundry console.
• For more detail, see the TrueFoundry docs.