# Vultr Cloud Inference (https://www.librechat.ai/docs/configuration/librechat_yaml/ai_endpoints/vultrcloudinference)

Vultr Cloud Inference serves open models through an OpenAI-compatible API.

## Get an API key

Create a key from the [Vultr Cloud Inference](https://docs.vultr.com/vultr-cloud-inference) console. Add it to your `.env` file:

```bash filename=".env"
VULTRINFERENCE_TOKEN=your-api-key
```

## Configuration

Add the endpoint under `endpoints.custom` in your `librechat.yaml`:

```yaml filename="librechat.yaml"
    - name: 'Vultr Cloud Inference'
      apiKey: '${VULTRINFERENCE_TOKEN}'
      baseURL: 'https://api.vultrinference.com/v1/chat/completions'
      models:
        default: [
          "llama2-7b-chat-Q5_K_M.gguf",
          "llama2-13b-chat-Q5_K_M.gguf",
          "mistral-7b-Q5_K_M.gguf",
          "zephyr-7b-beta-Q5_K_M.gguf",
        ]
        fetch: true
      titleConvo: true
      titleModel: "llama2-7b-chat-Q5_K_M.gguf"
      modelDisplayLabel: "Vultr Cloud Inference"
```

## Notes

- The example lists four models optimized for chat, last updated June 28, 2024.
- Only `llama2-7b-chat-Q5_K_M.gguf` currently supports title generation.