LibreChatAnyscale
Configure Anyscale as a custom endpoint in LibreChat.
Archived documentation
You are reading the frozen documentation for v0.8.7, which is no longer maintained. Go to the latest docs.
Anyscale Endpoints serves open models such as Llama through an OpenAI-compatible API that you can add to LibreChat as a custom endpoint.
Get an API key
Create a key from your Anyscale credentials. Add it to your .env file:
ANYSCALE_API_KEY=your-api-keyConfiguration
Add the endpoint under endpoints.custom in your librechat.yaml:
- name: "Anyscale"
apiKey: "${ANYSCALE_API_KEY}"
baseURL: "https://api.endpoints.anyscale.com/v1"
models:
default: [
"meta-llama/Llama-2-7b-chat-hf",
]
fetch: true
titleConvo: true
titleModel: "meta-llama/Llama-2-7b-chat-hf"
summarize: false
summaryModel: "meta-llama/Llama-2-7b-chat-hf"
modelDisplayLabel: "Anyscale"Notes
- With
fetch: true, LibreChat loads the available model list from Anyscale, so thedefaultarray is only the initial selection.