LibreChatAnyscale
Configure Anyscale as a custom endpoint in LibreChat.
Anyscale Endpoints serves open models such as Llama through an OpenAI-compatible API that you can add to LibreChat as a custom endpoint.
Get an API key
Create a key from your Anyscale credentials. Add it to your .env file:
ANYSCALE_API_KEY=your-api-keyConfiguration
Add the endpoint under endpoints.custom in your librechat.yaml:
- name: "Anyscale"
apiKey: "${ANYSCALE_API_KEY}"
baseURL: "https://api.endpoints.anyscale.com/v1"
models:
default: [
"meta-llama/Llama-2-7b-chat-hf",
]
fetch: true
titleConvo: true
titleModel: "meta-llama/Llama-2-7b-chat-hf"
summarize: false
summaryModel: "meta-llama/Llama-2-7b-chat-hf"
modelDisplayLabel: "Anyscale"Notes
- With
fetch: true, LibreChat loads the available model list from Anyscale, so thedefaultarray is only the initial selection.
How is this guide?