LibreChatHuggingface
Configure Huggingface as a custom endpoint in LibreChat.
Archived documentation
You are reading the frozen documentation for v0.8.7, which is no longer maintained. Go to the latest docs.
Huggingface exposes hosted models through an OpenAI-compatible inference API, which you can add to LibreChat as a custom endpoint.
Get an API key
Create a token at huggingface.co/settings/tokens. Add it to your .env file:
HUGGINGFACE_TOKEN=your-api-keyConfiguration
Add the endpoint under endpoints.custom in your librechat.yaml:
- name: 'HuggingFace'
apiKey: '${HUGGINGFACE_TOKEN}'
baseURL: 'https://api-inference.huggingface.co/v1'
models:
default: [
"codellama/CodeLlama-34b-Instruct-hf",
"google/gemma-1.1-2b-it",
"google/gemma-1.1-7b-it",
"HuggingFaceH4/starchat2-15b-v0.1",
"HuggingFaceH4/zephyr-7b-beta",
"meta-llama/Meta-Llama-3-8B-Instruct",
"microsoft/Phi-3-mini-4k-instruct",
"mistralai/Mistral-7B-Instruct-v0.1",
"mistralai/Mistral-7B-Instruct-v0.2",
"mistralai/Mixtral-8x7B-Instruct-v0.1",
"NousResearch/Nous-Hermes-2-Mixtral-8x7B-DPO",
]
fetch: true
titleConvo: true
titleModel: "NousResearch/Nous-Hermes-2-Mixtral-8x7B-DPO"
dropParams: ["top_p"]
modelDisplayLabel: "HuggingFace"The model list above was last updated on May 09, 2024.
Notes
- The listed models are free but rate limited, and answers can be very short on the free tier. Some models work better than others.
- Fetching the model list is not supported, so set the
defaultarray yourself. dropParams: ["top_p"]is required. Without it, requests fail because Huggingface rejects thetop_pparameter. SeedropParams.