# Huggingface (https://www.librechat.ai/docs/configuration/librechat_yaml/ai_endpoints/huggingface) Huggingface exposes hosted models through an OpenAI-compatible inference API, which you can add to LibreChat as a custom endpoint. ## Get an API key Create a token at [huggingface.co/settings/tokens](https://huggingface.co/settings/tokens). Add it to your `.env` file: ```bash filename=".env" HUGGINGFACE_TOKEN=your-api-key ``` ## Configuration Add the endpoint under `endpoints.custom` in your `librechat.yaml`: ```yaml filename="librechat.yaml" - name: 'HuggingFace' apiKey: '${HUGGINGFACE_TOKEN}' baseURL: 'https://api-inference.huggingface.co/v1' models: default: [ "codellama/CodeLlama-34b-Instruct-hf", "google/gemma-1.1-2b-it", "google/gemma-1.1-7b-it", "HuggingFaceH4/starchat2-15b-v0.1", "HuggingFaceH4/zephyr-7b-beta", "meta-llama/Meta-Llama-3-8B-Instruct", "microsoft/Phi-3-mini-4k-instruct", "mistralai/Mistral-7B-Instruct-v0.1", "mistralai/Mistral-7B-Instruct-v0.2", "mistralai/Mixtral-8x7B-Instruct-v0.1", "NousResearch/Nous-Hermes-2-Mixtral-8x7B-DPO", ] fetch: true titleConvo: true titleModel: "NousResearch/Nous-Hermes-2-Mixtral-8x7B-DPO" dropParams: ["top_p"] modelDisplayLabel: "HuggingFace" ``` The model list above was last updated on May 09, 2024. ## Notes - The listed models are free but rate limited, and answers can be very short on the free tier. Some models work better than others. - Fetching the model list is not supported, so set the `default` array yourself. - `dropParams: ["top_p"]` is required. Without it, requests fail because Huggingface rejects the `top_p` parameter. See [`dropParams`](/docs/configuration/librechat_yaml/object_structure/custom_endpoint#dropparams). Here's a list of the other models that were tested along with their corresponding errors ```yaml models: default: [ "CohereForAI/c4ai-command-r-plus", # Model requires a Pro subscription "HuggingFaceH4/zephyr-orpo-141b-A35b-v0.1", # Model requires a Pro subscription "meta-llama/Llama-2-7b-hf", # Model requires a Pro subscription "meta-llama/Meta-Llama-3-70B-Instruct", # Model requires a Pro subscription "meta-llama/Llama-2-13b-chat-hf", # Model requires a Pro subscription "meta-llama/Llama-2-13b-hf", # Model requires a Pro subscription "meta-llama/Llama-2-70b-chat-hf", # Model requires a Pro subscription "meta-llama/Llama-2-7b-chat-hf", # Model requires a Pro subscription "------", "bigcode/octocoder", # template not found "bigcode/santacoder", # template not found "bigcode/starcoder2-15b", # template not found "bigcode/starcoder2-3b", # template not found "codellama/CodeLlama-13b-hf", # template not found "codellama/CodeLlama-7b-hf", # template not found "google/gemma-2b", # template not found "google/gemma-7b", # template not found "HuggingFaceH4/starchat-beta", # template not found "HuggingFaceM4/idefics-80b-instruct", # template not found "HuggingFaceM4/idefics-9b-instruct", # template not found "HuggingFaceM4/idefics2-8b", # template not found "kashif/stack-llama-2", # template not found "lvwerra/starcoderbase-gsm8k", # template not found "tiiuae/falcon-7b", # template not found "timdettmers/guanaco-33b-merged", # template not found "------", "bigscience/bloom", # 404 status code (no body) "------", "google/gemma-2b-it", # stream` is not supported for this model / unknown error "------", "google/gemma-7b-it", # AI Response error likely caused by Google censor/filter "------", "bigcode/starcoder", # Service Unavailable "google/flan-t5-xxl", # Service Unavailable "HuggingFaceH4/zephyr-7b-alpha", # Service Unavailable "mistralai/Mistral-7B-v0.1", # Service Unavailable "OpenAssistant/oasst-sft-1-pythia-12b", # Service Unavailable "OpenAssistant/oasst-sft-4-pythia-12b-epoch-3.5", # Service Unavailable ] ```