Web Search

LibreChat’s web search feature allows you to search the internet and retrieve relevant information to enhance your conversations. The feature consists of three main components that work together to provide comprehensive search results.

Quick Start

To get started with web search, you’ll need to configure API keys for all three components to ensure effective search results. You can do this in two ways:

Environment Variables (Recommended for admins):

# Search Provider (Required)
SERPER_API_KEY=your_serper_api_key
 
# Scraper (Required)
FIRECRAWL_API_KEY=your_firecrawl_api_key
# Optional: Custom Firecrawl API URL
FIRECRAWL_API_URL=your_firecrawl_api_url
 
# Reranker (Required)
JINA_API_KEY=your_jina_api_key
# or
COHERE_API_KEY=your_cohere_api_key

User Interface (If environment variables are not set):
- Users will be prompted to enter the required API keys when they first use the web search feature
- They can choose which reranker service to use (Jina or Cohere)

Obtaining API Keys

Each component of the web search feature requires its own API key. Here’s how to obtain them:

Search Provider: Serper

Visit Serper.dev
Sign up for an account
Navigate to the API Key section
Copy your API key
Set it in your environment variables or provide it through the UI

Scraper: Firecrawl

Visit Firecrawl.dev
Sign up for an account
Navigate to the API Key section
Copy your API key
Set it in your environment variables or provide it through the UI
(Optional) If you’re using a custom Firecrawl instance, you’ll also need to set the API URL

Rerankers

Jina

Visit Jina.ai
Sign up for an account
Navigate to the API Dashboard
Copy your API key
Set it in your environment variables or provide it through the UI

Cohere

Visit Cohere Dashboard
Sign up for an account
Navigate to the API Keys section
Copy your API key
Set it in your environment variables or provide it through the UI

Components

1. Search Providers

Search providers are responsible for performing the initial web search and returning relevant results.

Available Providers:

Serper: A Google Search API that provides high-quality search results
- Get your API key from Serper.dev

Planned Providers:

SearXNG: Open-source, self-hosted meta search engine
Additional third-party search services

2. Scrapers

Scrapers extract the actual content from web pages returned by the search provider.

Available Scrapers:

Firecrawl: A powerful web scraping service that extracts content from web pages
- Get your API key from Firecrawl.dev
- API URL is optional (defaults to Firecrawl’s hosted service)

Planned Scrapers:

Local Firecrawl: Self-hosted version of Firecrawl
Additional third-party scraping services

3. Rerankers

Rerankers analyze the scraped content to determine the most relevant parts and reorder them for better results.

Available Rerankers:

Jina: AI-powered reranking service
- Get your API key from Jina.ai
Cohere: Advanced reranking service
- Get your API key from Cohere Dashboard

Planned Rerankers:

RAG API: Open-source reranking using RAG (Retrieval-Augmented Generation)
Additional third-party reranking services

Configuration

Admin Configuration

Admins can configure the web search feature using environment variables. The YAML configuration allows you to specify custom environment variable names for each component.

⚠️ Important: Never put actual API keys or values in the YAML file (they won’t work)- only use environment variable names.

webSearch:
  # Search Provider Configuration
  serperApiKey: "${CUSTOM_SERPER_API_KEY}"  # ✅ Correct: Using environment variable name
  # serperApiKey: "sk-123..."               # ❌ Wrong: Never put actual API keys here
 
  # Scraper Configuration
  firecrawlApiKey: "${CUSTOM_FIRECRAWL_API_KEY}"
  firecrawlApiUrl: "${CUSTOM_FIRECRAWL_API_URL}"
  # firecrawlApiKey: "fc-123..."            # ❌ Wrong: Never put actual API keys here
  # firecrawlApiUrl: "https://..."          # ❌ Wrong: Never put actual URLs here
 
  # Reranker Configuration
  jinaApiKey: "${CUSTOM_JINA_API_KEY}"
  cohereApiKey: "${CUSTOM_COHERE_API_KEY}"
  # jinaApiKey: "jn-123..."                 # ❌ Wrong: Never put actual API keys here
  # cohereApiKey: "ch-123..."               # ❌ Wrong: Never put actual API keys here
 
  # General Settings
  safeSearch: 1 # Options: 0 (OFF), 1 (MODERATE - default), 2 (STRICT)

Note: The YAML configuration should only contain environment variable names (in the format ${VARIABLE_NAME}). This flexibility enables:

Using different variable names in different environments
Supporting multiple configurations for different user groups
Future integration with role-based configurations

If you want to restrict the system to use only specific services, you can specify the service types:

webSearch:
  # ... variable configurations ...
  searchProvider: "serper"    # Only use Serper for search
  scraperType: "firecrawl"    # Only use Firecrawl for scraping
  rerankerType: "jina"        # Only use Jina for reranking

User Configuration

If the admin hasn’t configured all the necessary API keys, users will be prompted to provide them through the UI. The interface allows users to:

Choose their preferred reranker (Jina or Cohere)
Enter API keys for the required services
Configure the Firecrawl API URL if needed (optional)

Usage

Once configured, you can use web search in two ways:

Chat Interface: Click the web search button in the chat interface to enable web search for your conversation
Agents: Use the web_search capability in agents to allow them to search the web

Notes

All three components (search provider, scraper, and reranker) are required for effective search results
The Firecrawl API URL is optional and defaults to their hosted service
Safe search provides three levels of content filtering: OFF (0), MODERATE (1 - default), and STRICT (2)
The scraper timeout is set to 7.5 seconds (7500ms) by default
API keys can be revoked at any time through the UI
Future updates will include more open-source, self-hosted options for all components
Additional customization options are planned, including:
- Control over the number of links to scrape
- Domain allowlist/blocklist for scraping
- Custom scraping rules and filters
- Advanced result filtering and ranking options
- Rate limiting and request throttling controls

Artifacts - Generative UI Image Generation