# Web Search (https://www.librechat.ai/docs/features/web_search)

LibreChat's web search feature allows you to search the internet and retrieve relevant information to enhance your conversations. The feature consists of three main components that work together to provide comprehensive search results.

## Quick Start

To get started with web search, configure a search provider and scraper. Most providers require API keys; Keenable works without one for both search and page fetch, with an optional key to raise its public rate limits. Reranking can use Jina or Cohere, or be disabled with `rerankerType: "none"`. You can configure the stack in two ways:

<Callout type="warning" title="Private self-hosted endpoints">
  Web search, scrape, and rerank connections block private, loopback, link-local, and cloud-metadata destinations by default. If SearXNG, Firecrawl, Jina, or another configured provider endpoint is private, add its exact host and port to [`webSearch.allowedAddresses`](/docs/configuration/librechat_yaml/object_structure/web_search#ssrf-protection-and-private-providers).
</Callout>

1. **Environment Variables** (Recommended for admins):
   ```bash
   # Search Provider (choose one)
   SERPER_API_KEY=your_serper_api_key
   # or
   SEARXNG_INSTANCE_URL=your_searxng_instance_url
   SEARXNG_API_KEY=your_searxng_api_key  # Optional
   # or
   TAVILY_API_KEY=your_tavily_api_key
   # or use Keenable keyless; this optional key raises public rate limits
   KEENABLE_API_KEY=your_keenable_api_key

   # Scraper (choose one)
   FIRECRAWL_API_KEY=your_firecrawl_api_key
   # Optional: Custom Firecrawl API URL
   FIRECRAWL_API_URL=your_firecrawl_api_url
   # Optional: Firecrawl API version (v0 or v1)
   # FIRECRAWL_VERSION=v1
   # or
   TAVILY_API_KEY=your_tavily_api_key
   # or use Keenable keyless with the same optional key

   # Reranker (Optional - choose one, or set rerankerType: "none")
   JINA_API_KEY=your_jina_api_key
   # Optional: Custom Jina API URL
   JINA_API_URL=your_jina_api_url
   # or
   COHERE_API_KEY=your_cohere_api_key
   ```

   Using Keenable without an environment key still requires provider selection in `librechat.yaml`:

   ```yaml filename="librechat.yaml"
   webSearch:
     searchProvider: keenable
     scraperProvider: keenable
     rerankerType: none
   ```

   See the [full Keenable configuration](/docs/configuration/librechat_yaml/object_structure/web_search#keenablescraperoptions) for optional search, fetch, and rate-limit settings.

2. **User Interface** (If environment variables are not set):
   - Users will be prompted to enter the required API keys when they first use the web search feature
   - They can choose which search provider (Serper, SearXNG, Tavily, or Keenable), scraper, and reranker service (Jina, Cohere, or none) to use

## Obtaining API Keys

Each enabled external service requires its own API key. Here's how to obtain them:

### Search Providers

#### Serper
1. Visit [Serper.dev](https://serper.dev)
2. Sign up for an account
3. Navigate to the API Key section
4. Copy your API key
5. Set it in your environment variables or provide it through the UI

#### SearXNG
1. Follow the setup instructions in the [Web Search Configuration](/docs/configuration/librechat_yaml/object_structure/web_search#setting-up-searxng) documentation
2. Set `SEARXNG_INSTANCE_URL` to your instance URL
3. Optionally set `SEARXNG_API_KEY` if your instance requires authentication
4. Add your instance's exact `host:port` to [`allowedAddresses`](/docs/configuration/librechat_yaml/object_structure/web_search#ssrf-protection-and-private-providers) if it is on a private or loopback address, otherwise LibreChat blocks the connection
5. Optionally tune which engines your instance queries with [`searxngSearchOptions`](/docs/configuration/librechat_yaml/object_structure/web_search#searxngsearchoptions). The default set is only three engines and includes DuckDuckGo, which serves CAPTCHAs to most self-hosted instances, so widening it helps if searches come back empty

#### Tavily
1. Visit [Tavily](https://app.tavily.com/home)
2. Sign up for an account
3. Copy your API key
4. Set `TAVILY_API_KEY` in your environment variables or provide it through the UI
5. Tavily can be used as both a search provider and a scraper provider

#### Keenable

1. Select Keenable as the search provider, scraper, or both
2. No API key is required for the public search and fetch endpoints
3. Optionally set `KEENABLE_API_KEY` to raise the public rate limits
4. Use `KEENABLE_API_URL` only to override the search endpoint and `KEENABLE_FETCH_URL` only to override the page-fetch endpoint

### Scraper: Firecrawl

1. Visit [Firecrawl.dev](https://docs.firecrawl.dev/introduction#api-key)
2. Sign up for an account
3. Navigate to the API Key section
4. Copy your API key
5. Set it in your environment variables or provide it through the UI
6. (Optional) If you're using a custom Firecrawl instance, you'll also need to set the API URL

### Rerankers

#### Jina
1. Visit [Jina.ai](https://jina.ai/api-dashboard/)
2. Sign up for an account
3. Navigate to the API Dashboard
4. Copy your API key
5. Set it in your environment variables or provide it through the UI

#### Cohere
1. Visit [Cohere Dashboard](https://dashboard.cohere.com/welcome/login)
2. Sign up for an account
3. Navigate to the API Keys section
4. Copy your API key
5. Set it in your environment variables or provide it through the UI

## Components

### 1. Search Providers

Search providers are responsible for performing the initial web search and returning relevant results.

**Available Providers:**
- **Serper**: A Google Search API that provides high-quality search results
  - Get your API key from [Serper.dev](https://serper.dev/api-keys)
- **SearXNG**: Open-source, self-hosted meta search engine
  - Self-host your own instance
  - Privacy-focused search results
  - Configurable engines, result language, time range, and request timeout
- **Tavily**: AI-optimized search API
  - Get your API key from [Tavily](https://app.tavily.com/home)
  - Supports configurable search depth, topic filtering, domain filtering, and more
  - Can also serve as a scraper provider
- **Keenable**: Keyless web search with optional higher-rate authentication
  - Supports domain-restricted search, result limits, attribution titles, and request timeouts
  - Can also serve as a scraper provider

### 2. Scrapers

Scrapers extract the actual content from web pages returned by the search provider.

**Available Scrapers:**
- **Firecrawl**: A powerful web scraping service that extracts content from web pages
  - Get your API key from [Firecrawl.dev](https://docs.firecrawl.dev/introduction#api-key)
  - API URL is optional (defaults to Firecrawl's hosted service)

- **Tavily**: Batch URL extraction via Tavily Extract API
  - Uses the same `TAVILY_API_KEY` as the search provider
  - Supports configurable extract depth, image extraction, and favicon extraction

- **Keenable**: Keyless page fetch
  - Uses the same optional API key as Keenable search
  - Supports attribution titles and request timeouts

**Planned Scrapers:**
- **Local Firecrawl**: Self-hosted version of Firecrawl
- Additional third-party scraping services

### 3. Rerankers

Rerankers analyze the scraped content to determine the most relevant parts and reorder them for better results.

**Available Rerankers:**
- **Jina**: AI-powered reranking service
  - Get your API key from [Jina.ai](https://jina.ai/api-dashboard/)
  - API URL is optional (defaults to Jina's hosted service)
- **Cohere**: Advanced reranking service
  - Get your API key from [Cohere Dashboard](https://dashboard.cohere.com/welcome/login)
- **None**: Skips reranking when `rerankerType` is set to `"none"`

**Planned Rerankers:**
- **RAG API**: Open-source reranking using RAG (Retrieval-Augmented Generation)
- Additional third-party reranking services

## Configuration

### Admin Configuration

Admins can configure the web search feature using environment variables. The YAML configuration allows you to specify custom environment variable names for each component.

⚠️ **Important: Never put actual API keys or values in the YAML file (they won't work)- only use environment variable names.**

```yaml
webSearch:
  # Search Provider Configuration
  serperApiKey: "${CUSTOM_SERPER_API_KEY}"  # ✅ Correct: Using environment variable name
  # serperApiKey: "sk-123..."               # ❌ Wrong: Never put actual API keys here
  # or
  searxngInstanceUrl: "${CUSTOM_SEARXNG_INSTANCE_URL}"  # ✅ Correct: Using environment variable name
  searxngApiKey: "${CUSTOM_SEARXNG_API_KEY}"            # ✅ Correct: Using environment variable name
  # searxngInstanceUrl: "http://..."        # ❌ Wrong: Never put actual URLs here
  # searxngApiKey: "sk-123..."              # ❌ Wrong: Never put actual API keys here
  searxngSearchOptions:                     # Query options, not secrets, so real values belong here
    engines: "google,bing,startpage"
    language: "en"

  # Tavily Configuration (search and/or scraper)
  tavilyApiKey: "${CUSTOM_TAVILY_API_KEY}"
  tavilySearchUrl: "${CUSTOM_TAVILY_SEARCH_URL}"
  tavilyExtractUrl: "${CUSTOM_TAVILY_EXTRACT_URL}"

  # Keenable Configuration (search and/or scraper; keyless by default)
  keenableApiKey: "${CUSTOM_KEENABLE_API_KEY}" # Optional; raises rate limits
  keenableApiUrl: "${CUSTOM_KEENABLE_API_URL}" # Optional search URL override
  keenableSearchOptions:
    maxResults: 8
    site: "example.com"
    attributionTitle: "LibreChat"
    timeout: 15000
  keenableScraperOptions:
    attributionTitle: "LibreChat"
    timeout: 15000

  # Scraper Configuration
  firecrawlApiKey: "${CUSTOM_FIRECRAWL_API_KEY}"
  firecrawlApiUrl: "${CUSTOM_FIRECRAWL_API_URL}"
  # firecrawlApiKey: "fc-123..."            # ❌ Wrong: Never put actual API keys here
  # firecrawlApiUrl: "https://..."          # ❌ Wrong: Never put actual URLs here

  # Reranker Configuration
  jinaApiKey: "${CUSTOM_JINA_API_KEY}"
  jinaApiUrl: "${CUSTOM_JINA_API_URL}"
  cohereApiKey: "${CUSTOM_COHERE_API_KEY}"
  # jinaApiKey: "jn-123..."                 # ❌ Wrong: Never put actual API keys here
  # jinaApiUrl: "https://..."               # ❌ Wrong: Never put actual URLs here
  # cohereApiKey: "ch-123..."               # ❌ Wrong: Never put actual API keys here

  # General Settings
  safeSearch: 1 # Options: 0 (OFF), 1 (MODERATE - default), 2 (STRICT)
```

**Note:** The YAML configuration should only contain environment variable names (in the format `${VARIABLE_NAME}`). This flexibility enables:
- Using different variable names in different environments
- Supporting multiple configurations for different user groups
- Future integration with role-based configurations

If you want to restrict the system to use only specific services, you can specify the service types:

```yaml
webSearch:
  # ... variable configurations ...
  searchProvider: "serper"    # Only use Serper for search
  # searchProvider: "searxng" # Only use SearXNG for search
  # searchProvider: "tavily"  # Only use Tavily for search
  # searchProvider: "keenable" # Use keyless Keenable search
  scraperProvider: "firecrawl"    # Only use Firecrawl for scraping
  # scraperProvider: "tavily" # Only use Tavily for scraping
  # scraperProvider: "keenable" # Use keyless Keenable page fetch
  rerankerType: "jina"        # Options: "jina", "cohere", "none"
```

### User Configuration

If the admin hasn't configured all the necessary API keys, users will be prompted to provide them through the UI. The interface allows users to:

1. Choose their preferred reranker (Jina, Cohere, or none)
2. Enter API keys for the required services
3. Configure the Firecrawl API URL if needed (optional)
4. Configure the Jina API URL if needed (optional)
5. Configure Tavily or Keenable endpoint overrides if needed (optional)

## Usage

Once configured, you can use web search in two ways:

1. **Chat Interface**: Click the web search button in the chat interface to enable web search for your conversation
2. **Agents**: Use the `web_search` capability in agents to allow them to search the web

## Notes

- Search provider and scraper configuration are required; reranking can be disabled with `rerankerType: "none"`
- The Firecrawl API URL is optional and defaults to their hosted service
- The Jina API URL is optional and defaults to their hosted service
- The Tavily Search and Extract API URLs are optional and default to Tavily's hosted services
- Keenable search and page fetch are keyless by default; an optional key raises public rate limits
- `KEENABLE_API_URL` overrides only search, while `KEENABLE_FETCH_URL` overrides only page fetch
- Pair `searchProvider: "keenable"`, `scraperProvider: "keenable"`, and `rerankerType: "none"` for a fully keyless stack
- Safe search provides three levels of content filtering: OFF (0), MODERATE (1 - default), and STRICT (2)
- Tavily does not inherit the global `safeSearch` setting by default; use `tavilySearchOptions.safeSearch` only if your Tavily account supports `safe_search`
- The scraper timeout is set to 7.5 seconds (7500ms) by default
- API keys can be revoked at any time through the UI
- Future updates will include more open-source, self-hosted options for all components
- Additional customization options are planned, including:
  - Control over the number of links to scrape
  - Domain allowlist/blocklist for scraping
  - Custom scraping rules and filters
  - Advanced result filtering and ranking options
  - Rate limiting and request throttling controls