Google

For the Google Endpoint, you can either use the Generative Language API (for Gemini models), or the Vertex AI API (for Gemini, PaLM2 & Codey models).

The Generative Language API uses an API key, which you can get from Google AI Studio.

For Vertex AI, you need a Service Account JSON key file, with appropriate access configured.

Instructions for both are given below.

Generative Language API (Gemini)

See here for Gemini API pricing and rate limits

⚠️ While Google models are free, they are using your input/output to help improve the model, with data de-identified from your Google Account and API key. ⚠️ During this period, your messages “may be accessible to trained reviewers.”

To use Gemini models through Google AI Studio, you'll need an API key. If you don't already have one, create a key in Google AI Studio.

Get an API key here: aistudio.google.com

Once you have your key, provide the key in your .env file, which allows all users of your instance to use it.

GOOGLE_KEY=mY_SeCreT_w9347w8_kEY

Or, you can make users provide it from the frontend by setting the following:

GOOGLE_KEY=user_provided

Some reverse proxies do not support the X-goog-api-key header. You can configure LibreChat to use the Authorization header instead:

GOOGLE_AUTH_HEADER=true

Since fetching the models list isn't yet supported, you should set the models you want to use in the .env file.

For your convenience, these are the latest models as of 5/18/24 that can be used with the Generative Language API:

GOOGLE_MODELS=gemini-1.5-flash-latest,gemini-1.0-pro,gemini-1.0-pro-001,gemini-1.0-pro-latest,gemini-1.0-pro-vision-latest,gemini-1.5-pro-latest,gemini-pro,gemini-pro-vision

Setting GOOGLE_KEY=user_provided in your .env file sets both the Vertex AI Service Account JSON key file and the Generative Language API key to be provided from the frontend like so:

Vertex AI

See here for Vertex API pricing and rate limits

To setup Google LLMs (via Google Cloud Vertex AI), first, signup for Google Cloud: cloud.google.com

You can usually get $300 starting credit, which makes this option free for 90 days.

1. Once signed up, Enable the Vertex AI API on Google Cloud:

• Go to Vertex AI page on Google Cloud console
• Click on Enable API if prompted

2. Create a Service Account with Vertex AI role:

• Click here to create a Service Account
• Select or create a project
• Enter a service account ID (required), name and description are optional
  • 
• Click on "Create and Continue" to give at least the "Vertex AI User" role
  • 
• Click on "Continue/Done"

3. Create a JSON key to Save in your Project Directory:

• Go back to the Service Accounts page
• Select your service account
• Click on "Keys"
  • 
• Click on "Add Key" and then "Create new key"
  • 
• Choose JSON as the key type and click on "Create"
• Download the key file and rename it as 'auth.json'
• Save it within the project directory, in /api/data/
  • 

Saving your JSON key file in the project directory which allows all users of your LibreChat instance to use it.

Alternatively, you can make users provide it from the frontend by setting the following:

# Note: this configures both the Vertex AI Service Account JSON key file
# and the Generative Language API key to be provided from the frontend.
GOOGLE_KEY=user_provided

You can also specify the service account key file using the GOOGLE_SERVICE_KEY_FILE environment variable:

# Path to the service account JSON key file
GOOGLE_SERVICE_KEY_FILE=/path/to/auth.json
 
# Or provide as a URL
GOOGLE_SERVICE_KEY_FILE=https://example.com/path/to/auth.json
 
# Or provide as stringified JSON
GOOGLE_SERVICE_KEY_FILE='{"type":"service_account","project_id":"your-project",...}'
 
# Or provide as base64 encoded JSON
GOOGLE_SERVICE_KEY_FILE=eyJ0eXBlIjogInNlcnZpY2VfYWNjb3VudCIsICJwcm9qZWN0X2lkIjogInlvdXItcHJvamVjdC1pZCIsIC4uLn0=

This is particularly useful for features that require Vertex AI authentication, such as OCR capabilities.

You can also specify the Google Cloud location for Vertex AI API requests:

# Google Cloud region for Vertex AI
GOOGLE_LOC=us-central1
 
# Alternative region for Gemini Image Generation (defaults to global)
GOOGLE_CLOUD_LOCATION=global

Since fetching the models list isn't yet supported, you should set the models you want to use in the .env file.

For your convenience, these are the latest models as of 5/18/24 that can be used with the Generative Language API:

GOOGLE_MODELS=gemini-1.5-flash-preview-0514,gemini-1.5-pro-preview-0514,gemini-1.0-pro-vision-001,gemini-1.0-pro-002,gemini-1.0-pro-001,gemini-pro-vision,gemini-1.0-pro

version: '3.4'
 
services:
  api:
    volumes:
      - type: bind
        source: ./api/data/auth.json
        target: /app/api/data/auth.json

Google Safety Settings

To set safety settings for both Vertex AI and Generative Language API, you can set the following in your .env file:

GOOGLE_SAFETY_SEXUALLY_EXPLICIT=BLOCK_ONLY_HIGH
GOOGLE_SAFETY_HATE_SPEECH=BLOCK_ONLY_HIGH
GOOGLE_SAFETY_HARASSMENT=BLOCK_ONLY_HIGH
GOOGLE_SAFETY_DANGEROUS_CONTENT=BLOCK_ONLY_HIGH
GOOGLE_SAFETY_CIVIC_INTEGRITY=BLOCK_ONLY_HIGH

You can also exclude safety settings by setting the following in your .env file, which will use the provider defaults. This can be helpful if you are having issues with specific safety settings.

GOOGLE_EXCLUDE_SAFETY_SETTINGS=true

NOTE: You do not have access to the BLOCK_NONE setting by default. To use this restricted HarmBlockThreshold setting, you will need to either:

• (a) Get access through an allowlist via your Google account team
• (b) Switch your account type to monthly invoiced billing following this instruction: https://cloud.google.com/billing/docs/how-to/invoiced-billing

Notes:

• Google endpoint supports all Shared Endpoint Settings via the librechat.yaml configuration file, including streamRate, titleModel, titleMethod, titlePrompt, titlePromptTemplate, and titleEndpoint