# Token Usage (https://www.librechat.ai/docs/configuration/token_usage)

## Intro

As of `v0.6.0`, LibreChat accurately tracks token usage for supported endpoints. All token transactions are stored in the "Transactions" collection in your database. Current releases also show real-time context usage and cost in the conversation UI when enabled.

Currently, you can limit user token usage by enabling user balances. Instead of configuring token credit limits via environment variables, you now set these options in your `librechat.yaml` file under the **balance** section. Cost values are hidden by default and must be enabled with `interface.contextCost`.

## Viewing Context Usage and Cost

LibreChat shows a context gauge while a conversation is running. The gauge updates from usage events during streaming and can show:

- Current prompt/context usage against the model's context window
- A hover summary for quick token and cost details
- A color-coded meter that separates the token categories currently using the context window
- A click-through breakdown for prompt, completion, cached token usage, branch totals, and conversation totals
- For authorized administrators, a direct Langfuse session link when the conversation has a sampled trace in the active in-app connection

The detailed breakdown is collapsed by default so the categorized meter remains compact. Users can expand it when they need the totals and LibreChat remembers that choice for later conversations. Deferred tool definitions use the same category color with a hatched treatment, while estimates without a known category composition remain a single fill.

Usage breakdowns are persisted with messages and conversations. Reopened chats retain branch and total usage/cost details instead of relying only on the active streaming session.

When summarization compacts a long conversation, LibreChat records the compacted summary baseline and counts only post-summary turns on top of it for the context gauge. Usage and cost totals still include the full branch spend.

Admins can control these displays in `librechat.yaml`:

```yaml filename="librechat.yaml"
interface:
  contextUsage: true
  contextCost: true
  currency:
    code: EUR
    rate: 0.92
```

- `contextUsage` controls whether users see the context window and token usage gauge.
- `contextCost` controls whether users see cost values in the usage details. It defaults to `false`; set it to `true` to show costs.
- `currency` converts displayed USD costs using a static multiplier when cost display is enabled. Transactions are still recorded using LibreChat's token credit accounting.

## Custom Endpoint Token Config

For custom endpoints, define model-specific context windows and per-million-token rates with `endpoints.custom[].tokenConfig`:

```yaml filename="librechat.yaml"
endpoints:
  custom:
    - name: 'Mistral'
      apiKey: '${MISTRAL_API_KEY}'
      baseURL: 'https://api.mistral.ai/v1'
      models:
        default: ['mistral-large-latest']
      tokenConfig:
        mistral-large-latest:
          prompt: 2
          completion: 6
          context: 128000
```

`prompt`, `completion`, and `context` are required for each model entry. `cacheRead` and `cacheWrite` can be added for providers that report cached input usage. For Agents that use multiple endpoints, LibreChat uses the matching endpoint/model token config when recording usage and cost.

Fetched token config is cached with user scope when endpoint models, keys, URLs, or headers can vary by request context, so isolated custom endpoint pricing and context windows stay separated.

## Transaction Configuration

The transaction system controls whether token usage records are saved to the database. This can be configured separately from the balance system.

### Transaction Settings

```yaml filename="librechat.yaml"
version: 1.2.9

# Transaction settings
# Controls whether to save transaction records to the database
# Default is true (enabled)
transactions:
  enabled: false
```

**Important:** When `balance.enabled` is set to `true`, transaction recording is automatically enabled regardless of the `transactions.enabled` setting. This ensures that balance tracking functions correctly by maintaining a complete record of all token usage.

    Check out the [Transactions Configuration](/docs/configuration/librechat_yaml/object_structure/transactions) page for more details.

## Balance Configuration

The balance system in LibreChat allows administrators to configure how token credit balances are managed for users. All balance settings are now managed in your YAML configuration under the `balance` object.

**Note:** This replaces the previous environment variables (`CHECK_BALANCE` and `START_BALANCE`) and provides a more structured way to manage user balances.

### Complete Balance Settings

```yaml filename="librechat.yaml"
version: 1.3.5

# Balance settings
balance:
  enabled: true # Enable token credit balances for users
  startBalance: 20000 # Initial tokens credited upon registration
  autoRefillEnabled: false # Enable automatic token refills
  refillIntervalValue: 30 # Numerical value for refill interval
  refillIntervalUnit: 'days' # Time unit for refill interval (days, hours, etc.)
  refillAmount: 10000 # Tokens added during each refill
```

### Balance Settings Explained

- **enabled**: Activates token credit tracking and balance management for users. When set to `true`, the system will track token usage and enforce balance limits.

- **startBalance**: Specifies the initial number of tokens credited to a user upon registration. This is the starting balance for all new users.

- **autoRefillEnabled**: Determines whether automatic refilling of token credits is enabled. When set to `true`, the system will automatically add credits to user balances based on the refill interval.

- **refillIntervalValue**: Specifies the numerical value for the interval at which token credits are automatically refilled. Works in conjunction with `refillIntervalUnit`.

- **refillIntervalUnit**: Specifies the time unit for the refill interval. Supported values include "seconds", "minutes", "hours", "days", "weeks", and "months".

- **refillAmount**: Specifies the number of tokens to be added to the user's balance during each automatic refill.

Check out the [Balance Configuration](/docs/configuration/librechat_yaml/object_structure/balance) page for more details.

## How Auto-Refill Works

When a user's balance is tracked and **autoRefill** is enabled, the system will automatically add credits to the balance only when the specified time interval has passed since the last refill. This is achieved by comparing the current date with the `lastRefill` date plus the specified interval.

### Auto-Refill Process

1. When a user attempts to spend tokens, the system checks if the current balance is sufficient
2. If the balance would drop to zero or below after the transaction, the system checks if auto-refill is enabled
3. If auto-refill is enabled, the system checks if the time interval since the last refill has passed:
   - The system compares the current date with `lastRefill + refillInterval`
   - If the interval has passed, tokens are added to the user's balance
   - The `lastRefill` date is updated to the current date
4. The transaction proceeds if the balance is sufficient (either originally or after refill)

### Supported Time Units

The `refillIntervalUnit` can be set to any of the following values:

- "seconds"
- "minutes"
- "hours"
- "days"
- "weeks"
- "months"

For example, if `refillIntervalValue` is set to 30 and `refillIntervalUnit` is `days`, the system will add `refillAmount` tokens to the user's balance only if 30 days have passed since the last refill.

### Balance Synchronization

When a user logs in, the system automatically synchronizes their balance settings with the current global balance configuration. This ensures that any changes to the balance configuration are applied to all users.

The synchronization process:

1. Checks if the user has a balance record
2. If no record exists, creates one with the current `startBalance`
3. Updates the user's auto-refill settings to match the global configuration
4. Ensures the user's refill interval and amount match the global settings

## Managing Token Balances

You can manually add or set user balances. This is especially useful during development or if you plan to build out a full balance-accruing system in the future (for example, via an admin dashboard).

### Adding Balances

```bash filename="To manually add balances, run the following command:"
# Local Development
npm run add-balance

# Docker (default setup)
docker compose exec api npm run add-balance

# Docker (deployment setup)
docker exec -it LibreChat-API /bin/sh -c "cd .. && npm run add-balance"
```

```bash filename="You can also specify the email and token credit amount to add, e.g.:"
# Local Development
npm run add-balance danny@librechat.ai 1000

# Docker (default setup)
docker compose exec api npm run add-balance danny@librechat.ai 1000

# Docker (deployment setup)
docker exec -it LibreChat-API /bin/sh -c "cd .. && npm run add-balance danny@librechat.ai 1000"
```

### Setting Balances

Additionally, you can set a balance for a user. An existing balance will be overwritten by the new balance.

```bash filename="To manually set balances, run the following command:"
# Local Development
npm run set-balance

# Docker (default setup)
docker compose exec api npm run set-balance

# Docker (deployment setup)
docker exec -it LibreChat-API /bin/sh -c "cd .. && npm run set-balance"
```

```bash filename="You can also specify the email and token credit amount to set, e.g.:"
# Local Development
npm run set-balance danny@librechat.ai 1000

# Docker (default setup)
docker compose exec api npm run set-balance danny@librechat.ai 1000

# Docker (deployment setup)
docker exec -it LibreChat-API /bin/sh -c "cd .. && npm run set-balance danny@librechat.ai 1000"
```

### Listing of balances

```bash filename="To see the balances of your users, you can run:"
# Local Development
npm run list-balances

# Docker (default setup)
docker compose exec api npm run list-balances

# Docker (deployment setup)
docker exec -it LibreChat-API /bin/sh -c "cd .. && npm run list-balances"
```

This works well to track your own usage for personal use; 1000 credits = $0.001 (1 mill USD)

## Notes on Token Usage and Balance

- With summarization enabled, you will be blocked from making an API request if the cost of the content that you need to summarize + your messages payload exceeds the current balance
- Subagent child-run model usage is recorded against the parent transaction so parent agent runs include delegated usage in their totals.
- Counting Prompt tokens is really accurate for OpenAI calls, but not 100% for plugins (due to function calling). It is really close and conservative, meaning its count may be higher by 2-5 tokens.
- The system allows deficits incurred by the completion tokens. It only checks if you have enough for the prompt Tokens, and is pretty lenient with the completion. The graph below details the logic
- The above said, plugins are checked at each generation step, since the process works with multiple API calls. Anything the LLM has generated since the initial user prompt is shared to the user in the error message as seen below.
- There is a 150 token buffer for titling since this is a 2 step process, that averages around 200 total tokens. In the case of insufficient funds, the titling is cancelled before any spend happens and no error is thrown.

![image](https://github.com/danny-avila/LibreChat/assets/110412045/78175053-9c38-44c8-9b56-4b81df61049e)

## More details

source: [LibreChat/discussions/1640](https://github.com/danny-avila/LibreChat/discussions/1640#discussioncomment-8251970)

> "rawAmount": -000, // what's this?

Raw amount of tokens as counted per the tokenizer algorithm.

> "tokenValue": -00000, // what's this?

Token credits value. 1000 credits = $0.001 (1 mill USD)

> "rate": 00, // what's this?

The rate at which tokens are charged as credits.

For example, gpt-3.5-turbo-1106 has a rate of 1 for user prompt (input) and 2 for completion (output)

| Model              | Input               | Output              |
| ------------------ | ------------------- | ------------------- |
| gpt-3.5-turbo-1106 | $0.0010 / 1K tokens | $0.0020 / 1K tokens |

Given the provided example:

```sh
    "rawAmount": -137
    "tokenValue": -205.5
    "rate": 1.5
```

```math
\text{Token Value} = (\text{Raw Amount of Tokens}) \times (\text{Rate})
```

```math
137 \times 1.5 = 205.5
```

And to get the real amount of USD spend based on **Token Value**:

```math
\frac{\text{Token Value}}{1,000,000} = \left(\frac{\text{Raw Amount of Tokens} \times \text{Rate}}{1,000,000}\right)
```

```math
\frac{205.5}{1,000,000} = \$0.0002055 \text{ USD}
```

For custom endpoints, prefer `endpoints.custom[].tokenConfig` in `librechat.yaml` for per-model rates and context windows.

## Preview

![image](https://github.com/danny-avila/LibreChat/assets/110412045/39a1aa5d-f8fc-43bf-81f2-299e57d944bb)

![image](https://github.com/danny-avila/LibreChat/assets/110412045/e1b1cc3f-8981-4c7c-a5f8-e7badbc6f675)

## Additional Notes

- With summarization enabled, API requests are blocked if the cost of the content plus the messages payload exceeds the current balance.
- The system is lenient with completion tokens, focusing primarily on prompt tokens for balance checks.
- A buffer is added for titling (approximately 150 tokens) to account for the two-step process.
- Token credits translate to monetary value (e.g., 1000 credits = $0.001 USD).

For more details and customizations, please refer to the [LibreChat Documentation](https://www.librechat.ai/docs/configuration/librechat_yaml).