AWS Bedrock Object Structure

Integrating AWS Bedrock with your application allows you to seamlessly utilize multiple AI models hosted on AWS. This section details how to configure the AWS Bedrock endpoint for your needs.

Example Configuration

Example AWS Bedrock Object Structure

endpoints:
  bedrock:
    titleModel: "anthropic.claude-3-haiku-20240307-v1:0"
    streamRate: 35
    availableRegions:
      - "us-east-1"
      - "us-west-2"
    guardrailConfig:
      guardrailIdentifier: "your-guardrail-id"
      guardrailVersion: "1"
      trace: "enabled"

Note: AWS Bedrock endpoint supports all Shared Endpoint Settings, including streamRate, titleModel, titleMethod, titlePrompt, titlePromptTemplate, and titleEndpoint. The settings shown below are specific to Bedrock or have Bedrock-specific defaults.

titleModel

Key:

Key	Type	Description	Example
titleModel	String	Specifies the model to use for generating conversation titles.	Recommended: anthropic.claude-3-haiku-20240307-v1:0. Set to "current_model" to use the same model as the chat.

Default: Not specified

Example:

titleModel

titleModel: "anthropic.claude-3-haiku-20240307-v1:0"

streamRate

Key:

Key	Type	Description	Example
streamRate	Number	Sets the rate of processing each new token in milliseconds.	This can help stabilize processing of concurrent requests and provide smoother frontend stream rendering.

Default: Not specified

Example:

streamRate

streamRate: 35

availableRegions

Key:

Key	Type	Description	Example
availableRegions	Array	Specifies the AWS regions you want to make available for Bedrock.	If provided, users will see a dropdown to select the region. If not selected, the default region is used.

Default: Not specified

Example:

availableRegions

availableRegions:
  - "us-east-1"
  - "us-west-2"

guardrailConfig

Key:

Key	Type	Description	Example
guardrailConfig	Object	Configuration for AWS Bedrock Guardrails to filter and moderate model inputs and outputs.	Optional. When configured, all Bedrock requests will be validated against the specified guardrail.

Sub-keys:

Key	Type	Description	Example
guardrailIdentifier	String	The unique identifier of the guardrail to apply.	Required when using guardrails.
guardrailVersion	String	The version of the guardrail to use.	Required when using guardrails.
trace	String	Controls guardrail trace output for debugging. Options: "enabled", "enabled_full", or "disabled".	Optional. Default: "disabled"

Example:

guardrailConfig

endpoints:
  bedrock:
    guardrailConfig:
      guardrailIdentifier: "abc123xyz"
      guardrailVersion: "1"
      trace: "enabled"

Notes:

Guardrails help ensure responsible AI usage by filtering harmful content, PII, and other sensitive information
The guardrailIdentifier can be found in the AWS Bedrock console under Guardrails
Set trace to "enabled" or "enabled_full" during development to see which guardrail policies are triggered
For production, set trace to "disabled" to reduce response payload size

Notes

The main configuration for AWS Bedrock is done through environment variables, additional forms of authentication are in development.

Azure OpenAI Anthropic Vertex AI