Integrate with AWS Bedrock

In this guide we show how to integrate a Digital Human with AWS Bedrock. With AWS Bedrock the Digital Human can get access to a range of LLM's and AI agents through a unified API.

In order to integrate with AWS Bedrock we will create a Conversational Plugin that will generate responses to user messages using an AWS Bedrock LLM. In this guide we are focusing on the AWS Bedrock integration, to learn more about how to create a plugin and setup the digital human refer to Create a custom plugin.

To implement this plugin we are going to use python and Langchain.

Requirements

In your python environment install the following dependencies

pip install fastapi
pip install "uvicorn[standard]"
pip install langchain
pip install boto3
pip install python-dotenv

Credentials

To access AWS Bedrock you will need credentials to access your AWS account with permissions to run the desired model. In this guide we are going to use the "amazon.titan-text-express-v1" model. Refer to the following AWS Bedrock model access docs to see how to activate the model access.

Once you have the permissions you should get the access keys for programmatic access to AWS services. Let's create a .env file at the root folder of the project and copy the keys there

AWS_ACCESS_KEY_ID=<your access key>
AWS_SECRET_ACCESS_KEY=<your secret access key>
AWS_SESSION_TOKEN=<your session token>
AWS_DEFAULT_REGION=<region to access bedrock e.g. eu-central-1> 

Model Prediction

As a first step, let's create a python file to test from command line that we are able to access the model predictions.

Create a main.py with the following content.

import os
from dotenv import find_dotenv
from dotenv import load_dotenv
from langchain_community.llms import Bedrock
from langchain.chains import ConversationChain

# we load the AWS credentials
load_dotenv(find_dotenv())

llm = Bedrock(
    credentials_profile_name="default", 
    model_id="amazon.titan-text-express-v1"
)
llm.model_kwargs = {'temperature': 0.5, "maxTokenCount": 700}

conversation = ConversationChain(
        llm=llm, verbose=True,
)

conversation.prompt.template = """
System: The following is a friendly conversation 
between a helpful assistant and a customer.\n\n User: {input}\nBot:
"""

response = conversation.predict(input=user_message)

print(response)