Amazon Bedrock FAQs

Amazon Bedrock Agents are fully managed capabilities that make it easier for developers to create generative AI–based applications that can complete complex tasks for a wide range of use cases and deliver up-to-date answers based on proprietary knowledge sources. In just a few short steps, Amazon Bedrock Agents automatically break down tasks and create an orchestration plan–without any manual coding. The agent securely connects to company data through an API, automatically converting data into a machine-readable format, and augmenting the request with relevant information to generate the most accurate response. Agents can then automatically call APIs to fulfill a user’s request. For example, a manufacturing company might want to develop a generative AI application that automates tracking inventory levels, sales data, supply chain information and that can recommend optimal reorder points and quantities to maximize efficiency. As fully managed capabilities, Amazon Bedrock Agents remove the undifferentiated lifting of managing system integration and infrastructure provisioning, allowing developers to use generative AI to its full extent throughout their organization.

You can securely connect FMs to your company data sources using Amazon Bedrock Agents. With a knowledge base, you can use agents to give FMs in Amazon Bedrock access to additional data that helps the model generate more relevant, context-specific, and accurate responses without continually retraining the FM. Based on user input, agents identify the appropriate knowledge base, retrieve the relevant information, and add the information to the input prompt, giving the model more context information to generate a completion.

Amazon Bedrock Agents can help you increase productivity, improve your customer service experience, and automate workflows (such as processing insurance claims).

With agents, developers have seamless support for monitoring, encryption, user permissions, versioning, and API invocation management without writing custom code. Amazon Bedrock Agents automate the prompt engineering and orchestration of user-requested tasks. Developers can use the agent-created prompt template as a baseline to further refine it for an enhanced user experience. They can update the user input, orchestration plan, and the FM response. With access to the prompt template developers have better control over the Agent orchestration.

With fully managed agents, you don’t have to worry about provisioning or managing infrastructure and can take applications to production faster.

Any customer content processed by Amazon Bedrock is encrypted and stored at rest in the AWS Region where you are using Amazon Bedrock.

No. Users' inputs and model outputs are not shared with any model providers.

Amazon Bedrock offers several capabilities to support security and privacy requirements. Amazon Bedrock is in scope for common compliance standards such as Fedramp Moderate, Service and Organization Control (SOC), International Organization for Standardization (ISO), Health Insurance Portability and Accountability Act (HIPAA) eligibility, and customers can use Bedrock in compliance with the General Data Protection Regulation (GDPR). Amazon Bedrock is included in the scope of the SOC 1, 2, 3 reports, allowing customers to gain insights into our security controls. We demonstrate compliance through extensive third-party audits of our AWS controls. Amazon Bedrock is one of the AWS services under ISO Compliance for the ISO 9001, ISO 27001, ISO 27017, ISO 27018, ISO 27701, ISO 22301, and ISO 20000 standards. Amazon Bedrock is CSA Security Trust Assurance and Risk (STAR) Level 2 certified, which validates the use of best practices and the security posture of AWS cloud offerings. With Amazon Bedrock, your content is not used to improve the base models and is not shared with any model providers. You can use AWS PrivateLink to establish private connectivity from Amazon VPC to Amazon Bedrock, without having to expose your data to internet traffic.

No, AWS and the third-party model providers will not use any inputs to or outputs from Amazon Bedrock to train Amazon Titan or any third-party models.

Amazon Bedrock supports SDKs for runtime services. iOS and Android SDKs, as well as Java, JS, Python, CLI, .Net, Ruby, PHP, Go, and C++, support both text and speech input.

Streaming is supported on all the SDKs.

Please see the Amazon Bedrock pricing page for current pricing information.

Depending on your AWS Support contract, Amazon Bedrock is supported under Developer Support, Business Support and Enterprise Support plans.

You can use CloudWatch metrics to track the inputs and output token.

With Amazon Bedrock, you can privately customize FMs, retaining control over how your data is used and encrypted. Amazon Bedrock makes a separate copy of the base FM and trains this private copy of the model. Your data including prompts, information used to supplement a prompt, and FM responses. Customized FMs remain in the Region where the API call is processed.

When you’re fine-tuning a model, your data is never exposed to the public internet, never leaves the AWS network, is securely transferred through your VPC, and is encrypted in transit and at rest. Amazon Bedrock also enforces the same AWS access controls that you have with any of our other services.

We launched continued pretraining for Amazon Titan Text Express and Amazon Titan models on Amazon Bedrock. Continued pretraining allows you to continue the pretraining on an Amazon Titan base model using large amounts of unlabeled data. This type of training will adapt the model from a general domain corpus to a more specific domain corpus such as medical, law, finance, and so on, while still preserving most of the capabilities of the Amazon Titan base model. 

Enterprises may want to build models for tasks in a specific domain. The base models may not be trained on the technical jargon used in that specific domain. Thus, directly fine-tuning the base model requires large amounts of labeled training records and a long training duration to get accurate results. To ease this burden, the customer can instead provide large amounts of unlabeled data for a continued pretraining job. This job will adapt the Amazon Titan base model to the new domain. Then the customer may fine-tune the newly pretrained custom model to downstream tasks, using significantly fewer labeled training records and with a shorter training duration. 

Amazon Bedrock continued pretraining and fine-tuning have very similar requirements. For this reason, we are choosing to create unified APIs that support both continued pretraining and fine-tuning. Unification of the APIs reduces the learning curve and will help customers use standard features such as Amazon EventBridge to track long running jobs, Amazon S3 integration for fetching training data, resource tags, and model encryption. 

Continued pretraining helps you adapt the Amazon Titan models to your domain specific data while still preserving the base functionality of the Amazon Titan models. To create a continued pretraining job, navigate to the Amazon Bedrock console and click on "Custom Models." You will navigate to the custom model page that has two tabs: Models and Training jobs. Both tabs provide a “Customize Model” drop-down menu on the right. Select “Continued Pretraining” from the drop-down menu to navigate to “Create Continued Pretraining Job." You will provide the source model, name, model encryption, input data, hyper-parameters and output data. Additionally, you can provide tags, along with details about AWS Identity and Access Management (IAM) roles and resource policies for the job.

Exclusive to Amazon Bedrock, the Amazon Titan family of models incorporates 25 years of Amazon experience innovating with AI and machine learning across the business. Amazon Titan FMs provide customers with a breadth of high-performing image, multimodal, and text model choices through a fully managed API. Amazon Titan models are created by AWS and pretrained on large datasets, making them powerful, general-purpose models built to support a variety of use cases, while also supporting the responsible use of AI. Use them as is or privately customize them with your own data. Learn more about Amazon Titan.

To learn more about data processed to develop and train Amazon Titan FMs, visit Amazon Titan Model Training and Privacy page.

Supported data formats include .pdf, .txt, .md, .html, .doc and .docx, .csv, .xls, and .xlsx files. Files must be uploaded to Amazon S3. Point to the location of your data in Amazon S3, and Amazon Bedrock Knowledge Bases takes care of the entire ingestion workflow into your vector database.

Amazon Bedrock Knowledge Bases provides three options to chunk text before converting it to embeddings. 

1.  Default option: Amazon Bedrock Knowledge Bases automatically splits your document into chunks each containing 200 tokens, ensuring that a sentence is not broken in the middle. If a document contains less than 200 tokens, then it is not split any further. An overlap of 20% of tokens is maintained between two consecutive chunks.

2.  Fixed size chunking: In this option, you can specify the maximum number of tokens per chunk and the overlap percentage between chunks for Amazon Bedrock Knowledge Bases, so your document will be automatically split into chunks, ensuring that a sentence is not broken in the middle. 

3.  Create one embedding per document option: Amazon Bedrock creates one embedding per document. This option is suitable if you have preprocessed your documents by splitting them into separate files and do not want Amazon Bedrock to further chunk your documents.

At present, Amazon Bedrock Knowledge Bases uses the latest version of the Amazon Titan Text Embeddings model available in Amazon Bedrock. Titan Text Embeddings V2 model supports 8K tokens and 100+ languages and creates an embeddings of flexible 256, 512, and 1,024 dimension size. 

Amazon Bedrock Knowledge Bases takes care of the entire ingestion workflow of converting your documents into embeddings (vector) and storing the embeddings in a specialized vector database.  Amazon Bedrock Knowledge Bases supports popular databases for vector storage, including vector engine for Amazon OpenSearch Serverless, Pinecone, Redis Enterprise Cloud, Amazon Aurora (coming soon), and MongoDB (coming soon). If you do not have an existing vector database, Amazon Bedrock creates an OpenSearch Serverless vector store for you.

Depending on your use case, you can use Amazon EventBridge to create a periodic or event-driven sync between Amazon S3 and Amazon Bedrock Knowledge Bases.

Model Evaluation on Amazon Bedrock allows you to evaluate, compare, and select the best FM for your use case in just a few short steps. Amazon Bedrock offers a choice of automatic evaluation and human evaluation. You can use automatic evaluation with predefined metrics such as accuracy, robustness, and toxicity. You can use human evaluation workflows for subjective or custom metrics such as friendliness, style, and alignment to brand voice. For human evaluation, you can use your in-house employees or an AWS-managed team as reviewers. Model Evaluation on Amazon Bedrock provides built-in curated datasets or you can bring your own datasets.

You can evaluate variety of predefined metrics such as accuracy, robustness, and toxicity using automatic evaluations. You can also use human evaluation workflows for subjective or custom metrics, such as friendliness, relevance, style, and alignment to brand voice.

Automatic evaluations allow you to quickly narrow down the list of available FMs against standard criteria (such as accuracy, toxicity and robustness). Human-based evaluations are often used to evaluate more nuanced or subjective criteria that require human judgment and where automatic evaluations might not exist (such as brand voice, creative intent, friendliness).

You can quickly evaluate Amazon Bedrock models for metrics such as accuracy, robustness, and toxicity by using curated built-in data sets or by bringing your own prompt datasets. After your prompt datasets are sent to Amazon Bedrock models for inference, the model responses are scored with evaluation algorithms for each dimension. The backend engine aggregates individual prompt response scores into summary scores and presents them through easy-to-understand visual reports.

Amazon Bedrock allows you to set up human review workflows in a few short steps and bring your in-house employees, or use an expert team managed by AWS, to evaluate models. Through Amazon Bedrock’s intuitive interface, humans can review and give feedback on model responses by clicking thumbs up or down, rating on a scale of 1-5, choosing the best of multiple responses, or ranking prompts. For example, a team member can be shown how two models respond to the same prompt, and then be asked to select the model that shows more accurate, relevant, or stylistic outputs. You can specify the evaluation criteria that matter to you by customizing the instructions and buttons to appear on the evaluation UI for your team. You can also provide detailed instructions with examples and the overall goal of model evaluation, so users can align their work accordingly. This method is useful to evaluate subjective criteria that require human judgement or more nuanced subject matter expertise and that cannot be easily judged by automatic evaluations.

Amazon Bedrock Guardrails help you implement safeguards for your generative AI applications based on your use cases and responsible AI policies. Guardrails helps control the interaction between users and FMs by filtering undesirable and harmful content and will soon redact personally identifiable information (PII), enhancing content safety and privacy in generative AI applications. You can create multiple guardrails with different configurations tailored to specific use cases. Additionally, with the guardrails you can continually monitor and analyze user inputs and FM responses that might violate customer-defined policies.

Guardrails help you to define a set of policies to help safeguard your generative AI applications. You can configure the following policies in a guardrail.

• Denied topics: help you define a set of topics that are undesirable in the context of your application. For example, an online banking assistant can be designed to refrain from providing investment advice.
• Content filters: help you configure thresholds to filter harmful content across hate, insults, sexual, and violence categories.
• Word filters: help you define a set of words to block in user inputs and FM–generated responses.
• PII redaction: helps you select a set of PII that can be redacted in FM–generated responses. Based on the use case, Guardrails can also help you block a user input if it contains PII.
• Contextual grounding checks: help detect and filter hallucinations if the responses are not grounded (e.g., factually inaccurate or new information) in the source information and irrelevant to user’s query or instruction.

Guardrails can be used with all large language models (LLMs) available on Amazon Bedrock. It can also be used with fine-tuned FMs as well as Agents for Amazon Bedrock.

AWS offers an uncapped intellectual property (IP) indemnity for copyright claims arising from generative output of the following generally available Amazon generative AI services: Amazon Titan models, Amazon CodeWhisperer Professional, and other services listed in Section 50.10 of the Service Terms (the “Indemnified Generative AI Services”). This means that customers are protected from third-party claims alleging copyright infringement by the output generated by the Indemnified Generative AI Services in response to inputs or other data provided by the customer. Customers must also use the services responsibly, such as not inputting infringing data or disabling a service’s filtering features.

In addition, our standard IP indemnity for use of the services protects customers from third-party claims alleging IP infringement (including copyright claims) by the services and the data used to train them.

There are five guardrail policies each with different off-the-shelf protections

• Content filters – This has 6 off the shelf categories (hate, insults, sexual, violence, misconduct (incl. criminal activity) and prompt attack (jailbreak and prompt injection. Each category can have further customized thresholds in terms of aggressiveness of filtering - low/medium/high.
• Denied topic – These are customized topics that customers can define using simple natural language description
• Sensitive information filter – These come with 30+ off the shelf PIIs. It can be further customized by adding customer’s proprietary information that are sensitive.
• Word filters – It comes with off the shelf profanity filtering and can be further customized with custom words.
• Contextual grounding checks – It can help detect hallucinations for RAG, summarization, and conversational applications, where source information can be used as reference to validate the model response.

Foundations model have native safeguards and they are the default protections associated with each model. These native safeguards are NOT part of Amazon Bedrock Guardrails. Amazon Bedrock Guardrails is an added layer of customized safeguards that can be optionally applied by the customer based on their application requirements and responsible AI policies.

As part of Amazon Bedrock Guardrails, SSN and phone number detection are part of the 30+ off the shelf PIIs. Full list here.

There is a separate cost for using Amazon Bedrock Guardrails. It can be applied for both input and output. Pricing at the bottom of the page here.

Yes, Amazon Bedrock Guardrail APIs help customers run automated tests. “Test case builder” maybe something you want to use prior to deploying guardrails in production. There is no native test case builder yet. For ongoing monitoring of production traffic, guardrails help provide detailed logs of all violation for each input and output, so that customers can granularly monitor each and every input coming and going out of their gen AI application. These logs can be stored in CloudWatch or S3 and can be used to create custom dashboards based on customers’ requirements.