[SEO Subhead]
This Guidance helps you facilitate the migration of your digital assets to the cloud, allowing you to take full advantage of the latest advancements in asset management and prepare content for distribution to media consumers. It sets up serverless ingestion and analysis workflows to move your video assets and associated metadata to the AWS Cloud. During the migration, this Guidance analyzes and extracts machine learning metadata from your video. It also comes with a web interface so you can immediately start ingesting and analyzing your content.
Please note: [Disclaimer]
Architecture Diagram
[Architecture diagram description]
Step 1
An Amazon Cognito user pool to provide a user directory.
Step 2
An Amazon API Gateway RESTful API endpoint, which is configured to use AWS Identity and Access Management (IAM) authentication.
Step 3
An Amazon CloudFront distribution that hosts the web application artifacts, such as minimized JavaScript files and graphics stored in the web bucket.
Step 4
An AWS Step Functions main state machine which serves as the entry point to the backend ingestion and analysis workflows.
Step 5
A Step Functions ingestion sub-state machine that orchestrates the ingestion process by media file type and generates proxies for ingested media. It uses AWS Elemental MediaConvert for video and audio files and open-source tools for image files and documents.
Step 6
A Step Functions analysis sub-state machine that is responsible for the analysis process. It consists of Step Functions that run analysis jobs with Amazon Rekognition, Amazon Transcribe, Amazon Comprehend, and Amazon Textract.
Step 7
Amazon DynamoDB tables to store artifacts generated during the ingestion and analysis processes, such as overall status, pointers to where intermediate files are stored, and state machine run tokens.
Step 8
An Amazon OpenSearch Service cluster, which stores ingestion attributes and machine learning metadata, and facilitates your search and discovery needs.
Step 9
Four Amazon Simple Storage Service (Amazon S3) buckets store: uploaded content, file proxies that the Guidance generates during ingestion, static web application artifacts, and access logs for services used.
Step 10
Amazon CloudWatch event rules that are logged when specific tasks undergo state changes.
Step 11
Amazon EventBridge used by an internal queue management system where the backlog system notifies workflows (state machines) when a queued artificial intelligence and machine learning (AI/ML) request has been processed.
Step 12
An AWS IoT Core topic that allows the ingestion and analysis workflows to communicate with the front-end web application asynchronously through publish or subscribe MQTT messaging.
Step 13
Amazon Simple Notification Service (Amazon SNS) topics to allow Amazon Rekognition to publish job status in the video analysis workflow, and to support custom integration with your system.
Step 1
An Amazon Cognito user pool to provide a user directory.
Step 2
An Amazon API Gateway RESTful API endpoint, which is configured to use AWS Identity and Access Management (IAM) authentication.
Step 3
An Amazon CloudFront distribution that hosts the web application artifacts, such as minimized JavaScript files and graphics stored in the web bucket.
Step 4
An AWS Step Functions main state machine which serves as the entry point to the backend ingestion and analysis workflows.
Step 5
A Step Functions ingestion sub-state machine that orchestrates the ingestion process by media file type and generates proxies for ingested media. It uses AWS Elemental MediaConvert for video and audio files and open-source tools for image files and documents.
Step 6
A Step Functions analysis sub-state machine that is responsible for the analysis process. It consists of Step Functions that run analysis jobs with Amazon Rekognition, Amazon Transcribe, Amazon Comprehend, and Amazon Textract.
Step 7
Amazon DynamoDB tables to store artifacts generated during the ingestion and analysis processes, such as overall status, pointers to where intermediate files are stored, and state machine run tokens.
Step 8
An Amazon OpenSearch Service cluster, which stores ingestion attributes and machine learning metadata, and facilitates your search and discovery needs.
Step 9
Four Amazon Simple Storage Service (Amazon S3) buckets store: uploaded content, file proxies that the Guidance generates during ingestion, static web application artifacts, and access logs for services used.
Step 10
Amazon CloudWatch event rules that are logged when specific tasks undergo state changes.
Step 11
Amazon EventBridge used by an internal queue management system where the backlog system notifies workflows (state machines) when a queued artificial intelligence and machine learning (AI/ML) request has been processed.
Step 12
An AWS IoT Core topic that allows the ingestion and analysis workflows to communicate with the front-end web application asynchronously through publish or subscribe MQTT messaging.
Step 13
Amazon Simple Notification Service (Amazon SNS) topics to allow Amazon Rekognition to publish job status in the video analysis workflow, and to support custom integration with your system.
Well-Architected Pillars
The AWS Well-Architected Framework helps you understand the pros and cons of the decisions you make when building systems in the cloud. The six pillars of the Framework allow you to learn architectural best practices for designing and operating reliable, secure, efficient, cost-effective, and sustainable systems. Using the AWS Well-Architected Tool, available at no charge in the AWS Management Console, you can review your workloads against these best practices by answering a set of questions for each pillar.
The architecture diagram above is an example of a Solution created with Well-Architected best practices in mind. To be fully Well-Architected, you should follow as many Well-Architected best practices as possible.
-
Operational ExcellenceRead the Operational Excellence whitepaper
This Guidance pushes metrics to CloudWatch at various stages to provide observability into the infrastructure, such as AWS Lambda functions, AI services, and Amazon S3 buckets.
-
SecurityRead the Security whitepaper
We highly recommend that you encrypt sensitive data in transit and at rest. This Guidance automatically encrypts media files and metadata at rest with Amazon S3 server-side encryption (SSE-S3). The Guidance's Amazon SNS topics and DynamoDB tables are also encrypted at rest using SSE-S3. Furthermore, documents indexed to the OpenSearch Service cluster are encrypted at rest, and node-to-node communication within the cluster is also encrypted.
This Guidance deploys a static website hosted in an Amazon S3 bucket. To help reduce latency and improve security, this Guidance includes a CloudFront distribution with an origin access identity, which is a special CloudFront user that helps restrict access to website bucket contents. For more information, refer to Restricting access to an Amazon S3 origin.
-
ReliabilityRead the Reliability whitepaper
Guidance for Media2Cloud on AWS uses AWS serverless services wherever possible (for example, Lambda, API Gateway, Amazon S3, and DynamoDB) to ensure high availability and quick recovery from service failure.
-
Performance EfficiencyRead the Performance Efficiency whitepaper
You can create workflows with the pre-built AWS CloudFormation template. Because this Guidance is an open source project published on GitHub, you can customize this and build from the source code. The README provides a step-by-step guide to build and customize this Guidance, along with a mini-tutorial, and an in-depth explanation of each component and the overall design.
-
Cost OptimizationRead the Cost Optimization whitepaper
Because this Guidance uses a serverless and event-driven architecture, you are only charged for what you use. Its design allows you to configure and tailor your own media workflows, and use only the AWS services that you need. Furthermore, it allows your assets to be stored cost effectively using Amazon S3 with Lifecycle policies to store and archive ingested contents, proxies, and metadata.
-
SustainabilityRead the Sustainability whitepaper
A critical component for sustainability is maximizing the usage of the AWS AI services so you minimize the environmental impact of the backend services. And, the serverless design of this Guidance is designed to reduce your carbon footprint compared to the footprint of continually operating on-premises servers.
Implementation Resources
A detailed guide is provided to experiment and use within your AWS account. Each stage of building the Guidance, including deployment, usage, and cleanup, is examined to prepare it for deployment.
The sample code is a starting point. It is industry validated, prescriptive but not definitive, and a peek under the hood to help you begin.
Related Content
Announcing AWS Media Intelligence Solutions
Successful innovation for ESSENCE as Starchive and AWS help deliver the 2020 ESSENCE Festival of Culture
Related Content
[Title]
Disclaimer
The sample code; software libraries; command line tools; proofs of concept; templates; or other related technology (including any of the foregoing that are provided by our personnel) is provided to you as AWS Content under the AWS Customer Agreement, or the relevant written agreement between you and AWS (whichever applies). You should not use this AWS Content in your production accounts, or on production or other critical data. You are responsible for testing, securing, and optimizing the AWS Content, such as sample code, as appropriate for production grade use based on your specific quality control practices and standards. Deploying AWS Content may incur AWS charges for creating or using AWS chargeable resources, such as running Amazon EC2 instances or using Amazon S3 storage.
References to third-party services or organizations in this Guidance do not imply an endorsement, sponsorship, or affiliation between Amazon or AWS and the third party. Guidance from AWS is a technical starting point, and you can customize your integration with third-party services when you deploy the architecture.