Amazon EC2 DL2q Instances

Cost-efficient deep learning inference using Qualcomm’s AI technology stack

Get started with Amazon EC2 DL2q instances

Amazon EC2 DL2q instances, powered by Qualcomm AI 100 Standard accelerators, can be used to cost-efficiently deploy deep learning (DL) workloads in the cloud. They can also be used to develop and validate performance and accuracy of DL workloads that will be deployed on Qualcomm devices. DL2q instances are the first instances to bring Qualcomm’s AI technology to the cloud.

With 8 Qualcomm AI 100 Standard accelerators and 128 GiB of total accelerator memory, customers can also use DL2q instances to run popular generative AI applications, such as content generation, text summarization, and virtual assistants, as well as classic AI applications for natural language processing and computer vision. Additionally, Qualcomm AI 100 features the same AI technology used across smartphones, autonomous driving, personal compute, and extended reality headsets, so DL2q instances can be used to develop and validate these AI workloads before deployment.

Customers can use Qualcomm’s AI stack, which includes two software development kits (SDK) known as the Apps and Platform SDK to compile and deploy their DL workloads on the DL2q instances. To get started quickly, customers can use the AWS Deep Learning AMI (DLAMI), which comes prepackaged with the Qualcomm Apps and Platform SDK, and popular machine learning frameworks, such as PyTorch and TensorFlow.

Benefits

Cost-efficiency for deep learning inference

By using purpose-built acceleration, the DL2q instances deliver high throughput at a low cost. For example, DL2q instances deliver 100k ResNet50 images for less than $0.0017 or 100k BERT sequences for less than $0.0076.

Support for leading ML frameworks and models

DL2q instances and Qualcomm Apps and Platform SDK support leading ML frameworks such as TensorFlow and PyTorch, enabling you to continue using your preferred ML workflows. The Qualcomm Apps SDK also includes tools to enable easy porting and deployment of pre-trained models to DL2q instances while achieving the performance benefits of a purpose-built AI inference accelerator.

Consistent experience across edge and cloud

The same Qualcomm AI stack runs on Qualcomm edge devices and DL2q instances. This provides customers with a consistent developer experience, with a unified API across their cloud and edge development environments. So you can use DL2q instances to develop and validate AI solutions for smartphones, autonomous driving, personal compute, and extended reality headsets.

Features

Powered by Qualcomm AI 100 accelerators

DL2q instances feature 8 Qualcomm AI 100 accelerators with 16 GiB of memory per accelerator, 768 GiB of system memory, 2nd Generation Intel Xeon Scalable Processors, and 100 Gbps of network bandwidth.

Qualcomm Cloud AI Platform and Apps SDK

You can get started with DL2q instances using the Qualcomm Cloud AI Platform SDK. The SDK is comprised of kernel drivers for different OSs, user-space utilities, firmware, libraries, and validation tools. The Cloud AI Apps SDK contains the libraries and tools to help application developers and framework developers to interface with Qualcomm AI 100 accelerators. These SDKs are integrated with leading frameworks such as TensorFlow and PyTorch.

Built on AWS Nitro System

DL2q instances are built on the AWS Nitro System, which is a rich collection of building blocks that offloads many of the traditional virtualization functions to dedicated hardware and software to deliver high performance, high availability, and high security while also reducing virtualization overhead.

Product details

Instance Size	Qualcomm AI 100 Accelerators	Accelerator Memory (GB)	vCPU	Memory (GiB)	Local Storage	Inter-Accelerator Interconnect	Network Bandwidth (Gbps)	EBS Bandwidth (Gbps)	On-Demand Price	1-Year Reserved Instance	3-Year Reserved Instance
dl2q.24xlarge	8	128	96	768	EBS Only	No	100	19	$8.919	$5.352	$3.568

Getting started

The AWS Deep Learning AMIs (DLAMI) and AWS Deep Learning Containers (DLC)

AWS Deep Learning AMIs (DLAMI) and AWS Deep Learning Containers (DLC) provide data scientists, ML practitioners, and researchers with machine and container images that are pre-installed with deep learning frameworks. They make it easy to get started by letting you skip the complicated process of building and optimizing your software environments from scratch. The Qualcomm Apps and Platform SDK for Qualcomm AI 100 will be integrated into the AWS DL AMIs and DLCs enabling you to quickly get started with DL2q instances.