P5 instances provide up to 8 NVIDIA H100 GPUs with a total of up to 640 GB HBM3 GPU memory per instance. P5 instances support up to 900 GB/s of NVSwitch GPU interconnect (total of 3.6 TB/s bisectional bandwidth in each instance), so each GPU can communicate with every other GPU in the same instance with single-hop latency.

NVIDIA H100 GPUs have a new transformer engine that intelligently manages and dynamically chooses between FP8 and 16-bit calculations. This feature helps deliver faster DL training speedups on LLMs compared to previous-generation A100 GPUs. For HPC workloads, NVIDIA H100 GPUs have new DPX instructions that further accelerate dynamic programming algorithms as compared to A100 GPUs.

P5 instances deliver up to 3,200 Gbps of EFAv2 networking. EFAv2 delivers up to 50% improvement in collective communications performance for distributed training workloads. EFAv2 is also coupled with NVIDIA GPUDirect RDMA to enable low-latency GPU-to-GPU communication between servers with operating system bypass.

P5 instances support Amazon FSx for Lustre file systems so you can access data at the hundreds of GB/s of throughput and millions of IOPS required for large-scale DL and HPC workloads. Each P5 instance also supports up to 30 TB of local NVMe SSD storage for fast access to large datasets. You can also use virtually unlimited cost-effective storage with Amazon Simple Storage Service (Amazon S3).

P5 instances are deployed in second-generation EC2 UltraClusters, which provide a network fabric that enables greater scale, fewer network hops across the cluster, and lower latency than previous-generation UltraClusters. P5 instances in UltraClusters can scale up to 20,000 H100 GPUs interconnected with petabit-scale network and deliver 20 exaflops of aggregate compute capability.

P5 instances can be deployed using AWS Deep Learning AMIs (DLAMI) and AWS Deep Learning Containers. They are available through managed services such as Amazon SageMaker, Amazon Elastic Kubernetes Service (Amazon EKS), Amazon Elastic Container Service (Amazon ECS), AWS Batch, and more.

SageMaker is a fully managed service for building, training, and deploying ML models. When used together with P5 instances, you can more easily scale to tens, hundreds, or thousands of GPUs to train a model quickly at any scale without worrying about setting up clusters and data pipelines.

DLAMI provides ML practitioners and researchers with the infrastructure and tools to accelerate DL in the cloud, at any scale. Deep Learning Containers are Docker images preinstalled with DL frameworks to streamline the deployment of custom ML environments by letting you skip the complicated process of building and optimizing your environments from scratch.

If you prefer to manage your own containerized workloads through container orchestration services, you can deploy P5 instances with Amazon EKS or Amazon ECS.