How do I troubleshoot issues when setting up a new capacity provider or update an existing capacity provider for my Amazon ECS cluster?

5 minute read
1

I'm unable to set up a new capacity provider or update an existing capacity provider for my Amazon Elastic Container Service (Amazon ECS) cluster.

Short description

The following are common issues when you set up a new capacity provider or update an existing capacity provider for your Amazon ECS cluster:

  • When you use the AWS Command Line Interface (AWS CLI) to create an Auto Scaling group capacity provider, you receive errors.
  • You receive the error: "The specified capacity provider already exists. To change the configuration of an existing capacity provider, update the capacity provider."
  • You receive the error: "The specified Auto Scaling group ARN is already being used by another capacity provider. Specify a unique Auto Scaling group ARN and try again."
  • You receive the error when you create the capacity provider strategy: "The specified cluster is in a busy state. Cluster attachments must be in UPDATE_COMPLETE or UPDATE_FAILED state before they can be updated. Wait and try again."
  • You receive errors because of limitations with the capacity provider.
  • The capacity provider is created successfully with AWS CloudFormation, but it's not shown in the console.

Resolution

When you use the AWS CLI to create an Auto Scaling group capacity provider, you receive errors

When you use the AWS CLI to create a capacity provider, review your AWS CloudTrail events to check for the CreateCapacityProvider API. These errors are common CapacityProvider API errors:

  • ClientException: These errors are usually caused by a client action. The client action might use an action or resource on behalf of an AWS Identity and Access Management (IAM) user that doesn't have the required permissions to use the action or resource. This error can also specify that an identifier isn't valid.
  • UpdateInProgressException: You receive this error because there is already a current Amazon ECS container agent update in progress on the container instance that you specified. If the container agent disconnects while in a transitional stage, such as PENDING or STAGING, then the update can get stuck in that state. However, when the agent reconnects, it resumes where it previously stopped.

For more information, see Errors.

Note: If you receive errors when running AWS CLI commands, make sure that you're using the most recent version of the AWS CLI.

You receive the error: "The specified capacity provider already exists. To change the configuration of an existing capacity provider, update the capacity provider"

When you create the capacity provider, use a name that's different from any of the capacity provider names used previously. Or, you can delete the existing capacity provider with that name and then create a new capacity provider.

To view a list of all capacity providers created, run the following command:

aws ecs describe-capacity-providers --query capacityProviders\[\].name --region example-region

The output looks similar to the following:

[
    "FARGATE",
    "FARGATE_SPOT",
    "ecs-capacityprovider-1"
]

To delete a capacity provider, run the following command:

aws ecs delete-capacity-provider --capacity-provider ecs-capacityprovider-1 --region example-region

For sample output and more options, see delete-capacity-provider.

To create a new capacity provider, run this command:

aws ecs create-capacity-provider --name "example-capacity=provider" --auto-scaling-group-provider "autoScalingGroupArn=arn:aws:autoscaling:us-east-1:123456789012:autoScalingGroup:a1b2c3d4-5678-90ab-cdef-EXAMPLE11111:autoScalingGroupName/MyAutoScalingGroup,managedScaling={status=ENABLED,targetCapacity=100,minimumScalingStepSize=1,maximumScalingStepSize=100},managedTerminationProtection=ENABLED"

For sample output and more options, see create-capacity-provider.

You receive the error: "The specified Auto Scaling group ARN is already being used by another capacity provider. Specify a unique Auto Scaling group ARN and try again"

The Auto Scaling group and capacity provider must have a one-on-one relationship. You can attach an Auto Scaling group to exactly one capacity provider, and the other way around. Be sure that your capacity provider is associated with only one Auto Scaling group.

You receive the error: "The specified cluster is in a busy state. Cluster attachments must be in UPDATE_COMPLETE or UPDATE_FAILED state before they can be updated. Wait and try again"

This error auto resolves after several minutes. After waiting, retry updating the capacity provider. Or, run this CLI command to view the current status of the attachments. Then, wait until the attachment is in the UPDATE_COMPLETE status.

aws ecs describe-clusters --clusters example-ARN --include ATTACHMENTS --query clusters\[\].attachmentsStatus --region example-region

The output must look similar to the following:

[
    "UPDATE_COMPLETE"
]

You receive errors because of limitations with the capacity provider

Review these points before you create or update a capacity provider:

  • You can't specify more than six capacity providers for a capacity strategy.
  • You can use either Auto Scaling group capacity providers or AWS Fargate capacity providers in your cluster strategy. You can't use a combination of both types.
  • The base value for the capacity provider strategy is supported only when you run the tasks.

The capacity provider is created successfully with CloudFormation, but it isn't shown in the console

This can happen when you don't associate the capacity provider with the cluster in your CloudFormation template. To avoid this issue, use this snippet in your CloudFormation template to associate the capacity provider with the cluster:

...
ECSCluster:
  Type: 'AWS::ECS::Cluster'
  Properties:
    ClusterName: example-cluster
    CapacityProviders:
      - !Ref ECSCapacityProvider
...

Related information

Amazon ECS capacity providers

Auto Scaling group capacity providers

How do I resolve the DELETE_FAILED error when deleting the capacity provider in Amazon ECS?

AWS OFFICIAL
AWS OFFICIALUpdated a year ago
5 Comments

Hello! What might cause a capacity provider to remain in the "Update in progress" status for multiple days (>1 week)? We are unable to delete or update the capacity provider while it remains stuck in this state.

BMSP
replied 2 months ago

Thank you for your comment. We'll review and update the Knowledge Center article as needed.

profile pictureAWS
MODERATOR
replied 2 months ago

I am having the same issue as BMSP, as I've detailed in a different question. What could cause the capacity provider or ECS cluster to get stuck in “Update in progress” for multiple days?

profile picture
Miguel
replied a month ago

Thank you for your comment. We'll review and update the Knowledge Center article as needed.

profile pictureAWS
MODERATOR
replied a month ago

I am having the same issue with my capacity providers, I have 2 capacity provider which is stuck in "Update in progress" and "Delete failed" I cannot delete the both capacity provider Note : Created using terraform so cannot delete in cloud formation stacks

replied a month ago