How do I fix the "Unable to validate the following destination configurations" error in CloudFormation?

10 minute read

I subscribed to an AWS service and received the "Unable to validate the following destination configurations" error in AWS CloudFormation.

Resolution

To resolve the error message, follow these troubleshooting steps for your configuration.

Note: If you receive errors when you run the AWS Command Line Interface (AWS CLI) commands, then see Troubleshoot AWS CLI errors. Also, make sure that you're using the most recent AWS CLI version.

Issues with Lambda notification configuration

Error: "notification lambda function not exist"

This error occurs when you create or update an Amazon Simple Storage Service (Amazon S3) bucket that uses the AWS Lambda property LambdaConfigurations. If the Lambda function's ARN that's configured in the CloudFormation template doesn't exist or isn't valid, then you might receive this error message.

To check if the LambdaConfigurations property exists, run the following AWS CLI command:

aws lambda get-function --function-name YOUR-FUNCTION-ARN-VALUE

Note: Replace YOUR-FUNCTION-ARN-VALUE with the Lambda function's ARN.

If you receive an error in the command output, then the Lambda function's ARN isn't valid or doesn't exist. Update the template to include the correct ARN. Then, create a new stack or update the existing stack.

Lambda function doesn't have permission to invoke Amazon S3

Error: "The resource you requested does not exist"

If you receive this error, then a permission isn't attached to the Lambda function.

To resolve this issue, complete the following steps:

Run the following command to check the Lambda function permissions:

aws lambda get-policy --function-name YOUR-FUNCTION-ARN-VALUE --region YOUR-REGION

Note: Replace YOUR-FUNCTION-ARN-VALUE with the Lambda function's ARN and YOUR-REGION with your AWS Region.

Attach the following permission in the CloudFormation template to allow the Lambda function to invoke Amazon S3:

S3Permission:
    Type: AWS::Lambda::Permission
    Properties:
     FunctionName: YOUR-FUNCTION-ARN-VALUE
     Action: lambda:InvokeFunction
     Principal: s3.amazonaws.com
     SourceAccount: !Ref 'AWS::AccountId'

Note: Replace YOUR-FUNCTION-ARN-VALUE with the Lamba function's ARN and AccountID with your AWS account ID .

To make sure that the S3 bucket is created only after permission is granted to the Lambda function, add the DependsOn attribute:

S3Bucket:
    Type: AWS::S3::Bucket
    DependsOn:
         - "S3Permission"
    Properties:
     NotificationConfiguration:
       LambdaConfigurations:
        - Function: <<FUNCTION-ARN-VALUE>>
          Event: "s3:ObjectCreated:Put"

Issues with Amazon SNS notification configuration

Error: "SNS Topic does not exist or invalid"

This error occurs when an S3 bucket uses the TopicConfigurations property and the Amazon Simple Notification Service (Amazon SNS) topic doesn't exist or isn't valid. The ARN format and value must match the Amazon SNS topic's ARN.

Run the list-topics AWS CLI command to check that the SNS topic's ARN exists in your account:

aws sns list-topics \
--region YOUR-REGION \
--query "Topics[?TopicArn=='YOUR-TOPIC-ARN-VALUE']"

Note: Replace YOUR-REGION with your Region and YOUR-TOPIC-ARN-VALUE with the SNS topic's ARN.

If you didn't receive any records in the command output, then the SNS topic either doesn't exist or isn't valid. Create the SNS topic, and make sure that the topic is valid.

Issue with the SNS topic access policy

If the topics that are configured in the TopicConfigurations property are valid, then check the access policy that's attached to the SNS topic.

Complete the following steps:

Run the get-topic-attributes AWS CLI command to check the access policy that's attached to the SNS topic:

aws sns get-topic-attributes \
--topic-arn YOUR-TOPIC-ARN-VALUE \
--region YOUR-REGION \
--query 'Attributes.Policy'

Note: Replace YOUR-TOPIC-ARN-VALUE with your ARN and YOUR-REGION with your Region.

If the policy doesn't have access to the SNS topic, then replace the policy with the following one:

{
      "Sid": "S3AccessForNotification",
      "Effect": "Allow",
      "Principal": {
        "Service": "s3.amazonaws.com"
      },
      "Action": "SNS:Publish",
      "Resource": "YOUR-TOPIC-ARN-VALUE"
 }

Note: Replace YOUR-TOPIC-ARN-VALUE with your ARN and YOUR-REGION with your Region.

Create a new stack, or update the existing stack.

Issue with the AWS KMS key policy that's associated with the SNS topic

To resolve this issue, complete the following steps:

Run the get-topic-attributes AWS CLI command to check the AWS Key Management Service (AWS KMS) policy that's associated with the NotificationConfiguration property:

aws sns get-topic-attributes \
--topic-arn YOUR-TOPIC-ARN-VALUE \
--region YOUR-REGION \
--query "Attributes.KmsMasterKeyId"

Note: Replace YOUR-TOPIC-ARN-VALUE with your ARN and YOUR-REGION with your Region.

To check the policy permissions, run the get-key-policy AWS CLI command:

aws kms get-key-policy \
--key-id YOUR-KMS-KEY-ARN \
--policy-name default \
--region YOUR-REGION

Note: Replace YOUR-KMS-KEY-ARN with your ARN and YOUR-REGION with your Region.

{
    "Version": "2012-10-17",
    "Statement": [{
        "Effect": "Allow",
        "Principal": {"Service": "s3.amazonaws.com"},
        "Action": ["kms:GenerateDataKey*", "kms:Decrypt"],
        "Resource": "<<YOUR-KMS-KEY-ARN>>"
    }]
}

Issues with Amazon SQS notification configuration

Error: "SQS queue does not exist"

This error occurs when an S3 bucket uses the QueueConfigurations property and the Amazon SNS ARN doesn't exist or isn't valid.

To resolve this issue, complete the following steps:

Run the list-queues AWS CLI command to check that the Amazon SQS queue exists in the account:

aws sqs list-queues --queue-name-prefix YOUR-SQS-QUEUE-NAME --region YOUR-REGION

Note: Replace YOUR-SQS-QUEUE-NAME with your Amazon SQS queue name and YOUR-REGION with your AWS Region.

If the Amazon SQS queue doesn't exist, then either create a new queue or update the template with an existing queue.

Amazon SQS access policy doesn't have permission to S3 service

If the Amazon SQS queues that are configured in the QueueConfigurations property are valid, then check the access policy permissions.

Complete the following steps:

Run the list-queues AWS CLI command to check the Amazon SQS queue policy:

aws sqs list-queues --queue-name-prefix YOUR-SQS-QUEUE-NAME --region YOUR-REGION

aws sqs get-queue-attributes \
--queue-url YOUR-QUEUE-URL \
--region YOUR-REGION \
--attribute-names Policy

Note: Replace YOUR-SQS-QUEUE-NAME, YOUR-REGION, and YOUR-QUEUE-URL with your Amazon SQS values.

If the policy doesn't have access to Amazon S3, then replace the policy with the following one:

{
      "Sid": "S3AccessForNotification",
      "Effect": "Allow",
      "Principal": {
        "Service": "s3.amazonaws.com"
      },
      "Action": "SQS:SendMessage",
      "Resource": "YOUR-SQS-QUEUE-ARN-VALUE"
 }

Note: Replace YOUR-SQS-QUEUE-ARN-VALUE with your ARN.

Create or update the stack.

Issue with the AWS KMS key policy associated with the SQS queue

Run the list-queues AWS CLI command to get the AWS KMS key that's configured in the NotificationConfiguration property:

aws sqs list-queues --queue-name-prefix YOUR-SQS-QUEUE-NAME --region YOUR-REGION

aws sqs get-queue-attributes \
--queue-url YOUR-QUEUE-URL \
--region YOUR-REGION \
--attribute-names KmsMasterKeyId

Note: Replace YOUR-SQS-QUEUE-NAME, YOUR-REGION, and YOUR-QUEUE-URL with your Amazon SQS values.

Run the get-key-policy AWS CLI command to check that the Amazon S3 service has access to the key:

aws kms get-key-policy \
--key-id YOUR-KMS-KEY-ARN \
--policy-name default \
--region YOUR-REGION

Note: Replace YOUR-KMS-KEY-ARN with your ARN and YOUR-REGION with your Region.

In the output, make sure that the AWS KMS key policy has permission to allow Amazon S3 to use the key:

{
    "Version": "2012-10-17",
    "Statement": [{
        "Effect": "Allow",
        "Principal": {"Service": "s3.amazonaws.com"},
        "Action": ["kms:GenerateDataKey*", "kms:Decrypt"],
        "Resource": "YOUR-KMS-KEY-ARN"
    }]
}

Note: Replace YOUR-KMS-KEY-ARN with the AWS KMS key's ARN.

Circular dependency between resources

Because of the way that CloudFormation manages dependency ordering, Amazon S3 event notifications are defined as an attribute of the S3 bucket. These notifications are established when you create the S3 bucket resource.

To avoid an error, you must create resources in the following order:

Create the SNS topic because the S3 bucket references the SNS topic.
Create the S3 bucket because the SNS topic policy references both the S3 bucket and the SNS topic.

Before you subscribe an SNS topic to S3 event notifications, you must specify AWS::SNS::TopicPolicy with the correct permissions. The topic policy must exist before you create the subscription.

To avoid the "Unable to validate the following destination configurations" error, use on of the following methods:

Specify a value for BucketName in your CloudFormation template.
Create a stack, and then perform a stack update.

Specify a value for BucketName in your CloudFormation template

Specify a value for the BucketName property in the S3Bucket resource of your CloudFormation template to use a static name for your S3 bucket. You no longer need to include {"Ref": "paramBucketName"} in the SNS topic policy. The static name for the S3 bucket name value removes the intrinsic dependency between the SNS topic policy and Amazon S3.

The following example CloudFormation template specifies a hardcoded value (-Bucket-Name-) for the BucketName property:

{
  "Resources": {
    "SNSTopic": {
      "Type": "AWS::SNS::Topic"
    },
    "SNSTopicPolicy": {
      "Type": "AWS::SNS::TopicPolicy",
      "Properties": {
        "PolicyDocument": {
          "Id": "MyTopicPolicy",
          "Version": "2012-10-17",
          "Statement": [
            {
              "Sid": "Statement-id",
              "Effect": "Allow",
              "Principal": {
                "Service": "s3.amazonaws.com"
              },
              "Action": "sns:Publish",
              "Resource": {
                "Ref": "SNSTopic"
              },
              "Condition": {
                "ArnLike": {
                  "aws:SourceArn": {
                    "Fn::Join": [
                      "",
                      [
                        "arn:aws:s3:::",
                        "-Bucket-Name-"
                      ]
                    ]
                  }
                }
              }
            }
          ]
        },
        "Topics": [
          {
            "Ref": "SNSTopic"
          }
        ]
      }
    },
    "S3Bucket": {
      "Type": "AWS::S3::Bucket",
      "DependsOn": [
        "SNSTopicPolicy"
      ],
      "Properties": {
        "AccessControl": "BucketOwnerFullControl",
        "BucketName": "-Bucket-Name-",
        "NotificationConfiguration": {
          "TopicConfigurations": [
            {
              "Topic": {
                "Ref": "SNSTopic"
              },
              "Event": "s3:ObjectCreated:Put"
            }
          ]
        }
      }
    }
  }
}

Note: Replace -Bucket-Name- with your bucket's name. The S3Bucket resource has an explicit DependsOn attribute that's set to SNSTopicPolicy. The attribute specifies that the template creates the SNSTopicPolicy resource before the S3Bucket resource.

To use the same CloudFormation template for S3 buckets with different names, define a parameter for the bucket name. When you pass the bucket name as the parameter during stack creation, you can use the same template for different bucket names.

To use the following example template, you must pass the bucket name as the paramBucketName parameter during stack creation:

{
  "Parameters": {
    "paramBucketName": {
      "Type": "String",
      "Description": "Bucket Name"
    }
  },
  "Resources": {
    "SNSTopic": {
      "Type": "AWS::SNS::Topic"
    },
    "SNSTopicPolicy": {
      "Type": "AWS::SNS::TopicPolicy",
      "Properties": {
        "PolicyDocument": {
          "Id": "MyTopicPolicy",
          "Version": "2012-10-17",
          "Statement": [
            {
              "Sid": "Statement-id",
              "Effect": "Allow",
              "Principal": {
                "Service": "s3.amazonaws.com"
              },
              "Action": "sns:Publish",
              "Resource": {
                "Ref": "SNSTopic"
              },
              "Condition": {
                "ArnLike": {
                  "aws:SourceArn": {
                    "Fn::Join": [
                      "",
                      [
                        "arn:aws:s3:::",
                        {
                          "Ref": "paramBucketName"
                        }
                      ]
                    ]
                  }
                }
              }
            }
          ]
        },
        "Topics": [
          {
            "Ref": "SNSTopic"
          }
        ]
      }
    },
    "S3Bucket": {
      "Type": "AWS::S3::Bucket",
      "DependsOn": [
        "SNSTopicPolicy"
      ],
      "Properties": {
        "AccessControl": "BucketOwnerFullControl",
        "BucketName": {
          "Ref": "paramBucketName"
        },
        "NotificationConfiguration": {
          "TopicConfigurations": [
            {
              "Topic": {
                "Ref": "SNSTopic"
              },
              "Event": "s3:ObjectCreated:Put"
            }
          ]
        }
      }
    }
  }
}

Create a stack, and then update the stack

First, create the stack, but don't specify the NotificationConfiguration property in the S3Bucket resource. Then, update the stack to add the S3 event notification.

Create all the resources, including the SNS topic policy:

{
  "Resources": {
    "SNSTopic": {
      "Type": "AWS::SNS::Topic"
    },
    "SNSTopicPolicy": {
      "Type": "AWS::SNS::TopicPolicy",
      "Properties": {
        "PolicyDocument": {
          "Id": "MyTopicPolicy",
          "Version": "2012-10-17",
          "Statement": [
            {
              "Sid": "Statement-id",
              "Effect": "Allow",
              "Principal": {
                "Service": "s3.amazonaws.com"
              },
              "Action": "sns:Publish",
              "Resource": {
                "Ref": "SNSTopic"
              },
              "Condition": {
                "ArnLike": {
                  "aws:SourceArn": {
                    "Fn::Join": [
                      "",
                      [
                        "arn:aws:s3:::",
                        {
                          "Ref": "S3Bucket"
                        }
                      ]
                    ]
                  }
                }
              }
            }
          ]
        },
        "Topics": [
          {
            "Ref": "SNSTopic"
          }
        ]
      }
    },
    "S3Bucket": {
      "Type": "AWS::S3::Bucket",
      "Properties": {
        "AccessControl": "BucketOwnerFullControl"
      }
    }
  }
}

Update the stack to add the S3 event notification:

{
  "Resources": {
    "SNSTopic": {
      "Type": "AWS::SNS::Topic"
    },
    "SNSTopicPolicy": {
      "Type": "AWS::SNS::TopicPolicy",
      "Properties": {
        "PolicyDocument": {
          "Id": "MyTopicPolicy",
          "Version": "2012-10-17",
          "Statement": [
            {
              "Sid": "Statement-id",
              "Effect": "Allow",
              "Principal": {
                "Service": "s3.amazonaws.com"
              },
              "Action": "sns:Publish",
              "Resource": {
                "Ref": "SNSTopic"
              },
              "Condition": {
                "ArnLike": {
                  "aws:SourceArn": {
                    "Fn::Join": [
                      "",
                      [
                        "arn:aws:s3:::",
                        {
                          "Ref": "S3Bucket"
                        }
                      ]
                    ]
                  }
                }
              }
            }
          ]
        },
        "Topics": [
          {
            "Ref": "SNSTopic"
          }
        ]
      }
    },
    "S3Bucket": {
      "Type": "AWS::S3::Bucket",
      "Properties": {
        "AccessControl": "BucketOwnerFullControl",
        "NotificationConfiguration": {
          "TopicConfigurations": [
            {
              "Topic": {
                "Ref": "SNSTopic"
              },
              "Event": "s3:ObjectCreated:Put"
            }
          ]
        }
      }
    }
  }
}