Embedded stack was not successfully created

If you encounter an error similar to:

Embedded stack arn:aws:cloudformation:<region>:<AWS_ACCOUNT_ID>:stack/<NAME>-MetricStream-<UID> was not successfully created: Transform AWS::Include failed with: S3 URI must reference a valid S3 object to which you have access.

The Observe stack contains an AWS::Include transform which retrieves a template fragment from Amazon S3 at stack creation time. This file is publicly readable, but permission to read the S3 object may be denied based on permissions granted in your AWS Account. If those permissions are missing or misconfigured, the embedded stack creation fails.

Steps to troubleshoot your user or role has the appropriate permissions.

Step 1: Identify the active AWS principal Run the following commands as the user or role creating the stack:

aws sts get-caller-identity

Sample output:

{
    "UserId": "<USER_ID>",
    "Account": "<AWS_ACCOUNT>",
    "Arn": "arn:aws:sts::<AWS_ACCOUNT>:assumed-role/<ROLE_NAME>/<ROLE_SESSION_NAME>"
}

Step 2: Reformat the role, change sts to iam and assumed-role to role:

arn:aws:sts::<AWS_ACCOUNT>:assumed-role/<ROLE_NAME>
# now becomes
arn:aws:iam::<AWS_ACCOUNT>:role/<ROLE_NAME>

Step 3: Run the following using --policy-source-arn as the above:

aws iam simulate-principal-policy \
  --policy-source-arn arn:aws:iam::<AWS_ACCOUNT>:role/<ROLE_NAME> \
  --action-names s3:GetObject \
  --resource-arns arn:aws:s3:::observeinc/cloudwatchmetrics/filters/default.yaml \

This command evaluates the effective permissions of the IAM role and explicitly identifies which IAM policy statements allow or deny the s3:GetObject request. The output highlights the exact policy (managed or inline) responsible for the decision, making it possible to quickly determine whether the failure is due to a missing permission, an explicit deny, or a higher-level control such as a permission boundary or SCP.

Sample output:

{
    "EvaluationResults": [
        {
            "EvalActionName": "s3:GetObject",
            "EvalResourceName": "arn:aws:s3:::observeinc/cloudwatchmetrics/filters/default.yaml",
            "EvalDecision": "allowed",
            "MatchedStatements": [
                {
                    "SourcePolicyId": "AdministratorAccess",
                    "SourcePolicyType": "IAM Policy",
                    "StartPosition": {
                        "Line": 3,
                        "Column": 17
                    },
                    "EndPosition": {
                        "Line": 8,
                        "Column": 6
                    }
                }
            ],
            "MissingContextValues": [],
            "EvalDecisionDetails": {},
            "ResourceSpecificResults": [
                {
                    "EvalResourceName": "arn:aws:s3:::observeinc/cloudwatchmetrics/filters/default.yaml",
                    "EvalResourceDecision": "allowed",
                    "MatchedStatements": [
                        {
                            "SourcePolicyId": "AdministratorAccess",
                            "SourcePolicyType": "IAM Policy",
                            "StartPosition": {
                                "Line": 3,
                                "Column": 17
                            },
                            "EndPosition": {
                                "Line": 8,
                                "Column": 6
                            }
                        }
                    ],
                    "MissingContextValues": []
                }
            ]
        }
    ]
}

Stack creates, but no data was received

The stack (by default) also requires permissions to read and write data to Observe. The stack creates roles to allow the resources it creates to function, but if you have organizational-wide policies, such as SCPs, these actions may be blocked regardless. If this happens, please adjust your SCP to allow the actions.

If you are collecting metrics via Metric Streams, you will need to allow the following actions:

• s3:PutObject
• s3:PutObjectTagging

You can allow these actions for all buckets, and later scope the permissions down to the specific access point that the stack reaches out to. To do this, just before deploying the stack in AWS, note the DataAccessPointArn in the filled in template - it will look something like this: arn:aws:s3:us-west-2:123456789012:accesspoint/123412341234. Then, allow the above actions for this specific access point. Your policy might look something like this:

{
  "Version": "2012-10-17",
  "Statement": [
    {
      "Sid": "DenyPutOutsideAllowedBuckets",
      "Effect": "Deny",
      "Action": [
        "s3:PutObject",
        "s3:PutObjectTagging"
      ],
      "Resource": "*",
      "Condition": {
        "StringNotEquals": {
          "aws:RequestedRegion": [
            "us-east-1"
          ]
        }, 
        "StringNotLike": {
          "s3:DataAccessPointArn": [
              "arn:aws:s3:us-west-2:123456789012:accesspoint/123412341234"
          ]
        }
      }
    }
	]
}

Assuming s3:PutObject and s3:PutObjectTagging have been allowed in all regions, the above policy will limit the permissions to us-east-1 and specifically the required DataAccessPointArn" (which can be outside of us-east-1).

If you are collecting files from other buckets with the Forward files from S3 buckets option, you will need:

• s3:ListBucket
• s3:GetObject
• s3:GetObjectTagging

You will need to enable s3:ListBucket for all buckets you wish to collect from, and the other actions for all objects within the buckets.

S3 CopyObject AccessDenied

When integrating AWS Config with Observe, you may encounter errors such as:

error copying file "s3://aws-controltower-logs-<AWS_ACCOUNT>-<AWS_REGION>/<FILE_NAME>": operation error S3: CopyObject, https response error StatusCode: 403, RequestID: <REQUEST_ID>, HostID: <HOST_ID>, api error AccessDenied: Access Denied

This typically indicates a permissions or policy issue with S3, KMS, or AWS Control Tower configurations.

Check KMS Key Policy (if using SSE-KMS)

If the S3 bucket is encrypted with KMS, ensure the IAM role has kms:Decrypt and kms:DescribeKey permissions. Example KMS Policy:

{
  "Sid": "AllowObserveFiledropRoleDecryptConfigLogs",
  "Effect": "Allow",
  "Principal": {
    "AWS": "arn:aws:iam::<AWS_ACCOUNT>:role/<FORWARDER_ROLE>"
  },
  "Action": [
    "kms:Decrypt",
    "kms:DescribeKey"
  ],
  "Resource": "*"
}

If using SSE-S3, KMS is not a factor.

Check for Service Control Policies (SCPs) or GuardRails

SCPs at the AWS Organization level can override bucket and key policies.

Common issues:

• Deny rules for principals outside the organization (aws:PrincipalOrgID).
• Deny rules for all except specific roles.

Example restrictive SCP:

{
  "Effect": "Deny",
  "Action": "*",
  "Resource": "*",
  "Condition": {
    "StringNotEquals": {
      "aws:PrincipalOrgID": "<org-id>"
    }
  }
}

In the above example, to allow the Observe forwarder role while keeping the guardrail in place, add a targeted exception for the role. Because SCPs use explicit deny semantics and condition logic is easy to misapply, the exception must ensure the Deny statement does not match the forwarder’s principal ARN.

{
  "Sid": "DenyAllExceptOrgAndObserveRole",
  "Effect": "Deny",
  "Action": "*",
  "Resource": "*",
  "Condition": {
    "StringNotEquals": {
      "aws:PrincipalOrgID": "<org-id>",
      "aws:PrincipalArn": "arn:aws:iam::<AWS_ACCOUNT>:role/<FORWARDER_ROLE>"
    }
  }
}

Here's an example script using AWS CLI to list all SCPs and check for Deny rules:

for POLICY_ID in $(aws organizations list-policies --filter SERVICE_CONTROL_POLICY --query 'Policies[].Id' --output text)
do
  POLICY_NAME=$(aws organizations describe-policy --policy-id $POLICY_ID --query 'Policy.PolicySummary.Name' --output text)
  echo "Checking $POLICY_NAME ($POLICY_ID) ..."
  CONTENT=$(aws organizations describe-policy --policy-id $POLICY_ID --query 'Policy.Content' --output text)
  if echo "$CONTENT" | grep -q '"Effect": *"Deny"'; then
    echo "=== DENY FOUND in $POLICY_NAME ($POLICY_ID) ==="
    echo "$CONTENT" | jq .
    echo "================================"
  fi
done >> scp-deny.rslt