Copying S3 objects across accounts

It is possible to copy objects between accounts, but it takes a little massaging to achieve

Definitons

Source account - The account with the S3 bucket that you want to copy into another account

Destination account - The account with the S3 bucket that you want to copy the objects into

ACL - Access Control List (defines permissions attached to entities on buckets/objects) (AWS Documentation)

Canonical ID - Hexadecimal ID assigned to each account by AWS (not to be mistaken with Account ID - the 12-digit number used as part of ARNs and to login) (AWS Documentation)

Pre-requisites

• A user in the source account with S3 read/write permissions (s3:GetObject, s3:ListBucket, s3:HeadBucket on the S3 source bucket/objects, s3:PutObject, s3:PutObjectAcl on destination bucket/objects)
• A user in the destination account with S3 S3 bucket management permissions (s3:PutBucketAcl)
• CLI Setup with credentials in the source account
• Bucket with data in the source account
• Bucket in the destination account

1. Getting the Canonical ID of the source account

Via the console

1. Login to the source account via the AWS Console
2. Navigate to S3
3. Click on the name of the source bucket
4. Click on the 'Permissions' tab
5. Click on the 'Access Control List' button
6. Under the first table 'Access for bucket owner' copy the Canonical ID (this is the Canonical ID for the source account)

Via the CLI

Using a profile/credentials for the source account

aws s3api list-buckets --query "Owner.ID"

2. Preparing the destination bucket

1. Login to the destination account via the AWS console
2. Navigate to S3
3. Click on the name of the destination bucket
4. Click on the 'Permissions' tab
5. Click on the 'Access Control List' button
6. Under the Access for other AWS accounts section, click the + Add account button
7. Enter the Canonical ID of the Source account
8. Tick at least the List objects and Write objects check boxes
9. Click Save

3. Copying the objects across

As the aws cp command will (by default) preserve the ACL on the objects (i.e. restricts read/write permissions to the source account) therefore we have to set the ACL for the objects when we copy them in order to allow the destination account to read and manage these objects.

Using a profile/credentials for the source account:

aws s3 cp --recursive s3://example-source-bucket/ s3://example-destination-bucket/ \
  --acl bucket-owner-full-control

Obviously this command can be shortened if you want to only copy a single object or specified key prefix, but this command copies all objects from the source bucket into the destination bucket.

The --acl flag provides the destination account full access to all the objects by setting the ACL for the objects to full. This can be modified to read if the destination account should only be able to read the files. In this case it might be better to modify the bucket in the source account and grant read-only permissions to the destination account.