Tycloud DC

At A Glance:

AWS DataSync is a fully managed, high-speed transfer service that automates the movement of large amounts of data between locations. It can move files from on-premises systems or edge environments into AWS services like S3, EFS, or FSx, as well as to competing clouds such as Azure or Google Cloud. Organizations typically use it for migrations, backups, archiving, and hybrid cloud synchronization, which makes otherwise complex operations much easier to manage with less overhead.

How it Works:

While DataSync historically operated in Basic mode (using an agent spun up on a VM or physical hardware), AWS introduced Enhanced mode in October 2024, adding agentless support for storage across AWS, on-premises hardware, and other clouds. This new mode improved the listing, preparation, transfer, and verification of user data, delivering virtually unlimited scale. Most notably, enhanced mode enables agentless transfers, meaning no extra on-premises or VM infrastructure is required. Data can move securely between AWS services or across clouds without deploying a DataSync agent! This reduces operational overhead and complexity while still providing encryption, checksums, and detailed logging. Pricing for the enhanced service remains pay as you go:

As of August 2025:

For up to date information, please visit the AWS DataSync Pricing Guide

Use Case Highlight:

A recently exciting discovery was that DataSync can also bridge the AWS Commercial and AWS GovCloud Partitions – a task that is traditionally difficult to automate and not well documented (with either agentless or serverless methods).  By specially configuring a few settings in the AWS DataSync Task, a user using enhanced mode can securely authenticate to their bucket using an AWS IAM user as a service account. This capability makes it easier to support hybrid environments and move data between commercial and GovCloud workloads when needed.

DataSync can also bridge the AWS Commercial and AWS GovCloud Partitions – a task that is traditionally difficult to automate

How to Configure DataSync to sync data across Partitions:

In this example, a user needs to sync VM log data from the AWS Commercial partition into AWS GovCloud. For security reasons, data should not flow from a higher security partition into a lower one, so the GovCloud DataSync task will pull the data from Commercial instead…but hey, you do you, I’m not your CISO! 😋

Please note: These instructions are written assuming you have basic knowledge of AWS IAM, S3, and DataSync.

0.) Prerequisites: An S3 bucket already created in both commercial (source) and GovCloud (destination)

1.) Begin by logging into your AWS Commercial Account and navigating to IAM -> Policies -> Create Policy and insert this JSON, making sure to replace COMMERCIAL-BUCKET-NAME and COMMERCIAL-ACCOUNT-ID with correct values, then click ‘save’.

{
 "Version": "2012-10-17",
 "Statement": [
     {
         "Action": [
             "s3:GetBucketLocation",
             "s3:ListBucket",
             "s3:ListBucketMultipartUploads"
         ],
         "Effect": "Allow",
         "Resource": "arn:aws:s3:::COMMERCIAL-BUCKET-NAME",
         "Condition": {
             "StringEquals": {
                 "aws:ResourceAccount": "COMMERCIAL-ACCOUNT-ID"
             }
         }
     },
     {
         "Action": [
             "s3:GetObject",
             "s3:GetObjectTagging",
             "s3:GetObjectVersion",
             "s3:GetObjectVersionTagging",
             "s3:ListMultipartUploadParts"
           ],
         "Effect": "Allow",
         "Resource": "arn:aws:s3:::COMMERCIAL-BUCKET-NAME/*",
         "Condition": {
             "StringEquals": {
                 "aws:ResourceAccount": "COMMERCIAL-ACCOUNT-ID"
             }
         }
     }
 ]
}

This policy will grant your IAM user read only access to your S3 bucket. Because we only ever plan to fetch data from commercial (one way sync), the user will never need write permissions.

2.) Create a new IAM user and attach the policy to this user account.

3.) Once the user has been created, navigate to the newly created user account in IAM and generate Access Keys for this user. Save these keys for now – you will need them again in a future step.

4.) Switch over into the GovCloud Account, then navigate to AWS DataSync -> Create Task. Here we will configure the Source Location. Fill out the values making sure to replace/edit the below parameters.

Please note: The source ‘Server’ is region dependent, so be sure to replace the {region} with the region the source bucket is located in.

Server: https://s3.{region}.amazonaws.com

In my example, my source bucket is in us-east-1, so I use:

https://s3.us-east-1.amazonaws.com

5.) Scroll down to see the ‘Authentication’ Section and fill out these values with the Commercial Access Key and Secret generated in step 3.

6.) Navigate to the Destination Location page and fill out the appropriate values:

7.) Selecting ‘Next’ brings you to the ‘Configure Settings’ page. Be sure to fill this out based on your needs. Here, scheduled runs, file exclusions, and data integrity settings can be configured.

8.) Once completed, save and run the task. After a few minutes, you should see a success message and that the data has synced!