AzCopy Preview Adds AWS S3 Data Transfer Improvements -- Redmondmag.com

AzCopy Preview Adds AWS S3 Data Transfer Improvements

By Kurt Mackie
04/18/2019

Microsoft announced this week that it has improved the preview version of its AzCopy tool to better handle Amazon Web Services (AWS) S3 data.

AzCopy is a command-line tool that's typically used with the Azure File service or the Azure Blob service for copying data. It lets IT pros "copy data between a file system and a storage account, or between storage accounts," according to Microsoft's document description.

The latest AzCopy preview version 10.0.8 tool, released in March, has an improved buckets-to-blobs transfer capability for organizations wanting to move AWS S3 data to Microsoft's Azure Blob Storage service. An entire AWS S3 bucket can be copied. Even multiple buckets can be copied.

Earlier AzCopy versions were less streamlined because the data had to get copied from AWS S3 storage to the client machine running AzCopy before getting transferred to an Azure Storage account, Microsoft explained. The AzCopy preview 10.0.8 tool, though, will transfer data directly to an Azure Storage account.

AzCopy preview version 10.0.08 was described by Microsoft as its "next generation data transfer utility for Azure Storage." The tool was "redesigned from scratch" to better address resiliency and scale issues. Microsoft "rewrote its sync command" and added memory management improvements, as well. The ability of the tool to copy data directly from a URL stems from Microsoft's "new Azure Storage REST API operation Put Block from URL," the announcement explained.

The data copying rate was "50 Gbps" within the same region when moving data from AWS S3 storage to a Microsoft Storage account using the AzCopy preview tool, Microsoft claimed. The tool is suitable for moving terabytes of data, according to Microsoft.

The AzCopy preview also shows up as an option in the GUI-based Azure Storage Explorer tool, according to Microsoft's document. Azure Storage Explorer is used with Azure services, Azure Stack or cloud services to upload and download "blobs, files, queues, tables and Cosmos DB entities," according to Microsoft's description.

Another Microsoft option for moving AWS S3 data to Azure Storage is the Azure Data Factory service. It's a more sophisticated service that works with so-called "hybrid" scenarios (data in the cloud and on-premises). Azure Data Factory provides a "dashboard" for monitoring, the ability to move data over the public Internet or Azure ExpressRoute, the ability to schedule incremental loads, integration with Azure Key Vault for credentials management and multiple connectors support.

About the Author

Kurt Mackie is senior news producer for 1105 Media's Converge360 group.

Featured

Using Configuration Files to Control PowerShell Scripts, Part 2: Building Blocks

Learn how to use PowerShell's GUI capabilities to dynamically create configuration files that enhance script flexibility and usability.
U.S. House Leaders Unveil Bipartisan Strategy for AI Governance and Innovation

On Tuesday, U.S. House Speaker Mike Johnson and Democratic Leader Hakeem Jeffries received the final report from the bipartisan House Task Force on Artificial Intelligence, which outlines essential principles and recommendations to bolster American leadership in AI.
Using Configuration Files to Control PowerShell Scripts, Part 1: The Basics

Learn how to use JSON configuration files to make PowerShell scripts more dynamic, flexible and user-friendly.
Exchange Online Receiving Enhanced Message Trace

Microsoft has announced the public preview of its enhanced Message Trace feature in the Exchange admin center (EAC) for Exchange Online.
Major Microsoft Multifactor Authentication Vulnerability Revealed

A vulnerability in Microsoft’s multifactor authentication (MFA) system has been uncovered by cybersecurity firm Oasis Security, allowing attackers to bypass security measures and access sensitive data across services including Outlook, OneDrive, Teams and Azure.