AzCopy Preview Adds AWS S3 Data Transfer Improvements -- Redmondmag.com

AzCopy Preview Adds AWS S3 Data Transfer Improvements

By Kurt Mackie
04/18/2019

Microsoft announced this week that it has improved the preview version of its AzCopy tool to better handle Amazon Web Services (AWS) S3 data.

AzCopy is a command-line tool that's typically used with the Azure File service or the Azure Blob service for copying data. It lets IT pros "copy data between a file system and a storage account, or between storage accounts," according to Microsoft's document description.

The latest AzCopy preview version 10.0.8 tool, released in March, has an improved buckets-to-blobs transfer capability for organizations wanting to move AWS S3 data to Microsoft's Azure Blob Storage service. An entire AWS S3 bucket can be copied. Even multiple buckets can be copied.

Earlier AzCopy versions were less streamlined because the data had to get copied from AWS S3 storage to the client machine running AzCopy before getting transferred to an Azure Storage account, Microsoft explained. The AzCopy preview 10.0.8 tool, though, will transfer data directly to an Azure Storage account.

AzCopy preview version 10.0.08 was described by Microsoft as its "next generation data transfer utility for Azure Storage." The tool was "redesigned from scratch" to better address resiliency and scale issues. Microsoft "rewrote its sync command" and added memory management improvements, as well. The ability of the tool to copy data directly from a URL stems from Microsoft's "new Azure Storage REST API operation Put Block from URL," the announcement explained.

The data copying rate was "50 Gbps" within the same region when moving data from AWS S3 storage to a Microsoft Storage account using the AzCopy preview tool, Microsoft claimed. The tool is suitable for moving terabytes of data, according to Microsoft.

The AzCopy preview also shows up as an option in the GUI-based Azure Storage Explorer tool, according to Microsoft's document. Azure Storage Explorer is used with Azure services, Azure Stack or cloud services to upload and download "blobs, files, queues, tables and Cosmos DB entities," according to Microsoft's description.

Another Microsoft option for moving AWS S3 data to Azure Storage is the Azure Data Factory service. It's a more sophisticated service that works with so-called "hybrid" scenarios (data in the cloud and on-premises). Azure Data Factory provides a "dashboard" for monitoring, the ability to move data over the public Internet or Azure ExpressRoute, the ability to schedule incremental loads, integration with Azure Key Vault for credentials management and multiple connectors support.

About the Author

Kurt Mackie is senior news producer for 1105 Media's Converge360 group.

Featured

Choosing the Right SSID for Your Network

Attracting attention, whether through a default or funny Wi-Fi name, could lead to a security issue.
Microsoft Outlines Latest Security Efforts at Ignite

In the wake of last year's CrowdStrike incident, Microsoft has worked to publicly rehabilitate its security image through new initiatives and public commitments to improve. That push continued this week at Ignite, where Microsoft announced a handful of new security tools and updates.
Nadella on Copilot: 'Major Platform Shifts Are in the Air'

As should be no surprise, Copilot took center stage during Microsoft CEO Satya Nadella's keynote speech to open Microsoft Ignite 2024.
Azure AI Foundry Revealed at Microsoft Ignite

Microsoft is consolidating its Azure generative AI tools into a unified platform named Azure AI Foundry.
Microsoft Bolsters Fabric at Ignite

While Fabric has improved in the year since its release, there are still some glaring issues with the platform.