News

Microsoft Previews New Azure Purview Data Governance Service, Goes Live with Azure Synapse Analytics

• By Kurt Mackie
• 12/03/2020

Microsoft on Thursday announced the preview release of Azure Purview, a new data governance solution, as well as the "general availability" commercial release of Azure Synapse Analytics and Azure Synapse Studio.

Azure Synapse Analytics was introduced a year ago by CEO Satya Nadella at Microsoft's Ignite event. Formerly known as "Azure SQL Data Warehouse," Azure Synapse Analytics was redesigned to bolster the number of queries that could be run, while also adding a connection to Apache Spark. Now, the Azure Synapse Analytics product is commercially released.

Organizations using Azure SQL Data Warehouse don't have to move to Azure Synapse Analytics, though.

"If a customer does not want to use Azure Synapse at GA, they will be able to continue using their existing data warehouses, operations, automation, and tooling in dedicated SQL pools (formerly SQL DW) with no changes," Microsoft indicated in a Q&A session on Thursday.

The use of Azure Synapse Analytics requires having an Azure Data Lake Generation 2 account, Microsoft indicated. It serves as the default storage space.

Azure Purview Preview
The Azure Purview preview, which went by the internal Microsoft code name of "Project Babylon," is separate product that can be used with Azure Synapse Analytics. Azure Purview is new data governance solution that's integrated with the Microsoft Information Protection service. It relies on the same "sensitivity labels" approach of Microsoft Information Protection to classify an organization's data.

Microsoft suggested that Azure Purview can help protect data handled by non-Microsoft products, as well:

By extending Microsoft Information Protection's sensitivity labels with Azure Purview, organizations can now automatically discover, classify, and get insight into sensitivity across a broader range of data sources such as SQL Server, SAP, Teradata, Azure Data Services, and Amazon AWS S3, helping to minimize compliance risk.

Microsoft began working on Azure Purview after it introduced Azure Synapse Analytics last year. Azure Purview can work with other cloud-based and on-premises solutions, but it's also integrated with the Microsoft 365 Compliance Center, various Azure services, the Power Platform and SQL Server, according to a Microsoft-produced chart.

The availability of the Azure Purview preview was described during the Q&A portion of the talk. Here's that description:

Azure Purview metadata storage is currently available in 5 Azure regions (East US, East US 2, West Europe, South East Asia, Brazil South) and in-region scanning from 16 Azure regions including North Europe. Our goal is the make the metadata storage also available in North Europe by GA.

The Azure Purview preview is free to try until "January 2021," Microsoft indicated.

Keynotes and Demos
The two announcements were prefaced by an online event on Thursday (soon available on-demand) that made them a big splash as it was fronted by Nadella.

Nadella talked about the future of data and analytics in general and the need for digital transformation in organizations. To get there, "tech intensity" will be needed, which involves adopting technology, building a technology capability and gaining trust in business alignment. This tech intensity will help empower employees, optimize operations, secure supply chains and transform business models, he argued.

The online event also included a discussion of Microsoft's efforts to break down its own data silos and make information more accessible internally in Microsoft, which featured a dialog on that topic between Julia White, corporate vice president of Microsoft Azure, and Amy Hood, executive vice president and chief financial officer. The concept of addressing siloed data is also summarized in this announcement by White.

Azure Synapse product-use success stories were highlighted during the talk by Judson Althoff, executive vice president of Microsoft's worldwide commercial business. Althoff introduced luminaries from Starbucks, gsk (GlaxoSmithKline) and Grab, Southeast Asia's "largest mobile technology company," to talk about their use of the technologies.

Technical aspects of the products were described by Rohan Kumar, corporate vice president for Microsoft Azure Data. The talk also featured a couple of demos.

Q&A Segment
Microsoft officials answered hundreds of questions about Azure Synapse Analytics and Azure Purview during its Thursday online event.

Microsoft was asked if Azure Synapse Analytics will be replacing SQL Server Analysis Services and SQL Server Integration Services (SSIS). It appears not, according to the response:

Synapse pipelines enable you to perform data integration and ingestion capabilities, so it is a cloud-based alternative to SSIS. Azure Synapse SQL pools provide data warehousing capabilities enabling you to run reporting and BI workloads directly on it. SSAS can still be used on top of the SQL pools to provide multi-dimensional modeling and cube capabilities on top of the data warehouse, but it is optional.

Azure Synapse Analytics also is not replacing the Azure Databricks service. Microsoft indicated that while they are both based on Apache Spark, "they each have different scenarios they support in different ways."

Microsoft was asked if Azure Synapse works with the Amazon Web Services platform and said it doesn't.

Azure Purview isn't currently considered to be replacing the Azure Data Catalog service. "There is no change in status for Azure Data Catalog which continues to be a fully supported generally available Azure service," Microsoft indicated in the Q&A.

However, the Azure Purview product did evolve from the Azure Data Catalog service, Microsoft explained:

Azure Data Catalog Gen 2 program got rolled into Azure Purview which is public preview as of today. Azure Purview includes a Data Catalog designed for non-technical users to automatically populate the Purview Data Map and search for assets in the Data Map at no additional charge. For developers, Purview Data Map includes an open-source Apache Atlas 2.0 API to populate and query the Data Map programmatically.

Microsoft was asked if Azure Purview will integrate with other databases "such as Oracle." The response was that "support for Oracle is on our roadmap." Here's another characterization of its database support:

Azure Purview already supports scanning from other databases vendors like Teradata and SAP S4/HANA on-premises. In addition, scanning of documents in Amazon S3 is currently in Private Preview. We will add support for 10+ additional data sources in H1 CY2021 and continue to add more.

Whether these products are just for large organizations tapping "big data," Microsoft indicated that they will work across "a broad set of platforms including relational databases such as Azure SQL and SQL Server, Azure Blob and AWS S3."

Microsoft was asked about the future of its HDInsight big data analytics service, and answered by saying that "HDInsight will continue to evolve as a product and we continue to invest in it."