Power BI Soon Will Query Datasets with 1 Trillion Rows

Microsoft's Power BI can now query 10 billion rows of data, but a forthcoming release will blow that threshold to 1 trillion, a capability demonstrated at this week's annual PASS Summit, where the company also released the first on-premises version with Power BI Report Server.

Microsoft gave Power BI major play at PASS, held in Seattle, where the company also underscored the recently released SQL Server 2017 and its support for Linux and Docker, hybrid implementations of the on-premises database with SQL Azure and its next-generation NoSQL database CosmosDB.

Christian Wade, a Microsoft senior program manager, demonstrated the ability to search telemetry data from the cell phones of 20 million people traveling across the U.S., picking up their location data and battery usage as often as 500 times per day from each user. Though it wasn't actual usage data, Wade demonstrated how long it was taking people to reach various destinations based on their routes by merely dragging and dropping data from the Power BI dashboard interface. Wade queried Microsoft's Spark-based Azure HD Insights service.

"This is what you will be able to do with the Power BI interface and SQL Azure Analysis Services," Wade said during a brief interview following his demonstration. Wade performed the demo during a session focused on Power BI Wednesday, which followed the opening keynote by Rohan Kumar, Microsoft's general manager of database engineering. In the main keynote Wade demonstrated a query against a 9 TB instance with 10 billion rows supported in the current release.

"This is a vision of what's to come, of how we are going to unlock these massive data sets," Wade said, regarding the prototype demo. According to Wade, this new capability demonstrated was the first time anyone was able to use Power BI to perform both direct and in-memory queries against the same tabular data engine,

Kamal Hathi, general manager of Microsoft's BI engineering team, described the new threshold as a key milestone. "We have a history with analysis services and using the technology to build smart aggregations and apply them to large amounts of data," Hathi said in an interview. "It's something we have been working on for many years. Now we are bringing it to a point where we can bring it to standard big data systems like Spark and with the power of Power BI."

While he wouldn't say when it will be available, or if there will be a technical preview, Hathi said it's likely Microsoft will release it sometime next year.

What is now available is the new Power BI Report Server, which brings the SaaS-based capability on-premises for the first time. It still requires the SaaS service, but addresses organizations that can't let certain datasets leave their own environments.

Microsoft is offering the Power BI Report Server only for its Power BI Premium customers -- typically large organizations that pay based on capacity rather than on a per-user basis. The Power BI Report Server lets users embed reports and analytics directly into their apps by using the APIs of the platform, Kumar said during the keynote. "It essentially allows you to create report on-premises using your Power BI desktop tool. And once your report gets created you can either save it on premises to the reporting server [or the] cloud service," Kumar said. "The flexibility is quite amazing."

Posted by Jeffrey Schwartz on 11/03/2017 at 1:30 PM