Microsoft Targets IBM Watson with Azure Machine Learning in Big Data Race

Nearly a year after launching its Hadoop-based Azure HDInsight cloud analytics service, Microsoft believes it's a better and broader solution for real-time analytics and predictive analysis than IBM's widely touted Watson. Big Blue this year has begun commercializing its Watson technology, made famous in 2011 when it came out of the research labs to appear and win on the television game show Jeopardy.

Both companies had a large presence at this year's Strata + Hadoop World Conference in New York, attended by 5,000 Big Data geeks. At the Microsoft booth, Eron Kelly, general manager for SQL Server product marketing, highlighted some key improvements to Microsoft's overall Big Data portfolio since last year's release of Azure HDInsight including SQL Server 2014 with support for in-memory processing, PowerBI and the launch in June of Azure Machine Learning. In addition to bolstering the offering, Microsoft showcased Azure ML's ability to perform real-time predictive analytics for the retail chain Pier One.

"I think it's very similar," in terms of the machine learning capabilities of Watson and Azure ML, Kelly said. "We look at our offering as a self-service on the Web solution where you grab a couple of predictive model clips and you're in production. With Watson, you call in the consultants. It's just a difference fundamentally [that] goes to market versus IBM. I think we have a good advantage of getting scale and broad reach."

Not surprisingly, Anjul Bhambhri, vice president of Big Data for IBM's software group disagreed. "There are certain applications which could be very complicated which require consulting to get it right," she said. "There's also a lot of innovation that IBM has brought to market around exploration, visualization and discovery of Big Data which doesn't require any consulting." In addition to Watson, IBM offers its InfoSphere BigInsights for Hadoop and Big SQL offerings.

As it broadens its approach with a new "data culture," Microsoft has come on strong with Azure ML, noting it shares many of the real-time predictive analytics of the new personal assistant in Windows Phone called Cortana. Now Microsoft is looking to further broaden the reach of Azure ML with the launch of a new app store-type marketplace where Microsoft and its partners will offer APIs consisting of predictive models that can plug into Azure Machine Learning.

Kicking off the new marketplace, Joseph Sirosh, Microsoft's corporate VP for information management and machine learning, gave a talk at the Strata + Hadoop conference this morning. "Now's the time for us to try to build the new data science economy," he said in his presentation. "Let's see how we might be able to build that. What do data science and machine learning people do typically? They build analytical models. But can you buy them?"

Sirosh said with Microsoft's new data section of the Azure Marketplace, marketplace developers and IT pros can search for predictive analytics components. It consists of APIs developed both by Microsoft and partners. Among those APIs from Microsoft are Frequently Bought Together, Anomaly Detection, Cluster Manager and Lexicon Sentiment Analysis. Third parties selling their APIs and models include Datafinder, MapMechanics and Versium Analytics.

Microsoft's goal is to build up the marketplace for these data models. "As more of you data scientists publish APIs into that marketplace, that marketplace will become just like other online app stores -- an enormous of selection of intelligent APIs. And we all know as data scientists that selection is important," Sirosh said. "Imagine a million APIs appearing in a marketplace and a virtual cycle like this that us data scientists can tap into."

Also enabling the real-time predictive analytics support is support for Apache Storm clusters, announced today. Though it's in preview, Kelly said Microsoft is adhering to its SLAs with use of the Apache Storm capability, which enables complex event processing and stream analytics, providing much faster responses to queries.

Microsoft also said it would support the forthcoming Hortonworks Data Platform, which has automatic backup to Azure BLOB storage, Kelly said. "Any Hortonworks customer can back up all their data to an Azure Blob in a real low cost way of storing their data, and similarly once that data is in Azure, it makes it real easy for them to apply some of these machine learning models to it for analysis with Power BI [or other tools]."

Hortonworks is also bringing HDP to Azure Virtual Machines as an Azure certified partner. This will bring Azure HDInsight to customers who want more control over it in an infrastructure-as-a-service model, Kelly said. Azure HDInsight is currently a platform as a service that is managed by Microsoft.

Posted by Jeffrey Schwartz on 10/17/2014 at 9:54 AM