Dell Ups Datacenter Infrastructure Ante with New Liquid Cooling Tech

Every so often, someone reveals a major advance in keeping datacenter infrastructure cool -- the key to gaining efficiencies in running some of the largest deployments of server clusters. Dell, with major help from Intel, says it has found a breakthrough approach to using liquid-based cooling. Revealed yesterday, Dell said eBay will be the first to deploy this new hyper-converged system as a proof of concept.

The new technology is the latest iteration by the Dell Extreme Scale Infrastructure Group to push the envelope in applying liquid cooling to the rack in a manner that can push the performance and cost-effectiveness of deploying scale-out infrastructure for sites and compute functions that generate massive numbers of transactions per second. Running water through computing racks is considerably more expensive on a per-gallon basis than using forced-air cooling methods. But according to Dell officials, water is 25 times more efficient at removing heat from the CPUs (critical for ensuring reliability and performance) than air is, watt for watt.

At a briefing in Dell's ESI lab in Austin, Texas this week, Dell officials demonstrated the new system code-named Triton. Dell is not the first company to deliver liquid-cooling technology. IBM actually first started doing so for its mainframes back in the 1970s. Two years ago, Hewlett Packard (now known as Hewlett Packard Enterprise), introduced the Apollo 8000, a supercomputer-class system featuring liquid-based cooling.

Dell and Intel collaborated on a customized 200W 20-core Intel Xeon E5 Broadwell CPU that they claim can provide "double-digit" performance. By removing liquid-to-liquid heat exchangers of water pumping systems typically found in datacenter liquid-cooling racks, Dell is able to achieve higher efficiency and lower water consumption. Dell claims Triton has a power usage effectiveness (PUE) rating as low as 1.02 and requires 97 percent less datacenter cooling power than typical air-cooled systems and 62 percent less power than HPE's Apollo 8000.

The cooling technology enables the processor to stay in turbo mode all the time. The system pushes water at high pressure rates without having to add additional cooling to it. The water is distributed through copper pipes with specially sealed joints that support 350 PSI, based on current tests, and is rated above 5,000 PSI.

"Triton, unlike a lot of other liquid cooling solutions, is unique in that we don't have a distribution unit and we don't have a secondary cooling unit," explained Austin Shelnutt, principal thermal architect for datacenter solutions at Dell, who demonstrated Triton. "We bring that facility water directly into each one of our server nodes. Do not pass go. Collect $200 go straight to the node and strike the heat right from the CPU."

Each sled in the unit has the customized Intel Xeon Broadwell processor, four 2.5-inch hard disk drives and each has its own node power distribution board with leak detection sensors that, in the event of a leak, will kill off water flow into just that sled. The power to the sled is also shut off and an alert is issued via the management interface.

The system still does have fans to cool off the the PCIe devices, memory and hard drives, though the liquid-distribution technology provides 80 percent of the cooling, Shelnutt said. (Dell released a video with Shelnutt demonstrating Triton, which is accessible here.)

Dell and Intel designed this to meet the search performance required by eBay. In a statement, Nick Whyte, eBay's VP and fellow for search technology, said Triton produced a 70 percent increase in throughput in terms of queries per second (QPS). While not everyone has a need for such performance, Triton could find itself in more mainstream datacenter environments over time, said Gina Longoria, a senior analyst at Moor Insights and Technology.

"Triton today is designed for very large scale customers so it will only be relevant to a small set of customers," Longoria said. "Dell is currently looking at a 'closed loop' version of Triton that offers the same core liquid cooling technology and CPU support but removes the need for datacenters to have facility water at the rack. This has the potential to bring liquid cooling to an even broader set of scale-out customers." While liquid cooling will remain a technology used by a limited set of customers in the near term, she said it could be a good fit for customers with CPU-intensive workloads or those that have unique energy efficiency requirements. "It may be used more broadly as the technology evolves to be more easily integrated into existing datacenter," she said.

The fact that Triton can use direct cooling tower water with filtration and sensors to allow it to do so is what gives it an edge today, Longoria added. "Other systems, like [HPE's] Apollo, require a chilled water loop which is an intermediate exchange and use energy to cool the water," she said. "This makes the facilities preparation and operations less expensive with Triton which could result in an overall TCO advantage. Also since Triton requires no intermediate liquid-to-liquid heat exchange, it is more effective in removing heat. This theoretically allows the Triton system to cool components that other liquid-cooled solutions can't."

Posted by Jeffrey Schwartz on 06/03/2016 at 2:01 PM