Datacenter Trends

eBay 'Replatforming' with Its Own Servers and Decentralized Datacenters

The company's plan to modernize its infrastructure also includes tapping open source technologies and -- as Microsoft and Facebook have done -- open sourcing its server designs in turn.

• By John K. Waters
• 09/11/2018

E-commerce giant eBay is in the middle of a three-year plan to modernize its back end infrastructure with its own custom-designed servers that are "built by eBay, for eBay," the company revealed last week.

eBay is about halfway through the transformation, which it has described as a "replatforming effort." Along with the hardware design and development, the project has involved designing an artificial intelligence (AI) engine; decentralizing the company's cluster of datacenters; moving to an edge computing architecture; and leveraging a range of open source technologies, including Kubernetes, Envoy, MongoDB, Docker and Apache Kafka.

In fact, eBay plans to open source its server designs, just as Facebook, Microsoft and others did via the Open Compute Project starting in 2011. The company plans to make its server designs available to the public in Q4 of this year.

"By sharing our innovations with the community, we are hoping to give back," wrote Mazen Rawashdeh, eBay's vice president of platform engineering, in a blog post.

By taking the massive step for any organization (let alone an enterprise the size of eBay) to design and develop its own servers and hardware, the company reduces its dependence on third parties, Rawashdeh explained, which means that it can attend to the company's unique needs for efficiency and at considerable cost savings in the long term.

"It took us nine months to build our prototype and to deploy our custom hardware," he wrote. "With this shift, we are homogenizing our infrastructure, leading to significant development and operational efficiencies."

The demands on eBay's datacenters today is enormous. The company reportedly processes 300 billion data queries daily to support 175 million active users and more than 1.1 billion live listings, and its overall data footprint is more than 500 petabytes. "To put it into context," Rawashdeh wrote, "500 petabytes is the equivalent of one trillion songs, 2.5 million hours of movies, and enough to back up the American Library of Congress more than 300 times."

One key to this transformation is eBay's decision to decentralize its U.S.-based datacenter clusters. "We are deploying online services and data closer to our users," Rawashdeh explained, "enabling dynamic and static caching capabilities, decreasing latency and improving the experience." By using a Point of Presence (PoP) strategy, eBay is saving 600 to 800 milliseconds of load time, the company reported.

The replatforming project has been a success so far, Rawashdeh said, thanks in no small part to the company's decision to rework both the physical and logical layers of its entire technology stack.

"The stack is like connective tissue," Rawashdeh wrote. "You cannot isolate one of the layers; you must advance them together. To make a meaningful impact, the transformation should be cohesive and orchestrated from end-to-end. We systematically went through each layer of our technology stack and examined efficiency, capability and the opportunity to improve existing solutions."

Deploying online services and data closer to its users allows the company to deliver dynamic and static caching capabilities, decrease latency and improve the customer experience.

The company has also created customized models for the data layer, including a fault-tolerant, geo-distributed object and data store that will allow eBay to distribute data geographically. The result, according to the company: improved customer experience, more resilient services and data isolation solutions for countries that require them.

Another key component of eBay;s plan is support for what it hopes will become a dynamic software development community with its own developer program, where developers will find access to a range of tools and APIs.

"Developers and communities who leverage our tools improve upon what we are building and help us create better experiences overall," Rawashdeh said.