Most Network Data Sits Untouched

Statistically speaking, most data on enterprise networks rarely gets accessed after it is written to network storage, according to researchers from NetApp Inc. and the University of California (UC). Evidently, we are too busy writing new data to go back over old data.

Andrew Leung, a computer science researcher at the UC, presented the findings at the USENIX conference in Boston last week. Given those results, organizations might want to consider moving much of their data to slower but less expensive storage units since it rarely gets accessed, he said.

The team studied the traffic that flowed through NetApp's enterprise file servers, which manage more than 22T of material relating to all aspects of the company's business operations.

Leung said the study is the first large-scale examination of network traffic patterns. "How people have been deploying network file systems has been changing over the past five to 10 years," he said. "They are being used more commonly for different kinds of things. So what we would like to know is how this affects the workloads of the network."

During the three-month period that the network was under scrutiny, more than 90 percent of the material on the servers was never accessed. The researchers captured packets encoded using the Common Internet File System protocol, which Microsoft Windows applications use to save data via a network. About 1.5T of data was transferred.

"Compared to the full amount of allocated storage on the file servers, this represents only 10 percent of data," Leung said. "[This] means that 90 percent of the data is untouched during this three-month period."

Moreover, among the files that were opened, 65 percent were only opened once. And most of the rest were opened five or fewer times, though about a dozen files were open 100,000 times or more.

"What this suggests, in general, is that files are infrequently re-accessed," Leung said.

The team also observed that the ratio of data being read from storage versus the amount of data written to storage has changed from what had been seen in previous studies. Bytes written compared to bytes read by a ratio of about 2-1. "Past read-write ratios saw read-to-write ratios of 4-1 or higher," Leung added.

Developers of file systems might want to take into consideration the fact that their creations are spending almost as much time writing data as reading data. "The workloads are becoming more write-oriented, so the decrease in read-only traffic and the increase in write traffic suggests that file systems want to be more write-oriented," Leung said.

File server vendors also might want to consider re-jiggering their pre-fetching and caching algorithms to improve performance, given those findings. "If we know that files aren't frequently re-accessed, what this suggests is that [caching] algorithms may not be the best for network file systems" because the material cached will probably not get retrieved, he said.

Speaking to Government Computer News after the presentation, Leung described the 10 percent of data that was being re-accessed. Typically, it is in the file format most closely associated with the user's job. Architects might use computer-aided design files, while developers use source-code files. Also, files that are higher up in a file path or closer to the user's home directory tend to be accessed more often than those buried deeper down in a hierarchy of subfolders.

More than 75 percent of the files being opened were very small -- less than 20K each -- although another 12 percent were more than 5G each.

About the Author

Joab Jackson is the chief technology editor of Government Computing News (


  • Google IDs on Azure Active Directory B2B Service Now at 'General Availability'

    Microsoft announced on Wednesday that users of the Google identity and access service can use their personal log-in IDs with the Azure Active Directory B2B service to access resources as "guests."

  • Top 4 Overlooked Features of a Data Backup Strategy

    When it comes to implementing an airtight backup-and-recovery plan, these are the four must-have features that many enterprises nevertheless tend to forget.

  • Microsoft Bolsters Kubernetes with Azure Confidential Computing

    Microsoft on Tuesday announced various developments concerning the use of Kubernetes, an open source container orchestration solution fostered by Google.

  • Windows Will Have Support for Encrypted DNS

    Microsoft announced this week that the Windows operating system already has support for an encrypted Domain Name System option that promises to add greater privacy protections for Internet connections.

comments powered by Disqus

Office 365 Watch

Sign up for our newsletter.

Terms and Privacy Policy consent

I agree to this site's Privacy Policy.