News

Old IPv4 Router Settings May Be Causing Internet Slowdowns This Week

• By Kurt Mackie
• 08/14/2014

Internet performance degradations may occur this week and the next due to settings on some IPv4 routers that are used across the Internet.

In a nutshell, the number of routes across the Internet is beginning to exceed the number that IPv4 routers were set to handle. Internet service providers (ISPs) still use older IPv4 routing equipment in their networks, despite an impending shortage in the number of IPv4 Internet addresses (which is a different problem). However, many of those IPv4 routers were installed in ISP networks with default settings for handling a maximum of 512,000 routes. This year, though, the Internet has already exceeded 500,000 routes, according to assessments by various networking companies (see chart).

The actual tracking of Internet routes is performed in networking devices using a table that's accessed via Ternary Content Addressable Memory (TCAM) technology, which speeds up the access process. Network slowdowns or problems can occur if the number of routes to track exceeds this 512,000 default limit set in older IPv4 routers.

TCAM Resource Exhaustion
The 512,000 routes milestone has been long understood as a potential problem by various networking companies. Cisco noted three months ago that ISPs and organizations should prepare for possible TCAM resource exhaustion when using some of its networking products. Potentially affected Cisco products include Catalyst 6500 Switches, 7600 Series Routers, ASR 9000 Series Aggregation Services Routers with Trident-based line cards and ASR 1000 Series Aggregation Services Routers (4 GB).

Cisco offers a workaround here, which involves changing the default configurations on those products. Essentially, the workaround reallocates some shared routing entities from IPv6 to IPv4.

Internet Hiccups
Networking companies appear to have accepted the 512,000 Internet routes milestone as an agreed-upon consensus measure, but individual IPv4 routers used by ISPs and organizations may vary on that limit. It's also argued whether the Internet actually exceeded that 512,000 routes milestone. However, Vancouver, BC-based BGPMon, specializing in network monitoring and security technologies, claimed that the 512,000 routes mark was exceeded on Aug. 13 at 1:00 a.m. Pacific Time (or 8:00 am UTC).

BGPMon correlated the Aug. 13 milestone with reports of "hiccups on the Internet" happening on that same day by observing Level 3 customer data in Chicago. Other observers have anecdotally reported Level 3 service outages happening on August 12. BGPMon pointed to Verizon as the main cause for the spike in Internet routes, adding that Verizon quickly resolved the issue on its network.

ISPs and organizations can modify their routers settings or replace them with IPv6-based equipment, but that's likely being done on a case-by-case basis, which explains why some Internet users could experience service disruptions or outages in the interim. Those disruptions possibly could occur next week as ISPs respond to "underprovisioned equipment," according to a discussion of the problem by Internet performance company Renesys/Dyn.

Organizations should check for any latency issues with their Internet connections next week, according to Renesys.

"Enterprises that rely on the Internet for delivery of service should pay close attention to the latency and reachability of the paths to customers in the coming weeks, in order to identify affected service providers upstream and work around them while they perform appropriate upgrades to their infrastructure," the company advised.

Renesys described the routing-table issue as "more of an annoyance than a real Internet-wide threat." Routers with the settings limitations will be identified quickly by operators, Renesys contended. Possibly, all of the offending networking gear will be discovered "within a few weeks."