Warning: Stop setting BGP timers to 1 second on your peering links

I thought I was being clever tuning BGP timers down to 1 second on our transit links in Dallas. Figured it'd speed up failover and make us look sharp on the network monitoring dashboard. Turns out that setup caused three major flaps in two weeks because a single packet loss triggered a session reset every time. What tipped me off was our NOC calling me at 3 AM saying half our routes vanished for 30 seconds. Went back and looked at the logs and saw the constant reloading was actually making the edge routers unstable. Now I run 10 second keepalives with 30 second hold timers and everything is rock solid. Anyone else learn this lesson the hard way with aggressive BGP tuning?

2 comments

2 Comments

the_spencer27d ago

Did you check if the CPU on the edge routers was even able to handle that many BGP updates per second?

kai80727d ago

Honestly I gotta push back a bit here @the_spencer. Most modern edge routers are built with WAY more headroom than people give them credit for, especially on the control plane side. You're looking at hardware that's designed to handle full internet tables plus some, so a spike in BGP updates really isn't the bottleneck everyone assumes it is. The real issue is almost always the software side like the RIB processing and how the FIB is being updated. Plus a lot of these routers have dedicated processors for BGP now so the main CPU barely breaks a sweat. I've seen setups where the router's CPU is only at like 15% while handling thousands of updates per second.