Why does nobody talk about failover circuits going down silently

Had a primary MPLS link drop at 3 AM last Thursday and our backup LTE didn't kick in because the Cradlepoint had a firmware bug that nobody caught in 8 months. The failover tested fine with pings but the actual VPN tunnel never came up until I drove to the site and rebooted it manually. How does everyone else verify their backup circuits are actually working?

2 comments

2 Comments

wendy_garcia1624d ago

Man that is the worst feeling when you think you have backup and it just silently fails.

brookekelly23d ago

Heard a tech guy on a podcast say silent failures are the scariest kinds of bugs.