Friday 3 July 2015

How NOT to back-out from BGP redistribution into OSPF

Recently I learned the hard way how not to back-out from a DUAL ISP optimization project that included BGP redistribution into OSPF with prefix-list filtering.

At one of the large regional offices there are two internet routers (router-A and router-B) that peer with the two distinct ISPs (ISP-A and ISP-B) and learn the entire BGP table from both. Currently we prefer one of these carrier circuits (ISP-B) for outbound traffic which causes the other to ultimately sit idle. This is not efficient use of resources.

Goal: allow the internal network to have 2 equal cost paths to the internet via router-A and router-B.

This can be accomplished by redistributing a default route to the firewall sitting at the edge of the network. The internal network is running OSPF.

I could have accomplished the end goal by other means but there were already settings in place that dictated this direction. So I went with it.

1. create prefix-list that will match exactly what I want to allow
2. create the route-map that will match the prefix-list
3. redistribute BGP with the safety net of the route-map that matches the prefix-list

router-A(config)#ip prefix-list JUST-DEFAULT-PREFIX permit 0.0.0.0/0
!
router-A(config)#route-map JUST-DEFAULT permit 10
router-A(config-route-map)#match ip address prefix-list JUST-DEFAULT-PREFIX
router-A(config-route-map)#exi
!
!
router-A(config)#
router-A(config)#router ospf 1
router-A(config-router)#router-id 1.1.1.1
router-A(config-router)#redistribute bgp 1234 route-map JUST-DEFAULT subnets
!
router-A(config-router)#network 1.1.1.1 0.0.0.0 area 1
router-A(config-router)#end
router-A#
*May 20 09:27:02.066: %SYS-5-CONFIG_I: Configured from console by console
router-A#sho run | sec ospf
router ospf 1
 router-id 1.1.1.1
 redistribute bgp 1234 subnets route-map JUST-DEFAULT
 network 1.1.1.1 0.0.0.0 area 1
!

- At this stage I have achieved the outcome of redistributing only the default route into OSPF area 1. I followed up with similar configs for router-B (left out for brevity).

All is well........ not really. I have only learnt one default route because router-B is setting its local preference to 200 which makes it the only default route in the area. I need to go back and fix that so I want to be safe and back-out my configurations just in case I mess things up.

So here goes!


router-A#conf t
Enter configuration commands, one per line.  End with CNTL/Z.
router-A(config)#router ospf 1
router-A(config-router)#no redistribute bgp 1234 subnets route-map JUST-DEFAULT      
router-A(config-router)#end
router-A#sho run
*May 20 09:27:27.880: %SYS-5-CONFIG_I: Configured from console by console
router-A#sho run | sec ospf
router ospf 1
 router-id 1.1.1.1
 redistribute bgp 1234 subnets
 network 1.1.1.1 0.0.0.0 area 1
router-A#


Look closely at the running configuration relevant to OSPF. There is still a redistribute command, but it does not have the safety net of a route-map, prefix-list, or even an umbrella for the rain.. And it was raining BGP prefixes...

The entire network came crashing down. Every L3 device participating in OSPF got it's memory and CPU bludgeoned by a massive influx of the entire internet routing table. I've just brought down the entire global network! It was like a tsunami!

I've just redistributed the entire BGP table into OSPF.


Lessons learnt:

Do not redistribute BGP into OSPF if not entirely necessary. And if you have to, you can use other methods like a distribute-list.

If you want to backout from an BGP to OSPF redistribution, you only need to specify the 'no redistribute bgp AS'. This is so because the CLI would think that you mean to remove the supplementary config as opposed to anything else if you do anything else... To my outrage!


-Jason