Genie's Tech Blog

Where knowledge has no dimensions

MPLS TE with Fast-Reroute (FRR) - Link Protection

Hey Guys,

Today I have a very interesting topic to discuss. We are going to discuss about MPLS Traffic-engineering (TE) with Fast-Reroute (FRR). Actually, I came across some customer issue and I had to dig deeper into the platform level of 7600 to find out the root cause. But, then lately I realized that I should share the concept with everyone. So here it is.. 

MPLS Traffic-Engineering

MPLS TE combines the ATM's Traffic engineering capabilities with IP's flexibility and Class-of-Service differentiation. MPLS TE tunnels are unidirectional tunnels which are build on the LSP's across your network on which you forward the traffic. MPLS TE Tunnels let the headend of the TE tunnel control the path its traffic takes to a particular destination. It is not required to build adjacencies over the TE LSP's. MPLS TE Tunnels use a mechanism called autoroute to build the routing table using MPLS TE LSP's without forming a full mesh of routing neighbors.

Applications of MPLS TE

1. Optimizing your network utilization

2. Handling unexpected congestion

3. Handling link and node failures

In this post, I will be discussing more on the third use of MPLS TE, i.e. Handling link and node failures. One of major uses of MPLS TE is the quick recovery from the link and node failures. MPLS TE has a component called Fast Reroute (FRR) that allows you to drastically minimize packet loss when a link or node fails in your network.

Lets discuss a very generic problem which will help understand in a simple way the "need for FRR". Now, Consider an IGP network using a link-state protocol. SPF has to run once when the link goes down and again when the link comes back up. Now this situation is worsened with having MPLS TE on the place. If a link that is part of the LSP fails, the LSP is torn down. After the headend recomputes the new path. SPF has to be run again for the prefixes routed over the tunnel (situation where autoroute is in place), thus making convergence time even worse than in a pure IP network. Automatic Protection Switching (APS) is one solution to recover quickly from link failures but comes with an additional cost. 

Cisco came up with mechanisms to deal with the problem of how to make the MPLS TE tunnel resistant to failures and those mechanisms were collectively called Fast Reroute.

Protection - in the context of fast restoration, is having procedures in place that, when applied to selected resources, ensuring minimal traffic loss. Protection can be classified into two categories: 

1. Path Protection (end-to-end protection)

2. Local Protection - Link protection, Node protection

In this post, I will be specifically discussing about Link protection. The other topics shall be covered in separate posts.

If you consider high bandwidth links carrying critical traffic, with TE deployment, the LSP's over those links also become critical. These LSP's might be carrying critical information or time-sensitive data that requires real-time response. It would be highly preferred to protect these LSP's. FRR allows you to protect your TE tunnels which are carrying sensitive data and also which are not. A pre-signaled backup tunnel is created to bypass the protected links. Link protection means that, when a link goes down, the LSP's that would have gone over that link are sent across some other path.

Configuration

You need to put the configuration related to Link protection at two placed:

1. At the headend of the tunnel interface that you want to protect

2. At the PLR (point of local repair)

Lets consider a basic topology:

  

We are trying to protect the link between R1 and R2. Below are the details of the base configuration.

IP Addresses:

R1 ---- R2 : 57.35.54.152/30

R1 (Loopback99) : 57.35.1.166/32

R2 (Loopback99) : 57.35.0.29/32

R3 (Loopback99) : 57.35.2.52/32

R4 (Loopback0) : 1.1.1.1/32

IGP Protocol in above topology: ISIS Level-2 with bfd

Interface between R1 and R2 is TenGigabitEthernet interface. Rest all other interfaces are Gigabit Interfaces.

Enabling FRR protection on the router:

interface Tunnel150
 description Primary Tunnel
 ip unnumbered Loopback99
 no ip redirects
 no ip proxy-arp
 load-interval 30
 mpls ip
 mpls label protocol ldp
 tunnel mode mpls traffic-eng
 tunnel destination 57.35.0.29
 tunnel mpls traffic-eng autoroute announce
 tunnel mpls traffic-eng path-option 1 dynamic
 tunnel mpls traffic-eng record-route
 tunnel mpls traffic-eng fast-reroute  <<<<<<<<<
end

In the above configuration you can see that we have created a primary TE tunnel with Fast-Reroute feature enabled on it. Thus we are trying to protect the LSP created by this Tunnel. Also, please note that the path-option is dynamics and no other constrained is specified, so for path calculation will be based on the IGP. Since R1 and R2 are directly connected, thus, the link between R1 and R2 will be treated as the primary path. Now, lets consider the configuration of the backup tunnel.

R1#show run interface tunnel 250
Building configuration...

Current configuration : 275 bytes
!
interface Tunnel250
 ip unnumbered Loopback99
 no ip redirects
 no ip proxy-arp
 load-interval 30
 tunnel mode mpls traffic-eng
 tunnel destination 57.35.0.29
 tunnel mpls traffic-eng path-option 1 explicit name backup1_Tunnel250
 tunnel mpls traffic-eng record-route
end

R1#show run | s backup1_Tunnel250
...
ip explicit-path name backup1_Tunnel250 enable
 exclude-address 57.35.54.153    <<<<<<<<<<<<<<<<

If you see the above configuration, you can see that the backup tunnel is configured with an explicit path-option. But in the explicit-path we see that the we have an exclude-address statement which is the ip of R2 physically connected interface to R1. So, when this backup tunnel will kick-in it will just not take the path with 57.35.54.153 address, and since the primary tunnel is dynamic, the backup path will be through R3. So the path from R1 to R2 will look like R1---R3---R2. Please note that FRR helps the restoration of the traffic in just 50 ms. Lets now have a look at some of the outputs.

Interface configuration on R1:

interface TenGigabitEthernet3/2.117
 bandwidth 10000000
 encapsulation dot1Q 117
 ip address 57.35.54.154 255.255.255.252
 no ip redirects
 no ip unreachables
 no ip proxy-arp
 ip router isis 
 ip pim sparse-mode
 mpls ip
 mpls label protocol ldp
 mpls traffic-eng tunnels
 mpls traffic-eng backup-path Tunnel250
 bfd interval 100 min_rx 100 multiplier 3
 no cdp enable
 isis network point-to-point 
 isis metric 5000
 isis csnp-interval 10
 ip rsvp bandwidth 10000000
 ip rsvp signalling hello bfd
end       

Similar configuration will be present on R2. 

R1#sh ip route 1.1.1.1
Routing entry for 1.1.1.1/32
  Known via "isis", distance 115, metric 10000, type level-2
  Redistributing via isis
  Last update from 57.35.0.29 on Tunnel150, 00:04:40 ago
  Routing Descriptor Blocks:
  * 57.35.0.29, from 1.1.1.1, 00:04:40 ago, via Tunnel150
      Route metric is 10000, traffic share count is 1
R1#

Based on the above output, the route to 1.1.1.1 is learned via TE tunnel 150.

R1#sh mpls for 1.1.1.1
Local      Outgoing   Prefix           Bytes Label   Outgoing   Next Hop    
Label      Label      or Tunnel Id     Switched      interface              
5025  [T]  2521       1.1.1.1/32       0             Tu150      point2point <<<<<<<<<<

[T]     Forwarding through a LSP tunnel.
        View additional labelling info with the 'detail' option

R1#show mpls for lab 5025 detail 
Local      Outgoing   Prefix           Bytes Label   Outgoing   Next Hop    
Label      Label      or Tunnel Id     Switched      interface              
5025       2521       1.1.1.1/32       0             Tu150      point2point 
        MAC/Encaps=18/22, MRU=4470, Label Stack{2521}, via Te4/3.117  <<<<<<<
        6400F174B0C000135F216C80810000758847 009D9000
        No output feature configured

Please note that since this is a 7600 platform, so the SP (Switch Processor) also need have the same information as the RP (Route Processor).
Please be aware that all the forwarding decisions on the 7600 platform will be made on the detail that we have on the SP rather than on the RP.
If there are any DFC based linecards in the chassis, they will get programmed based on the information that is available on the SP not the RP.

Output from SP
		
R1#remote command switch show mpls for lab 5025 detail 

Local      Outgoing   Prefix           Bytes Label   Outgoing   Next Hop    
Label      Label      or Tunnel Id     Switched      interface              
5025       2521       1.1.1.1/32       0             Tu150      point2point 
        MAC/Encaps=14/18, MRU=4470, Label Stack{2521}, via Te4/3.117   <<<<<<<
        6400F174B0C000135F216C808847 009D9000
        No output feature configured
R1#

Output from the linecard:

R1#remote comm mod 4 show mpls for lab 5025 det

Local      Outgoing   Prefix           Bytes Label   Outgoing   Next Hop    
Label      Label      or Tunnel Id     Switched      interface              
5025       2521       1.1.1.1/32       0             Tu150      point2point 
        MAC/Encaps=14/18, MRU=4470, Label Stack{2521}, via Te4/3.117
        6400F174B0C000135F216C808847 009D9000
        No output feature configured
R1#

We now shutdown the link between R1 and R2.

R2(config)#interface te 3/2.117
R2(config-subif)#shut

R2(config-subif)#
*Jul  4 21:55:43.110: %PIM-5-NBRCHG: neighbor 57.35.54.154 DOWN on interface Te3/2.117 DR
*Jul  4 21:55:43.134: %MPLS_TE-5-LSP: LSP 57.35.0.29 150_7456: Path Error from 57.35.0.29: Notify: Tunnel locally repaired (flags 0)
*Jul  4 21:55:46.150: %MPLS_TE-5-TUN: Tun150: LSP path change 150_7749 for 150_7456, re-route path error
R2(config-subif)#

Once we have shut down the primary link between R1 and R2, we can see the following outputs on R1 (RP, SP and LC)

R1#show mpls for lab 5025 detail                       
Local      Outgoing   Prefix           Bytes Label   Outgoing   Next Hop    
Label      Label      or Tunnel Id     Switched      interface              
5025       2521       1.1.1.1/32       0             Tu150      point2point 
        MAC/Encaps=18/26, MRU=4466, Label Stack{5030 2521}, via Gi3/1.116
        00050077B01B00135F216C80810000748847 013A6000009D9000
        No output feature configured
R1#remote command switch show mpls for lab 5025 detail 

Local      Outgoing   Prefix           Bytes Label   Outgoing   Next Hop    
Label      Label      or Tunnel Id     Switched      interface              
5025       2521       1.1.1.1/32       0             Tu150      point2point 
        MAC/Encaps=14/22, MRU=4466, Label Stack{5030 2521}, via Gi3/1.116
        00050077B01B00135F216C808847 013A6000009D9000
        No output feature configured
R1#remote comm mod 4 show mpls for lab 5025 det        

Local      Outgoing   Prefix           Bytes Label   Outgoing   Next Hop    
Label      Label      or Tunnel Id     Switched      interface              
5025       2521       1.1.1.1/32       0             Tu150      point2point 
        MAC/Encaps=14/22, MRU=4466, Label Stack{5030 2521}, via Gi3/1.116
        00050077B01B00135F216C808847 013A6000009D9000
        No output feature configured
R1#remote comm mod 3 show mpls for lab 5025 det

Local      Outgoing   Prefix           Bytes Label   Outgoing   Next Hop    
Label      Label      or Tunnel Id     Switched      interface              
5025       2521       1.1.1.1/32       0             Tu150      point2point 
        MAC/Encaps=14/22, MRU=4466, Label Stack{5030 2521}, via Gi3/1.116
        00050077B01B00135F216C808847 013A6000009D9000
        No output feature configured
R1#

So, now if you see the above outputs, the outgoing interface is still Tunnel 150, but the exit interface has changed.
This means that FRR has kicked in and this happened in less than 50 ms. 

R2(config)#interface te 3/2.117
R2(config-subif)#no shut
R2(config-subif)#
*Jul  4 21:59:10.726: %PIM-5-DRCHG: DR change from neighbor 0.0.0.0 to 57.35.54.153 on interface Te3/2.117
*Jul  4 21:59:12.026: %PIM-5-NBRCHG: neighbor 57.35.54.154 UP on interface Te3/2.117 
*Jul  4 21:59:12.026: %PIM-5-DRCHG: DR change from neighbor 57.35.54.153 to 57.35.54.154 on interface Te3/2.117
*Jul  4 21:59:13.242: %CLNS-5-ADJCHANGE: ISIS: Adjacency to R1 (TenGigabitEthernet3/2.117) Up, new adjacency
R2(config-subif)#
R2(config-subif)#

R1#sh mpls traffic-eng fast-reroute database           
P2P Headend FRR information:
Protected tunnel               In-label Out intf/label   FRR intf/label   Status
---------------------------    -------- --------------   --------------   ------
Tunnel140                      Tun hd   Gi3/1.116:implic Tu240:implicit-n Ready 
Tunnel150                      Tun hd   Gi3/1.116:5030   Tu240:5030       Ready 

P2P LSP midpoint frr information:
LSP identifier                 In-label Out intf/label   FRR intf/label   Status
---------------------------    -------- --------------   --------------   ------

P2MP Sub-LSP FRR information:
*Sub-LSP identifier
src_lspid[subid]->dst_tunid    In-label Out intf/label   FRR intf/label   Status
---------------------------    -------- --------------   --------------   ------

* Sub-LSP identifier format: <TunSrc>_<LSP_ID>[SubgroupID]-><TunDst>_<Tun_ID>
  Note: Sub-LSP identifier may be truncated.
  Use 'detail' display for the complete key.
R1#
R1#


R1#sh mpls traffic-eng fast-reroute database 
P2P Headend FRR information:
Protected tunnel               In-label Out intf/label   FRR intf/label   Status
---------------------------    -------- --------------   --------------   ------
Tunnel140                      Tun hd   Gi3/1.116:implic Tu240:implicit-n Ready 
Tunnel150                      Tun hd   Te4/3.117:implic Tu250:implicit-n Ready 

P2P LSP midpoint frr information:
LSP identifier                 In-label Out intf/label   FRR intf/label   Status
---------------------------    -------- --------------   --------------   ------

P2MP Sub-LSP FRR information:
*Sub-LSP identifier
src_lspid[subid]->dst_tunid    In-label Out intf/label   FRR intf/label   Status
---------------------------    -------- --------------   --------------   ------

* Sub-LSP identifier format: <TunSrc>_<LSP_ID>[SubgroupID]-><TunDst>_<Tun_ID>
  Note: Sub-LSP identifier may be truncated.
  Use 'detail' display for the complete key.

The difference in the above two outputs is because the path has still not reoptimized. Here we have set the reoptimization timer to 300 sec. You can also set the reoptimization timer based on the link-up event.
mpls traffic-eng reoptimize timers frequency 300
  
R1#sh ip cef 1.1.1.1 inter                   
1.1.1.1/32, epoch 5, flags need ps clean, RIB[I], refcount 6, per-destination sharing
  sources: RIB, LTE 
  feature space:
   IPRM: 0x00028000
   Broker: linked, distributed at 4th priority
   LFD: 1.1.1.1/32 1 local label
   local label info: global/5025
        contains path extension list
        disposition chain 0x1D638F48
        label switch chain 0x1D638F48
  ifnums:
   Tunnel150(56)
  path 1D3E1C78, path list 1D5AEFC8, share 1/1, type attached nexthop, for IPv4
    MPLS short path extensions: MOI flags = 0x0 label 2521
  nexthop 57.35.0.29 Tunnel150 label 2521, adjacency IP midchain out of Tunnel150 1DAFDEE0
  output chain: label 2521 TAG midchain out of Tunnel150 1DAFEA40 label implicit-null FRR 1 TAG adj out of TenGigabitEthernet4/3.117, addr 57.35.54.153 1DAFED80
R1#

Hope the above outputs have helped understand how FRR works. 

If you still have any queries, please reply me on this blog post. I shall be happy to clarify those doubts. If you want some other example, please let me know as well.

Till then.. 

Happy Reading.. :)

Comments (10) -

  • Kamran

    7/6/2012 5:36:52 PM |

    Hi Genie,

    this is interesting stuff and the language used here is very simple. The labs mentioned with the setup details are very helpful in doing the hands on and the concept clear.

    Keep posting useful stuff like this.. Would appreciate if you could share something on EoMPLS/VPLS technologies too... Smile

    Best Regards,
    Kamran



  • Nik

    7/31/2012 9:17:56 AM |

    Nice post. Can I post a question.
    1. I see that "tunnel mpls traffic-eng fast-reroute" is enabled on the Primary Tunnel 150. Do we need this command for those Tunnel which we want to protect? Or this should be enabled on all the Tunnel interfaces on head end router?

    Apologies in advance... if its too silly. Have been banging my head on FRR link protection for few days. But finally got to understand the concept through your blog Smile

  • Nik

    7/31/2012 1:48:22 PM |

    2. If we want to have FRR at mid-points.  So I just need to have this command on mid-point router "mpls traffic-eng backup-path Tunnel250" and rest of things will be taken care by TE? Or is there anything else I should be doing if I want to protect the TE via Mid-Points interfaces?

    Thanks,

  • Genie

    7/31/2012 2:13:51 PM |

    Hello Nik
    Thanks for posting questions. Please don't say sorry for posting question. Please feel free them to post as many as you have. Now coming to your first question.
    - The command "tunnel mpls traffic-eng fast-reroute" command will be there on the primary Tunnel. By enabling this command we tell the primary tunnel that there is already a backup tunnel in case there is any failure on the Tunnel LSP. There may be different causes to it. This post is basically about Link protection. That means, within the path of the TE tunnel, there will be a link which we are trying to protect as the Tunnel is supposed to carry critical data traffic. Now, we protect a link which in case of any failure, moves to the protected path. The FRR concept works on the funda of "Make Before Break". So there is a backup LSP already prepared vie the backup tunnel path so that in case the primary link fails, the Tunnel wont go down or impact the traffic, rather it will shift to the backup LSP and reoptimize. So based on your question, this command should be enabled on the Tunnel which we want to protect.

    Regarding your second question,
    - The command that you mention is put on the link which we want to protect. For more detailed explanation on MID-POINTs TE FRR, request you to wait for my next post. That shall cover all that section.

    Hope your questions have been answered.
    Please let me know if you have any other questions.
    Cheers...!!!

    Genie

  • Nik

    7/31/2012 3:44:52 PM |

    Thanks Genie for answering questions. I am looking forward to MID-POINTS FRR protection.

  • Nik

    8/2/2012 10:48:35 AM |

    Morning Genie, Can you please shed some light on "mpls traffic-eng backup-path Tunnel X" If my understanding is correct then this command would be applied on the interface for which we want to create backup of Primary tunnel LSPs?

    Appreciate you efforts...!!!

    Thanks,

  • Genie

    8/7/2012 1:49:43 PM |

    Hi Nik
    Thanks for asking the question. As mentioned previously, this command is added on the link which is being protected. The backup Tunnel is created on the same node on which the protected link is.
    There are multiple kind of protection that we can created. Path protection and Local protection. Local protection has two categories 1) Link Protection 2) Node protection. So over here this command is being used for Local protection but Path Protection. Protecting LSPs is not discussed yet in this section. I will try to cover them later.

  • Sabriye

    8/11/2012 9:22:42 AM |

    Hi Genie,
    Can we have a Head End node to be also a PLR node and  and Tail End node as MP node ...Like in your topology can we say R1(Head end and PLR) and R2(Tail end and MP).If so,will the configuration differ much
    Thanks

  • Genie

    8/13/2012 6:24:07 PM |

    Hello Sabriye
    Thanks for your question. yes, the same node can be a head-end and a PLR and the same Tail-end can be MP. If you see the configuration, you will notice that the headend and tailend routers are both the directly connected nodes and we are here trying to protect the link between these two nodes. So, if the link fails, the purpose is to already have a path over which the traffic could shift.

    For further more understanding, you can refer to my another post on MPLS TE

    blog.codergenie.com/.../...ection)-Next-level.aspx

    Hope you got the answer to your question.
    Cheers...
    Genie

  • Sabriye

    8/13/2012 7:22:53 PM |

    Thanks Genie Smile

Comments are closed