Genie's Tech Blog

Where knowledge has no dimensions

MPLS TE with FRR (Link Protection) - Next level

Hello Friends...

I am back with further more discussion in continuation to my previous post on MPLS TE with FRR. Today I will be discussing a different example and will be showing different outputs that will help understand the MPLS TE and FRR concepts.

Lets consider a topology as stated below in the diagram:

 

We have 6 routers in whole which are using 12.2(33)SRE4 IOS code. Each of the loopback address has the loopback address in the format x.x.x.x/32 where x is the router number. The link ip addresses is defined between any two router x and y as xy.xy.xy.0/24 address. I have defined TE tunnels on R1 destined towards loopback of R3, R5 and R6. and on R3 towards R5 with dynamic option. Lets have a look at the config below from both these routers.

On R1:
=====

interface Tunnel100
 ip unnumbered Loopback0
 mpls ip
 tunnel destination 3.3.3.3
 tunnel mode mpls traffic-eng
 tunnel mpls traffic-eng autoroute announce
 tunnel mpls traffic-eng path-option 1 dynamic
 tunnel mpls traffic-eng fast-reroute
!

interface Tunnel200
 ip unnumbered Loopback0
 mpls ip
 tunnel destination 5.5.5.5
 tunnel mode mpls traffic-eng
 tunnel mpls traffic-eng autoroute announce
 tunnel mpls traffic-eng path-option 1 dynamic
 tunnel mpls traffic-eng fast-reroute
!

interface Tunnel201
 ip unnumbered Loopback0
 mpls ip
 tunnel destination 6.6.6.6
 tunnel mode mpls traffic-eng
 tunnel mpls traffic-eng autoroute announce
 tunnel mpls traffic-eng path-option 1 dynamic
 tunnel mpls traffic-eng fast-reroute
!

On R3:
=====

interface Tunnel200
 ip unnumbered Loopback0
 mpls ip
 tunnel mode mpls traffic-eng
 tunnel destination 5.5.5.5
 tunnel mpls traffic-eng autoroute announce
 tunnel mpls traffic-eng path-option 1 dynamic
 tunnel mpls traffic-eng fast-reroute
!

I have also defined the backup tunnel for Fast-Reroute on both R1 and R3. Since I have defined dynamic TE tunnels, the path from R1 to R3 will be the directly connected interface between R1 and R3 or from R1 to R5 shall take the path via R3 i.e. R1 -- R3 -- R5. Thus we have made the directly connected link as the protected link. Lets have a look at the configuration below.

interface Ethernet1/0
 description "Connected to R3"
 bandwidth 100000
 ip address 13.13.13.1 255.255.255.0
 ip router isis 
 mpls traffic-eng tunnels
 mpls traffic-eng backup-path Tunnel150
 mpls ip
 isis metric 5000
 ip rsvp bandwidth 10000

!

interface Tunnel150
 ip unnumbered Loopback0
 tunnel destination 3.3.3.3
 tunnel mode mpls traffic-eng
 tunnel mpls traffic-eng path-option 1 explicit name xyz
!

Thus the interface Ethernet 1/0 on R1 will be acting as the protected link as well as the primary link for the primary Tunnel - Tunnel 100, Tunnel 200 and Tunnel 201. Tunnel 150 is defined as a backup tunnel with the explicit path "xyz" which specifies to exclude the primary link address. Lets have a look at that config as well.

ip explicit-path name xyz enable
 exclude-address 13.13.13.3
!

Similar configuration has been done on R3 as well for the tunnel destined to R5. We have also configured the reoptimization event trigger to the link-up event.

mpls traffic-eng reoptimize events link-up
!

This command triggers the head-end reoptimization when the link comes back up.

One important terminology that we need to understand here is the PLR - Point of Local Repair. PLR is the router which is the head-end of the backup Tunnel. Another term that we need to understand is the Merge Point which is the Tail-end of the backup Tunnel. At this point, the Backup Tunnel merges with the primary tunnel to restore the broken LSP.

Now, consider the topology above. If the link between R3 and R5 fails, R1 will receive a Path Error from R2 and a Resv Tear message. R1 will receive a new LSA / LSP indicating that the R3-R5 link is down and will conclude the LSP failed (considering the scenario where R1 is in the same area as the failed network element). An optimization of the Path Error allows to remove failed link from the TE database to prevent to retry the same failed link (if the ISIS LSP or the OSPF LSA has not been received yet).

A key principle of Local repair is to guaranty a very fast traffic recovery with or without QoS guaranty during a transient phase while the other mechanisms (reoptimization) are used over a longer time scale. The local repair is always controlled by the PLR. When we configure the TE tunnel for local repair, "Local Protection Desired" bit of the SESSION_ATTRIBUTE object flag is set and the rerouteable LSPs will be backed up.

In case of Link protection, the path message for the old Path are still forwarded onto the Backup LSP. Actually the RSVP code is modified so that PLR router could receive a Resv message from a different interface than the one used to send the Path message and the Merge point router could receive a Path message from a different interface other than the primary. The PLR should send a PathErr message with error code "Notify" (Error code = 25) and an error value field of ss00 cccc cccc cccc where ss = 0 and the sub-code = 3 which implies that "Tunnel Locally repaired". This message will trigger the head-end reoptimization. Then the TE LSP will be rerouted over an alternate Path using the FRR (Make Before Break).

Now, lets have a better understanding of the above stated through some practical example.

Since all the configuration has been discussed earlier in this post, we shall now look at few command outputs. Lets have a look at the Tunnel 200 on R1 which is destined towards R5 loopback 5.5.5.5/32.

R1#sh ip route 5.5.5.5
Routing entry for 5.5.5.5/32
  Known via "isis", distance 115, metric 10000, type level-2
  Redistributing via isis
  Last update from 5.5.5.5 on Tunnel200, 4d23h ago
  Routing Descriptor Blocks:
  * 5.5.5.5, from 5.5.5.5, 4d23h ago, via Tunnel200
      Route metric is 10000, traffic share count is 1
R1#


R1#sh mpls for 5.5.5.5 det
Local  Outgoing      Prefix            Bytes Label   Outgoing   Next Hop    
Label  Label or VC   or Tunnel Id      Switched      interface              
23     Pop Label     5.5.5.5/32        0             Tu200      point2point 
        MAC/Encaps=14/18, MRU=1500, Label Stack{27}, via Et1/0
        AABBCC016101AABBCC015F018847 0001B000
        No output feature configured


R1#sh mpls traffic-eng tunnels tunnel 200

Name: R1_t200                             (Tunnel200) Destination: 5.5.5.5
  Status:
    Admin: up         Oper: up     Path: valid       Signalling: connected
    path option 1, type dynamic (Basis for Setup, path weight 10000)

  Config Parameters:
    Bandwidth: 0        kbps (Global)  Priority: 7  7   Affinity: 0x0/0xFFFF
    Metric Type: TE (default)
    AutoRoute:  enabled   LockDown: disabled  Loadshare: 0        bw-based
    auto-bw: disabled
  Active Path Option Parameters:
    State: dynamic path option 1 is active
    BandwidthOverride: disabled  LockDown: disabled  Verbatim: disabled


  InLabel  :  - 
  OutLabel : Ethernet1/0, 27   <<<<<<<<<<<<<<
  FRR OutLabel : Tunnel150, 27    <<<<<<<<<<<
  RSVP Signalling Info:
       Src 1.1.1.1, Dst 5.5.5.5, Tun_Id 200, Tun_Instance 408
    RSVP Path Info:
      My Address: 13.13.13.1   
      Explicit Route: 13.13.13.3 35.35.35.3 35.35.35.5 5.5.5.5 
      Record   Route:   NONE
      Tspec: ave rate=0 kbits, burst=1000 bytes, peak rate=0 kbits
    RSVP Resv Info:
      Record   Route:  3.3.3.3(27) 5.5.5.5(0)
      Fspec: ave rate=0 kbits, burst=1000 bytes, peak rate=0 kbits
  Shortest Unconstrained Path Info:
    Path Weight: 10000 (TE)
    Explicit Route: 13.13.13.1 13.13.13.3 35.35.35.3 35.35.35.5 
                    5.5.5.5 
  History:
    Tunnel:
      Time since created: 20 days, 14 hours, 17 minutes
      Time since path change: 4 days, 23 hours, 13 minutes
      Number of LSP IDs (Tun_Instances) used: 408
    Current LSP:
      Uptime: 4 days, 23 hours, 13 minutes
      Selection: reoptimization
    Prior LSP:
      ID: path option 1 [405]
      Removal Trigger: reoptimization completed
R1#


R1#sh mpls tra fast-reroute database detail 
FRR Database Summary:
  Number of protected interfaces: 1
  Number of protected tunnels: 3
  Number of backup tunnels: 1
  Number of active interfaces: 0
Protected tunnel Tunnel100, ready
  Input label Tun hd, Output label Et1/0:implicit-null, FRR label Tu150:implicit-null
  Role Head Head Hop 1.1.1.1 Tail Hop 3.3.3.3
Protected tunnel Tunnel200, ready   <<<<<<<<<<<<
  Input label Tun hd, Output label Et1/0:27, FRR label Tu150:27
  Role Head Head Hop 1.1.1.1 Tail Hop 5.5.5.5
Protected tunnel Tunnel201, ready
  Input label Tun hd, Output label Et1/0:29, FRR label Tu150:29
  Role Head Head Hop 1.1.1.1 Tail Hop 6.6.6.6
R1#


R1#sh mpls traffic-eng fast-reroute database 
Headend frr information:
Protected tunnel              In-label Out intf/label   FRR intf/label   Status
Tunnel100                     Tun hd   Et1/0:implicit-n Tu150:implicit-n ready  
Tunnel200                     Tun hd   Et1/0:27         Tu150:27         ready  
Tunnel201                     Tun hd   Et1/0:29         Tu150:29         ready  

LSP midpoint frr information:
LSP identifier                In-label Out intf/label   FRR intf/label   Status
R1#

If you see the above outputs, we notice that the R1 can reach R5 loopback via Tunnel 200. The local label assigned is 23. and R5 has sent a Pop Label. Thus any traffic that hits R1 in order to reach R5 will have the label value of 23. This is the simple mpls fundamentals 1-on-1. If we see the output of "show mpls traffic-eng tunnel Tunnel 200", we notice that the path taken is as follows in the output:

Explicit Route: 13.13.13.3 35.35.35.3 35.35.35.5 5.5.5.5 

We also have the OutLabel as 27 and the FRR OutLabel as 27 as well. If you notice these labels these are not MPLS labels but are RSVP generated labels. The MPLS label for reaching 5.5.5.5 is 23 where as the RSVP label value is 27. You can cross verify the same on R3 by checking the show mpls traffic-eng tunnels output:

R3# show mpls traffic-eng tunnel
. . . 
LSP Tunnel R1_t200 is signalled, connection is up
  InLabel  : Ethernet1/0, 27   <<<<<<<<<<<<<
  Prev Hop : 13.13.13.1
  OutLabel : Ethernet0/2, implicit-null  <<<<<<<<<<
  Next Hop : 35.35.35.5
  FRR OutLabel : Tunnel150, explicit-null 
  RSVP Signalling Info:
       Src 1.1.1.1, Dst 5.5.5.5, Tun_Id 200, Tun_Instance 408
    RSVP Path Info:
      My Address: 35.35.35.3   
      Explicit Route: 35.35.35.5 5.5.5.5 
      Record   Route:   NONE
      Tspec: ave rate=0 kbits, burst=1000 bytes, peak rate=0 kbits
    RSVP Resv Info:
      Record   Route:  5.5.5.5(0)
      Fspec: ave rate=0 kbits, burst=1000 bytes, peak rate=0 kbits

If you see the above highlighted output, the InLabel value is 27. R3 generated this label and sent it to R1. Also, as i mentioned previously about the Local Protection Desired bit of the  SESSION_ATTRIBUTE object, we can also verify the same in the "show ip rsvp fast-reroute detail" output

R1# show ip rsvp fast-reroute detail
. . .
. . .
PATH:
  Tun Dest:   5.5.5.5  Tun ID: 200  Ext Tun ID: 1.1.1.1
  Tun Sender: 1.1.1.1  LSP ID: 408
  Path refreshes:
    sent:     to   NHOP 13.13.13.3 on Ethernet1/0
  Session Attr: <<<<<<<<<
    Setup Prio: 7, Holding Prio: 7
    Flags: (0x7) Local Prot desired, Label Recording, SE Style   <<<<<<<<<
    Session Name: R1_t200 
  ERO: (incoming)
    1.1.1.1 (Strict IPv4 Prefix, 8 bytes, /32)
    13.13.13.1 (Strict IPv4 Prefix, 8 bytes, /32)
    13.13.13.3 (Strict IPv4 Prefix, 8 bytes, /32)
    35.35.35.3 (Strict IPv4 Prefix, 8 bytes, /32)
    35.35.35.5 (Strict IPv4 Prefix, 8 bytes, /32)
    5.5.5.5 (Strict IPv4 Prefix, 8 bytes, /32)
  ERO: (outgoing)
    13.13.13.3 (Strict IPv4 Prefix, 8 bytes, /32)
    35.35.35.3 (Strict IPv4 Prefix, 8 bytes, /32)
    35.35.35.5 (Strict IPv4 Prefix, 8 bytes, /32)
    5.5.5.5 (Strict IPv4 Prefix, 8 bytes, /32)
  Traffic params - Rate: 0 bits/sec, Max. burst: 1K bytes
    Min Policed Unit: 0 bytes, Max Pkt Size 2147483647 bytes
  Fast-Reroute Backup info:
    Inbound  FRR: Not active
    Outbound FRR: Ready -- backup tunnel selected
      Backup Tunnel: Tu150      (label 27) <<<<<<<<<<<<
      Bkup Sender Template: 
        Tun Sender: 12.12.12.1  LSP ID: 408
      Bkup FilerSpec:       
        Tun Sender: 12.12.12.1, LSP ID: 408
  Path ID handle: 10000417.
  Incoming policy: Accepted. Policy source(s): MPLS/TE
  Status: Proxied
  Output on Ethernet1/0. Policy status: Forwarding. Handle: 0C000418
    Policy source(s): MPLS/TE

Please see the above highlighted text. In the "Session Attr:" section its mentioned as Local Prot desired. Also, we can see that the backup Tunnel is Tunnel 150 having the label value of 27 which is same as the FRR OutLabel. Now every link down event of the protected link is dealt with a counter known as the "Cutover Count". This is from the programming perspective of IOS to deal with Link failures of the protected link and to trigger the event for the FRR to kick in. We can also check the same with the following command - "show mpls infrastructure frr-db", but in order to run this command you need to enable "service internal" from the config mode. This command is not available on all the IOS versions, but is available for sure in SRE4. Lets have a look at the output before the cutover.

R1#sh mpls infrastructure frr-db 
Et1/0 is fast-reroute protected, cutover count is 0
         frroce=0x3A58690, parent=0x4393F88, pri_if=6, state=1, primary=0x4393F88, nolab_backup=0x4393778, label_backup=0x0
                 primary oce: TAG adj out of Ethernet1/0, addr 13.13.13.3 04393F18
                 nolab_backup oce: TAG midchain out of Tunnel150 04393708
         frroce=0x3A58700, parent=0x43940E0, pri_if=6, state=1, primary=0x43940E0, nolab_backup=0x43938D0, label_backup=0x0
                 primary oce: IP adj out of Ethernet1/0, addr 13.13.13.3 04394070
                 nolab_backup oce: IP midchain out of Tunnel150 04393860

In the above output we see that the cutover count is 0. Actually, there is a fast reroute database maintained by the IOS software to keep track of the protected links. I am not going into the much details of the OCE chains as this are mostly programming terminologies. But please remember that the cutover count should always be 0 else the traffic is supposed to forward through the backup path even if the primary tunnel is up. There is already a IOS defect CSCto57899 for the same. Please remember whenever the cutover event happens, the output will not be seen for the protected interface. The output is seen only when the backup LSP is present or in other words both the primary LSP and the backup LSP are present.

Now, lets try to shutdown the link between R1 and R3 and see how it creates a difference.

R1#sh mpls for 5.5.5.5 det
Local  Outgoing      Prefix            Bytes Label   Outgoing   Next Hop    
Label  Label or VC   or Tunnel Id      Switched      interface              
23     Pop Label     5.5.5.5/32        0             Tu200      point2point 
        MAC/Encaps=14/18, MRU=1500, Label Stack{29}, via Et0/0
        AABBCC016000AABBCC015F008847 0001D000
        No output feature configured
R1#

R1#sh mpls tra tu tu200

Name: R1_t200                             (Tunnel200) Destination: 5.5.5.5
  Status:
    Admin: up         Oper: up     Path: valid       Signalling: connected
    path option 1, type dynamic (Basis for Setup, path weight 15000)

  Config Parameters:
    Bandwidth: 0        kbps (Global)  Priority: 7  7   Affinity: 0x0/0xFFFF
    Metric Type: TE (default)
    AutoRoute:  enabled   LockDown: disabled  Loadshare: 0        bw-based
    auto-bw: disabled
  Active Path Option Parameters:
    State: dynamic path option 1 is active
    BandwidthOverride: disabled  LockDown: disabled  Verbatim: disabled


  InLabel  :  - 
  OutLabel : Ethernet0/0, 29  <<<<<<<<<
  RSVP Signalling Info:
       Src 1.1.1.1, Dst 5.5.5.5, Tun_Id 200, Tun_Instance 529
    RSVP Path Info:
      My Address: 12.12.12.1   
      Explicit Route: 12.12.12.2 23.23.23.2 23.23.23.3 35.35.35.3 
                      35.35.35.5 5.5.5.5 
      Record   Route:   NONE
      Tspec: ave rate=0 kbits, burst=1000 bytes, peak rate=0 kbits
    RSVP Resv Info:
      Record   Route:  2.2.2.2(29) 3.3.3.3(26)
                       5.5.5.5(0)
      Fspec: ave rate=0 kbits, burst=1000 bytes, peak rate=0 kbits
  Shortest Unconstrained Path Info:
    Path Weight: 15000 (TE)
    Explicit Route: 12.12.12.1 12.12.12.2 23.23.23.2 23.23.23.3 
                    35.35.35.3 35.35.35.5 5.5.5.5 
  History:
    Tunnel:
      Time since created: 20 days, 15 hours, 31 minutes
      Time since path change: 24 seconds
      Number of LSP IDs (Tun_Instances) used: 529
    Current LSP:
      Uptime: 27 seconds
      Selection: reoptimization
    Prior LSP:
      ID: path option 1 [408]
      Removal Trigger: re-route path verification failed
R1#

Now after shutting down the link between R1 and R3 we notice that the outgoing interface has changed from E1/0 to E0/0. but the path is still taken by the Tunnel 200. Though in the label stack we see that there is label 29 which is the RSVP label. In the "show mpls traffic-eng tunnel tunnel 200" output, we see that there is no more FRR OutLabel and the OutLabel is 29. Now if we see the output on R2 & R3 we see the following:

R2# show mpls traffic-eng tunnel
. . . 
LSP Tunnel R1_t200 is signalled, connection is up
  InLabel  : Ethernet0/0, 29
  Prev Hop : 12.12.12.1
  OutLabel : Ethernet0/1, 26
  Next Hop : 23.23.23.3
  RSVP Signalling Info:
       Src 1.1.1.1, Dst 5.5.5.5, Tun_Id 200, Tun_Instance 529
    RSVP Path Info:
      My Address: 23.23.23.2   
      Explicit Route: 23.23.23.3 35.35.35.3 35.35.35.5 5.5.5.5 
      Record   Route:   NONE
      Tspec: ave rate=0 kbits, burst=1000 bytes, peak rate=0 kbits
    RSVP Resv Info:
      Record   Route:  3.3.3.3(26) 5.5.5.5(0)
      Fspec: ave rate=0 kbits, burst=1000 bytes, peak rate=0 kbits

R3#  show mpls traffic-eng tunnel
. . . 
LSP Tunnel R1_t200 is signalled, connection is up
  InLabel  : Ethernet0/1, 26
  Prev Hop : 23.23.23.2
  OutLabel : Ethernet0/2, implicit-null
  Next Hop : 35.35.35.5
  FRR OutLabel : Tunnel150, explicit-null 
  RSVP Signalling Info:
       Src 1.1.1.1, Dst 5.5.5.5, Tun_Id 200, Tun_Instance 529
    RSVP Path Info:
      My Address: 35.35.35.3   
      Explicit Route: 35.35.35.5 5.5.5.5 
      Record   Route:   NONE
      Tspec: ave rate=0 kbits, burst=1000 bytes, peak rate=0 kbits
    RSVP Resv Info:
      Record   Route:  5.5.5.5(0)
      Fspec: ave rate=0 kbits, burst=1000 bytes, peak rate=0 kbits

R2 was having the OutLabel of 26 which was given to it by R3 where as the InLabel is 29 which R2 sent it to R1. This is how the LSP is completed again. This was an example of TE Headend FRR. Now lets see an example of TE Mid Point FRR, but before that we need to understand the purpose of MPLS TE midpoints. When the router is configured as a Tunnel midpoint, the enhancements reduce the time required to establish large number of TE Tunnels. There is no difference between the way we configure the midpoint routers than the headend routers. You do need to configure a TE Tunnel destined to tailend router or a transit router which the primary headend tunnel router is taking the path. As soon as you bring the Tunnel up, midpoints are created in the FRR database. Here in this example, we are going to see how midpoint tunnels are constructed and much more. As stated initially the router R3 is configured with a Tunnel 200 destined to R5 and the backup Tunnel 150 which will be used for FRR. Now, when we bring up the Tunnel 200 and Tunnel 150, we shall see what is actually happening on R1 and R3.

On R1:
=====


R1#sh mpls tra tu tu201

Name: R1_t201                             (Tunnel201) Destination: 6.6.6.6
  Status:
    Admin: up         Oper: up     Path: valid       Signalling: connected
    path option 1, type dynamic (Basis for Setup, path weight 15000)

  Config Parameters:
    Bandwidth: 0        kbps (Global)  Priority: 7  7   Affinity: 0x0/0xFFFF
    Metric Type: TE (default)
    AutoRoute:  enabled   LockDown: disabled  Loadshare: 0        bw-based
    auto-bw: disabled
  Active Path Option Parameters:
    State: dynamic path option 1 is active
    BandwidthOverride: disabled  LockDown: disabled  Verbatim: disabled


  InLabel  :  - 
  OutLabel : Ethernet1/0, 26
  FRR OutLabel : Tunnel150, 26 
  RSVP Signalling Info:
       Src 1.1.1.1, Dst 6.6.6.6, Tun_Id 201, Tun_Instance 699
    RSVP Path Info:
      My Address: 13.13.13.1   
      Explicit Route: 13.13.13.3 35.35.35.3 35.35.35.5 56.56.56.5 
                      56.56.56.6 6.6.6.6 
      Record   Route:   NONE
      Tspec: ave rate=0 kbits, burst=1000 bytes, peak rate=0 kbits
    RSVP Resv Info:
      Record   Route:  3.3.3.3(26) 5.5.5.5(34)
                       6.6.6.6(0)
      Fspec: ave rate=0 kbits, burst=1000 bytes, peak rate=0 kbits
  Shortest Unconstrained Path Info:
    Path Weight: 15000 (TE)
    Explicit Route: 13.13.13.1 13.13.13.3 35.35.35.3 35.35.35.5 
                    56.56.56.5 56.56.56.6 6.6.6.6 
  History:
    Tunnel:
      Time since created: 26 days, 9 hours, 19 minutes
      Time since path change: 13 minutes, 46 seconds
      Number of LSP IDs (Tun_Instances) used: 699
    Current LSP:
      Uptime: 10 minutes, 2 seconds
      Selection: reoptimization
    Prior LSP:
      ID: path option 1 [685]
      Removal Trigger: path error


On R3:
=====


R3#sh mpls tra tu

P2P TUNNELS/LSPs:

Name: R3_t150                             (Tunnel150) Destination: 5.5.5.5
  Status:
    Admin: up         Oper: up     Path: valid       Signalling: connected
    path option 1, type explicit sonal (Basis for Setup, path weight 10000)

  Config Parameters:
    Bandwidth: 0        kbps (Global)  Priority: 7  7   Affinity: 0x0/0xFFFF
    Metric Type: TE (default)
    AutoRoute announce: disabled LockDown: disabled Loadshare: 0        bw-based
    auto-bw: disabled
  Active Path Option Parameters:
    State: explicit path option 1 is active
    BandwidthOverride: disabled  LockDown: disabled  Verbatim: disabled


  InLabel  :  -
  OutLabel : Ethernet0/0, 29
  Next Hop : 34.34.34.4
  RSVP Signalling Info:
       Src 3.3.3.3, Dst 5.5.5.5, Tun_Id 150, Tun_Instance 297
    RSVP Path Info:
      My Address: 34.34.34.3   
      Explicit Route: 34.34.34.4 45.45.45.4 45.45.45.5 5.5.5.5 
      Record   Route:   NONE
      Tspec: ave rate=0 kbits, burst=1000 bytes, peak rate=0 kbits
    RSVP Resv Info:
      Record   Route:   NONE
      Fspec: ave rate=0 kbits, burst=1000 bytes, peak rate=0 kbits
  History:
    Tunnel:
      Time since created: 10 days, 21 hours, 17 minutes
      Time since path change: 11 minutes, 30 seconds
      Number of LSP IDs (Tun_Instances) used: 297
    Current LSP: [ID: 297]
      Uptime: 11 minutes, 30 seconds
      Selection: reoptimization
    Prior LSP: [ID: 296]
      ID: path option unknown
      Removal Trigger: configuration changed

Name: R3_t200                             (Tunnel200) Destination: 5.5.5.5
  Status:
    Admin: up         Oper: up     Path: valid       Signalling: connected
    path option 1, type dynamic (Basis for Setup, path weight 5000)

  Config Parameters:
    Bandwidth: 0        kbps (Global)  Priority: 7  7   Affinity: 0x0/0xFFFF
    Metric Type: TE (default)
    AutoRoute announce: enabled  LockDown: disabled Loadshare: 0        bw-based
    auto-bw: disabled
  Active Path Option Parameters:
    State: dynamic path option 1 is active
    BandwidthOverride: disabled  LockDown: disabled  Verbatim: disabled


  InLabel  :  -
  OutLabel : Ethernet0/2, implicit-null
  Next Hop : 35.35.35.5
  FRR OutLabel : Tunnel150, explicit-null 
  RSVP Signalling Info:
       Src 3.3.3.3, Dst 5.5.5.5, Tun_Id 200, Tun_Instance 295
    RSVP Path Info:
      My Address: 35.35.35.3   
      Explicit Route: 35.35.35.5 5.5.5.5 
      Record   Route:   NONE
      Tspec: ave rate=0 kbits, burst=1000 bytes, peak rate=0 kbits
    RSVP Resv Info:
      Record   Route:  5.5.5.5(0)
      Fspec: ave rate=0 kbits, burst=1000 bytes, peak rate=0 kbits
  History:
    Tunnel:
      Time since created: 10 days, 21 hours, 18 minutes
      Time since path change: 11 minutes, 34 seconds
      Number of LSP IDs (Tun_Instances) used: 295
    Current LSP: [ID: 295]
      Uptime: 11 minutes, 34 seconds
      Selection: reoptimization
    Prior LSP: [ID: 294]
      ID: path option unknown
      Removal Trigger: configuration changed

. . . . .	  
. . . . .	  

LSP Tunnel R1_t200 is signalled, connection is up
  InLabel  : Ethernet1/0, 29
  Prev Hop : 13.13.13.1
  OutLabel : Ethernet0/2, implicit-null
  Next Hop : 35.35.35.5
  FRR OutLabel : Tunnel150, explicit-null 
  RSVP Signalling Info:
       Src 1.1.1.1, Dst 5.5.5.5, Tun_Id 200, Tun_Instance 666
    RSVP Path Info:
      My Address: 35.35.35.3   
      Explicit Route: 35.35.35.5 5.5.5.5 
      Record   Route:   NONE
      Tspec: ave rate=0 kbits, burst=1000 bytes, peak rate=0 kbits
    RSVP Resv Info:
      Record   Route:  5.5.5.5(0)
      Fspec: ave rate=0 kbits, burst=1000 bytes, peak rate=0 kbits

LSP Tunnel R1_t201 is signalled, connection is up
  InLabel  : Ethernet1/0, 26
  Prev Hop : 13.13.13.1
  OutLabel : Ethernet0/2, 34
  Next Hop : 35.35.35.5
  FRR OutLabel : Tunnel150, 34 
  RSVP Signalling Info:
       Src 1.1.1.1, Dst 6.6.6.6, Tun_Id 201, Tun_Instance 699
    RSVP Path Info:
      My Address: 35.35.35.3   
      Explicit Route: 35.35.35.5 56.56.56.5 56.56.56.6 6.6.6.6 
      Record   Route:   NONE
      Tspec: ave rate=0 kbits, burst=1000 bytes, peak rate=0 kbits
    RSVP Resv Info:
      Record   Route:  5.5.5.5(34) 6.6.6.6(0)
      Fspec: ave rate=0 kbits, burst=1000 bytes, peak rate=0 kbits

P2MP TUNNELS:

P2MP SUB-LSPS:


P2MP SUB-LSPS:
R3#sh mpls for 6.6.6.6 det 
Local      Outgoing   Prefix           Bytes Label   Outgoing   Next Hop    
Label      Label      or Tunnel Id     Switched      interface              
20         16         6.6.6.6/32       0             Tu200      point2point 
        MAC/Encaps=14/18, MRU=1500, Label Stack{16}, via Et0/2
        AABBCC016320AABBCC0161208847 00010000
        No output feature configured
    Per-destination load-sharing, slots: 0
           20         6.6.6.6/32       0             Et0/0      34.34.34.4  
        MAC/Encaps=14/18, MRU=1500, Label Stack{20}
        AABBCC016200AABBCC0161008847 00014000
        No output feature configured
    Per-destination load-sharing, slots: 1
R3#sh mpls for 5.5.5.5 det
Local      Outgoing   Prefix           Bytes Label   Outgoing   Next Hop    
Label      Label      or Tunnel Id     Switched      interface              
19         Pop Label  5.5.5.5/32       0             Tu200      point2point 
        MAC/Encaps=14/14, MRU=1504, Label Stack{}, via Et0/2
        AABBCC016320AABBCC0161208847 
        No output feature configured
R3#



R3#sh ip cef 6.6.6.6 int
6.6.6.6/32, epoch 0, flags need ps clean, RIB[I], refcount 5, per-destination sharing
  sources: RIB, LTE 
  feature space:
   IPRM: 0x00028000
   LFD: 6.6.6.6/32 1 local label
   local label info: global/20
        contains path extension list
        disposition chain 0x04E78E40
        label switch chain 0x04E78E40
  ifnums:
   Ethernet0/0(2): 34.34.34.4
   Tunnel200(20)
  path 04C0EFF0, path list 04275020, share 1/1, type attached nexthop, for IPv4
    MPLS short path extensions: MOI flags = 0x0 label 16
  nexthop 5.5.5.5 Tunnel200 label 16, adjacency IP midchain out of Tunnel200 02F8EE48
  path 04C0EB20, path list 04275020, share 1/1, type attached nexthop, for IPv4
    MPLS short path extensions: MOI flags = 0x0 label 20
  nexthop 34.34.34.4 Ethernet0/0 label 20, adjacency IP adj out of Ethernet0/0, addr 34.34.34.4 04C21B78
  output chain:
    loadinfo 04E78E40, per-session, 2 choices, flags 009B, 7 locks
    flags: Per-session, for-rx-IPv4, for-mpls-at-eos, for-mpls-not-at-eos, 2buckets
    2 hash buckets
      < 0 > label 16 TAG midchain out of Tunnel200 02F8EFA8 label implicit-null FRR 1 TAG adj out of Ethernet0/2, addr 35.35.35.5 02F8EA28
      < 1 > label 20 TAG adj out of Ethernet0/0, addr 34.34.34.4 02F8F3C8
    Subblocks:
     None
R3#

If you notice the above outputs on R1, we see that in order for R1 to reach R6, the path is being taken as R1 --- R3 --- R5 --- R6. When we check on R3, we notice that R3 can reach R6 via Tunnel 200 which is destined to R5. Thus is order to complete the LSP from R1 to R6, Tunnel 200 can be used to calculate the path. This will also help to make the path calculation quicker as the LSP to R6 is already completed by Tunnel 200. Since Tunnel 200 on R3 is in the path of R1 to R6, thus this router will act as midpoint router for the Tunnel 201 on R1. Since Tunnel 200 on R3 is used to reach R5, this router will also act as a midpoint router for the Tunnel 200 on R1. Now, lets have a look at the output:

On R3:
======

R3#sh mpls traffic-eng fast-reroute database 
P2P Headend FRR information:
Protected tunnel               In-label Out intf/label   FRR intf/label   Status
---------------------------    -------- --------------   --------------   ------
Tunnel200                      Tun hd   Et0/2:implicit-n Tu150:implicit-n Ready 

P2P LSP midpoint frr information:
LSP identifier                 In-label Out intf/label   FRR intf/label   Status
---------------------------    -------- --------------   --------------   ------
1.1.1.1 200 [666]              29       Et0/2:implicit-n Tu150:implicit-n Ready 
1.1.1.1 201 [699]              26       Et0/2:34         Tu150:34         Ready 

P2MP Sub-LSP FRR information:
*Sub-LSP identifier
src_lspid[subid]->dst_tunid    In-label Out intf/label   FRR intf/label   Status
---------------------------    -------- --------------   --------------   ------

* Sub-LSP identifier format: <TunSrc>_<LSP_ID>[SubgroupID]-><TunDst>_<Tun_ID>
  Note: Sub-LSP identifier may be truncated.
  Use 'detail' display for the complete key.
R3#

From the above output, we see that the router R3 is acting as a midpoint router for both Tunnel 200 and Tunnel 201 on R1. You can also see their In and Out Labels along with the FRR labels. Lets now try to shut the link between R3 and R5 and see what changes get affected on routers R1 and R3.

On R3:
======

R3(config)#int e0/2
R3(config-if)#shut
R3(config-if)#end
R3#

R3#sh mpls traffic-eng tunnels tu200

Name: R3_t200                             (Tunnel200) Destination: 5.5.5.5
  Status:
    Admin: up         Oper: up     Path: valid       Signalling: connected
    path option 1, type dynamic (Basis for Setup, path weight 10000)

  Config Parameters:
    Bandwidth: 0        kbps (Global)  Priority: 7  7   Affinity: 0x0/0xFFFF
    Metric Type: TE (default)
    AutoRoute announce: enabled  LockDown: disabled Loadshare: 0        bw-based
    auto-bw: disabled
  Active Path Option Parameters:
    State: dynamic path option 1 is active
    BandwidthOverride: disabled  LockDown: disabled  Verbatim: disabled


  InLabel  :  -
  OutLabel : Ethernet0/0, 27
  Next Hop : 34.34.34.4
  RSVP Signalling Info:
       Src 3.3.3.3, Dst 5.5.5.5, Tun_Id 200, Tun_Instance 334
    RSVP Path Info:
      My Address: 34.34.34.3   
      Explicit Route: 34.34.34.4 45.45.45.4 45.45.45.5 5.5.5.5 
      Record   Route:   NONE
      Tspec: ave rate=0 kbits, burst=1000 bytes, peak rate=0 kbits
    RSVP Resv Info:
      Record   Route:  4.4.4.4(27) 5.5.5.5(0)
      Fspec: ave rate=0 kbits, burst=1000 bytes, peak rate=0 kbits
  Shortest Unconstrained Path Info:
    Path Weight: 10000 (TE)
    Explicit Route: 34.34.34.3 34.34.34.4 45.45.45.4 45.45.45.5 
                    5.5.5.5 
  History:
    Tunnel:
      Time since created: 11 days, 12 minutes
      Time since path change: 12 seconds
      Number of LSP IDs (Tun_Instances) used: 334
    Current LSP: [ID: 334]
      Uptime: 15 seconds
      Selection: reoptimization
    Prior LSP: [ID: 295]
      ID: path option 1 [295]
      Removal Trigger: re-route path error
      Last Error: RSVP:: Path Error from 3.3.3.3: Notify: Tunnel locally repaired (flags 0)
R3#


R3#sh mpls traffic-eng fast-reroute database 
P2P Headend FRR information:
Protected tunnel               In-label Out intf/label   FRR intf/label   Status
---------------------------    -------- --------------   --------------   ------

P2P LSP midpoint frr information:
LSP identifier                 In-label Out intf/label   FRR intf/label   Status
---------------------------    -------- --------------   --------------   ------

P2MP Sub-LSP FRR information:
*Sub-LSP identifier
src_lspid[subid]->dst_tunid    In-label Out intf/label   FRR intf/label   Status
---------------------------    -------- --------------   --------------   ------

* Sub-LSP identifier format: <TunSrc>_<LSP_ID>[SubgroupID]-><TunDst>_<Tun_ID>
  Note: Sub-LSP identifier may be truncated.
  Use 'detail' display for the complete key.
R3#   

On R1:
=====


R1#sh mpls tra tu tu201

Name: R1_t201                             (Tunnel201) Destination: 6.6.6.6
  Status:
    Admin: up         Oper: up     Path: valid       Signalling: connected
    path option 1, type dynamic (Basis for Setup, path weight 15000)

  Config Parameters:
    Bandwidth: 0        kbps (Global)  Priority: 7  7   Affinity: 0x0/0xFFFF
    Metric Type: TE (default)
    AutoRoute:  enabled   LockDown: disabled  Loadshare: 0        bw-based
    auto-bw: disabled
  Active Path Option Parameters:
    State: dynamic path option 1 is active
    BandwidthOverride: disabled  LockDown: disabled  Verbatim: disabled


  InLabel  :  - 
  OutLabel : Ethernet0/0, 29
  RSVP Signalling Info:
       Src 1.1.1.1, Dst 6.6.6.6, Tun_Id 201, Tun_Instance 740
    RSVP Path Info:
      My Address: 12.12.12.1   
      Explicit Route: 12.12.12.2 24.24.24.2 24.24.24.4 46.46.46.4 
                      46.46.46.6 6.6.6.6 
      Record   Route:   NONE
      Tspec: ave rate=0 kbits, burst=1000 bytes, peak rate=0 kbits
    RSVP Resv Info:
      Record   Route:  2.2.2.2(29) 4.4.4.4(26)
                       6.6.6.6(0)
      Fspec: ave rate=0 kbits, burst=1000 bytes, peak rate=0 kbits
  Shortest Unconstrained Path Info:
    Path Weight: 15000 (TE)
    Explicit Route: 12.12.12.1 12.12.12.2 24.24.24.2 24.24.24.4 
                    46.46.46.4 46.46.46.6 6.6.6.6 
  History:
    Tunnel:
      Time since created: 26 days, 12 hours, 16 minutes
      Time since path change: 1 minutes, 48 seconds
      Number of LSP IDs (Tun_Instances) used: 740
    Current LSP:
      Uptime: 1 minutes, 51 seconds
      Selection: reoptimization
    Prior LSP:
      ID: path option 1 [699]
      Removal Trigger: re-route path error
R1#

With the above outputs, we see that on R1, a new path has been calculated and now the path to R6 from R1 is taken as R1 --- R2 --- R4 --- R6. Also, if we see on R3, There are no more backup Tunnels in the database and no more midpoints. Once we bring back the interface between R3 and R5, we will be able to see the same results as seen previously. 

Hope, now you have got a better understanding of how MPLS TE link protection works and also the concepts of TE midpoints.

Cheers...!!!

Comments (2) -

  • Tarique

    8/8/2012 5:13:21 PM |

    Hello Genie,

    Excellent post!  One doubt that has triggered my mind for some time is MPLS TE Tunnels and load balancing.  Let say we are to have two equal cost links between R3 and R5, call it linkR35 -1 and linkR35-2.  We are trying to build a TE tunnel with R1 has head end and R5 as tail end.  Assuming cost of all links are exactly same.  Would there be two tunnels formed at R3 going to same tail end R5 one via link35-1 and another linkR35-2.  Would they actually load balance?  I do not think so because CEF Hashing will separate the flows I believe.  Then is there a way for us to load balance in this scenario?

    I heard its possible only via Multipoint MPLS TE but I am not that knowledgeable on MP-MPLS TE will read up on that and revert back.

    Thank you for your posting.

    regards,
    Tarique

  • Genie

    8/8/2012 7:43:15 PM |

    Hello Tarique,
    Thanks for posting your questions / comments. Yes, you are right. The flows will be seperated by the cef. I will share a simple example with you. Lets add another link between R3 and R5 with similar configuration as other links. We shut down the tunnel 200 in the above scenario and we notice that the traffic is getting load Balanced:


    R3#sh ip cef 5.5.5.5 internal
    5.5.5.5/32, epoch 0, flags need ps clean, RIB[I], refcount 5, per-destination sh
    aring
      sources: RIB, LTE
      feature space:
       IPRM: 0x00028000
       LFD: 5.5.5.5/32 1 local label
       local label info: global/20
            contains path extension list
            disposition chain 0x051F28C8
            label switch chain 0x051F2988
      ifnums:
       Ethernet0/2(4): 35.35.35.5
       Ethernet0/3(5): 53.53.53.5
      path 051560D8, path list 0514A4C0, share 1/1, type attached nexthop, for IPv4
        MPLS short path extensions: MOI flags = 0x0 label implicit-null
      nexthop 35.35.35.5 Ethernet0/2, adjacency IP adj out of Ethernet0/2, addr 35.3
    5.35.5 051578E8
      path 05156618, path list 0514A4C0, share 1/1, type attached nexthop, for IPv4
        MPLS short path extensions: MOI flags = 0x0 label implicit-null
      nexthop 53.53.53.5 Ethernet0/3, adjacency IP adj out of Ethernet0/3, addr 53.5
    3.53.5 0515A170
      output chain:
        loadinfo 042AB360, per-session, 2 choices, flags 0083, 6 locks
        flags: Per-session, for-rx-IPv4, 2buckets
        2 hash buckets
          < 0 > IP adj out of Ethernet0/2, addr 35.35.35.5 051578E8
          < 1 > IP adj out of Ethernet0/3, addr 53.53.53.5 0515A170
        Subblocks:
         None
    R3#

    As we all know the load-balancing is per-destination based by default, so there will be two has buckets formed. one for the interface Ethernet 0/2 and the other one for Ethernet 0/3. This means that the hash algorithm is based on a per source/destination IP address pair of the IP packet. For a labelled packet this requires packet inspection past the label headers to determine the IP addresses.
    If per-packet load sharing is selected then it will round-robin packets over the paths. Per packet load sharing is generally discouraged due to some applications such as voice being affected by out of order packets. Per-destination will work well the greater the number of source/destination pairs there are and is the recommended way of doing configuring load balancing in the core. It is important to understand that when TE provisions a label switch path, the computation process only considers a single path regardless of whether there are equal cost paths available between routers. The TE computation will only select a single outgoing interface of the many that may be available via CEF for a particular TE tail loopback. The explicit path created is not subject to the entries that appear in the routing table but rather to other constraints such as bandwidth and resource affinity bits.
    Now, Since in the above example we see that the tunnel 200 is a dynamic tunnel with autoroute announce option, The dynamic path option creates an LSP that follows the IGP routing table (based on a constraint of zero bandwidth) but only a single path will be chosen if multiple equal cost paths exist at each hop of the TE LSP. Lets now have a look at the output.
    R3#sh ip route 5.5.5.5    
    Routing entry for 5.5.5.5/32
      Known via "isis", distance 115, metric 5000, type level-2
      Redistributing via isis
      Last update from 5.5.5.5 on Tunnel200, 00:00:09 ago
      Routing Descriptor Blocks:
      * 5.5.5.5, from 5.5.5.5, 00:00:09 ago, via Tunnel200
          Route metric is 5000, traffic share count is 1
    R3#
    R3#sh ip cef 5.5.5.5 internal
    5.5.5.5/32, epoch 0, flags need ps clean, RIB[I], refcount 5, per-destination sharing
      sources: RIB, LTE
      feature space:
       IPRM: 0x00028000
       LFD: 5.5.5.5/32 1 local label
       local label info: global/20
            contains path extension list
            disposition chain 0x0519BB70
            label switch chain 0x0519C3B0
      ifnums:
       Tunnel200(21)
      path 051561B8, path list 0514A258, share 1/1, type attached nexthop, for IPv4
        MPLS short path extensions: MOI flags = 0x0 label implicit-null
      nexthop 5.5.5.5 Tunnel200, adjacency IP midchain out of Tunnel200 0515A010
      output chain: IP midchain out of Tunnel200 0515A010 label implicit-null FRR 1 IP adj out of Ethernet0/2, addr 35.35.35.5 051578E8
    R3#
    From the above output, we notice that only single interface was selected which is Ethernet 0/2.
    Once this tunnel is created, autoroute will announce its availability to OSPF which will then use it in its SPF at Router R3. The result is that a single path will be used from R3 to R5 across the MPLS core as confirmed.


    R3#trace mpls traffic-eng tunnel 200
    Tracing MPLS TE Label Switched Path on Tunnel200, timeout is 2 seconds

    Codes: '!' - success, 'Q' - request not sent, '.' - timeout,
      'L' - labeled output interface, 'B' - unlabeled output interface,
      'D' - DS Map mismatch, 'F' - no FEC mapping, 'f' - FEC mismatch,
      'M' - malformed request, 'm' - unsupported tlvs, 'N' - no label entry,
      'P' - no rx intf label prot, 'p' - premature termination of LSP,
      'R' - transit router, 'I' - unknown upstream index,
      'X' - unknown return code, 'x' - return code 0

    Type escape sequence to abort.
      0 35.35.35.3 MRU 1500 [Labels: implicit-null Exp: 0]
    ! 1 35.35.35.5 388 ms
    R3#

    If we need to perform load-balancing via TE tunnels, we need to create two TE tunnels between same source and destination taking different paths. That way CEF will again kick in and perform the load-balancing. Regarding the multipoint TE tunnels, I would like to discuss this in a seperate post.
    Hope your question has been answered.

    Genie

Comments are closed