Genie's Tech Blog

Where knowledge has no dimensions

MPLS VPN - Basic

Hello Everyone,

Today we will be discussing about configuring & troubleshooting MPLS VPN along with different scenarios in the core. For this example, I expect you to be familiar with basic concepts of MPLS, LDP and  understanding of MPLS VPN. To start with we first need to understand what are the major components of a MPLS VPN scenario and a basic understanding of the same. Following are the major components of MPLS VPN:

1. LDP (Label Discovery protocol) : LDP has been defined in RFC 3036 (and version 2 in RFC 5036). LDP is a signalling protocol which is assigns distributes and installs (in forwarding) labels for prefixes advertised by unicast routing protocols like ISIS, OSPF and Static routing. LDP does not assigns labels for BGP generated prefixes. Uses UDP for neighbor discovery (Hellos) and TCP for connection and message exchange with peers (TCP and UDP port number 646). LDP is also extended as signalling protocol for other application like L2VPN Pseudo-wire, ICCP, mLDP. Typical deployment scenarios are MPLS as part of core for L3 and L2 VPN, BGP free core with MPLS, MPLS L2VPN signalliing services, MPLS L3VPN Inter-As Option C, MPLS based CsC VPN, ICCP signallying services and mLDP signalling services.

2. Virtual Routing and Forwarding (VRF) : VRF allows a router to have multiple and seperate instances of routing table at the same time. These routing tables co-exist on the same router, but yet are segmented and independent of each other.

3. Route Distinguisher (RD) : It's a way to identify a VPN route in an MPLS network. What this does is add an 8-byte value to an IPv4 prefix to create a VPNv4 prefix, typically referred to as an VPN-IPv4 address. Since each customer is assigned a unique RD, their addresses are guaranteed to be unique. So if you have Customer A that has a 10/8 address space and Customer B that also has a 10/8 address space, the addition of the RD brings uniqueness to each of them. A RD is in the format of ASN:NN or IP_Add:NN where NN represents a number.

4. Route-Target (RT) : An RT is a BGP extended community that indicates which routes should be imported from MP-BGP into the VRF. Exporting an RT means that the exported vpnv4 route receives an additional BGP extended community as configured under ip vrf on the PE router, when the route is redistributed from the VRF routing table into MP-BGP. Importing an RT means that the received vpnv4 route from MP-BGP is checked for a matching extended community with the ones in the configuration. If the result is a match, the prefix is put into the VRF routing table as an IPv4 route. If a match does not occur, the prefix is rejected.

5. Multi-Protocol BGP (MP-BGP) : Since RFC4577 many Service Providers offer Virtual Private Network (VPN) services to their customers, using a technique in which customer edge routers (CE routers) are routing peers of provider edge routers (PE routers). The Multi-Protocol Border Gateway Protocol (MP-BGP) is used to distribute the customers routes across the providers IP backbone network, and Multiprotocol Label Switching (MPLS) is used to tunnel customer packets across the providers backbone. This is known as a “BGP/MPLS IP VPN”. Now the VRF part of BGP comes into play as it provides the ability to redistribute between BGP and the defined customer VRF. Your PE to CE relationship can be any routing protocol you choose – RIP, OSPF, EIGRP, Static, BGP, etc and – based on the RD defined – you have the ability to redistribute the routes between your protocol of choice and the BGP VRF.

MPLS TE has already been covered in one of our posts, so will request everyone to refer it from there. Now lets start with the most fun part i.e configuring and troubleshooting MPLS VPNs.

Configuring MPLS VPN:

Lets consider the following topology:

 

 

The very first process as a part of configuring MPLS VPN is configuring the IGP. I will be using OSPF as the IGP protocol. I will also be sharing the MPLS LDP config. Lets have a look at the below config

Config on PE1:
=============
mpls label protocol ldp
!
interface Loopback0
 ip address 65.0.0.1 255.255.255.255
!        
interface Ethernet1/0
 ip address 23.23.23.1 255.255.255.252
 mpls ip
!
router ospf 100
 router-id 65.0.0.1
 network 23.23.23.1 0.0.0.0 area 0
 network 65.0.0.1 0.0.0.0 area 0
!
mpls ldp router-id Loopback0 force
!

Config on P:
============
mpls label protocol ldp
!
interface Loopback0
 ip address 65.0.0.2 255.255.255.255
!        
interface Ethernet0/0
 ip address 34.34.34.1 255.255.255.252
 mpls ip
!
interface Ethernet1/0
 ip address 23.23.23.2 255.255.255.252
 mpls ip
!
router ospf 100
 router-id 65.0.0.2
 network 23.23.23.2 0.0.0.0 area 0
 network 34.34.34.1 0.0.0.0 area 0
 network 65.0.0.2 0.0.0.0 area 0
!
mpls ldp router-id Loopback0 force
!

Config on PE2:
=============
mpls label protocol ldp
!
interface Loopback0
 ip address 65.0.0.3 255.255.255.255
!        
interface Ethernet0/0
 ip address 34.34.34.2 255.255.255.252
 mpls ip
!
router ospf 100
 router-id 65.0.0.3
 network 34.34.34.2 0.0.0.0 area 0
 network 65.0.0.3 0.0.0.0 area 0
!
mpls ldp router-id Loopback0 force
!

Once we have configured the IGP and the MPLS ldp, lets now have a look at the following outputs which will clarify the flow of routing and label generation:

Outputs on PE1:
==============
PE1#sh ip route
Codes: L - local, C - connected, S - static, R - RIP, M - mobile, B - BGP
       D - EIGRP, EX - EIGRP external, O - OSPF, IA - OSPF inter area 
       N1 - OSPF NSSA external type 1, N2 - OSPF NSSA external type 2
       E1 - OSPF external type 1, E2 - OSPF external type 2
       i - IS-IS, su - IS-IS summary, L1 - IS-IS level-1, L2 - IS-IS level-2
       ia - IS-IS inter area, * - candidate default, U - per-user static route
       o - ODR, P - periodic downloaded static route, H - NHRP, l - LISP
       + - replicated route, % - next hop override

Gateway of last resort is not set

      23.0.0.0/8 is variably subnetted, 2 subnets, 2 masks
C        23.23.23.0/30 is directly connected, Ethernet1/0
L        23.23.23.1/32 is directly connected, Ethernet1/0
      34.0.0.0/30 is subnetted, 1 subnets
O        34.34.34.0 [110/20] via 23.23.23.2, 5d01h, Ethernet1/0
      65.0.0.0/32 is subnetted, 3 subnets
C        65.0.0.1 is directly connected, Loopback0
O        65.0.0.2 [110/11] via 23.23.23.2, 5d01h, Ethernet1/0
O        65.0.0.3 [110/21] via 23.23.23.2, 5d01h, Ethernet1/0
PE1#
PE1#sh mpls forwarding-table 
Local      Outgoing   Prefix           Bytes Label   Outgoing   Next Hop    
Label      Label      or Tunnel Id     Switched      interface              
16         Pop Label  34.34.34.0/30    0             Et1/0      23.23.23.2  
17         Pop Label  65.0.0.2/32      0             Et1/0      23.23.23.2  
18         17         65.0.0.3/32      0             Et1/0      23.23.23.2  
19         No Label   l2ckt(4097)      2256          Et0/0.10   point2point 
20         No Label   l2ckt(8217)      229806        Et0/0      point2point 
PE1#
PE1#sh mpls ldp nei
    Peer LDP Ident: 65.0.0.2:0; Local LDP Ident 65.0.0.1:0
        TCP connection: 65.0.0.2.49430 - 65.0.0.1.646
        State: Oper; Msgs sent/rcvd: 8411/8410; Downstream
        Up time: 5d02h
        LDP discovery sources:
          Ethernet1/0, Src IP addr: 23.23.23.2
        Addresses bound to peer LDP Ident:
          34.34.34.1      23.23.23.2      65.0.0.2        

Outputs on P:
=============
P#sh ip route
Codes: L - local, C - connected, S - static, R - RIP, M - mobile, B - BGP
       D - EIGRP, EX - EIGRP external, O - OSPF, IA - OSPF inter area 
       N1 - OSPF NSSA external type 1, N2 - OSPF NSSA external type 2
       E1 - OSPF external type 1, E2 - OSPF external type 2
       i - IS-IS, su - IS-IS summary, L1 - IS-IS level-1, L2 - IS-IS level-2
       ia - IS-IS inter area, * - candidate default, U - per-user static route
       o - ODR, P - periodic downloaded static route, H - NHRP, l - LISP
       + - replicated route, % - next hop override

Gateway of last resort is not set

      23.0.0.0/8 is variably subnetted, 2 subnets, 2 masks
C        23.23.23.0/30 is directly connected, Ethernet1/0
L        23.23.23.2/32 is directly connected, Ethernet1/0
      34.0.0.0/8 is variably subnetted, 2 subnets, 2 masks
C        34.34.34.0/30 is directly connected, Ethernet0/0
L        34.34.34.1/32 is directly connected, Ethernet0/0
      65.0.0.0/32 is subnetted, 3 subnets
O        65.0.0.1 [110/11] via 23.23.23.1, 5d02h, Ethernet1/0
C        65.0.0.2 is directly connected, Loopback0
O        65.0.0.3 [110/11] via 34.34.34.2, 5d02h, Ethernet0/0
P#
P#sh mpls for
P#sh mpls forwarding-table 
Local      Outgoing   Prefix           Bytes Label   Outgoing   Next Hop    
Label      Label      or Tunnel Id     Switched      interface              
16         Pop Label  65.0.0.1/32      5583640       Et1/0      23.23.23.1  
17         Pop Label  65.0.0.3/32      5578816       Et0/0      34.34.34.2  
P#
P#sh mpls ldp nei
    Peer LDP Ident: 65.0.0.1:0; Local LDP Ident 65.0.0.2:0
        TCP connection: 65.0.0.1.646 - 65.0.0.2.49430
        State: Oper; Msgs sent/rcvd: 8411/8412; Downstream
        Up time: 5d02h
        LDP discovery sources:
          Ethernet1/0, Src IP addr: 23.23.23.1
        Addresses bound to peer LDP Ident:
          23.23.23.1      65.0.0.1        
    Peer LDP Ident: 65.0.0.3:0; Local LDP Ident 65.0.0.2:0
        TCP connection: 65.0.0.3.53359 - 65.0.0.2.646
        State: Oper; Msgs sent/rcvd: 8414/8421; Downstream
        Up time: 5d02h
        LDP discovery sources:
          Ethernet0/0, Src IP addr: 34.34.34.2
        Addresses bound to peer LDP Ident:
          34.34.34.2      65.0.0.3        
P#

Outputs on PE2:
===============
PE2#sh ip route
Codes: L - local, C - connected, S - static, R - RIP, M - mobile, B - BGP
       D - EIGRP, EX - EIGRP external, O - OSPF, IA - OSPF inter area 
       N1 - OSPF NSSA external type 1, N2 - OSPF NSSA external type 2
       E1 - OSPF external type 1, E2 - OSPF external type 2
       i - IS-IS, su - IS-IS summary, L1 - IS-IS level-1, L2 - IS-IS level-2
       ia - IS-IS inter area, * - candidate default, U - per-user static route
       o - ODR, P - periodic downloaded static route, H - NHRP, l - LISP
       + - replicated route, % - next hop override

Gateway of last resort is not set

      23.0.0.0/30 is subnetted, 1 subnets
O        23.23.23.0 [110/20] via 34.34.34.1, 5d02h, Ethernet0/0
      34.0.0.0/8 is variably subnetted, 2 subnets, 2 masks
C        34.34.34.0/30 is directly connected, Ethernet0/0
L        34.34.34.2/32 is directly connected, Ethernet0/0
      65.0.0.0/32 is subnetted, 3 subnets
O        65.0.0.1 [110/21] via 34.34.34.1, 5d02h, Ethernet0/0
O        65.0.0.2 [110/11] via 34.34.34.1, 5d02h, Ethernet0/0
C        65.0.0.3 is directly connected, Loopback0
PE2# 
PE2#sh mpls for
Local      Outgoing   Prefix           Bytes Label   Outgoing   Next Hop    
Label      Label      or Tunnel Id     Switched      interface              
16         16         65.0.0.1/32      0             Et0/0      34.34.34.1  
17         Pop Label  23.23.23.0/30    0             Et0/0      34.34.34.1  
18         Pop Label  65.0.0.2/32      0             Et0/0      34.34.34.1  
19         No Label   l2ckt(4097)      3510          Et1/0.10   point2point 
20         No Label   l2ckt(8217)      229866        Et1/0      point2point 
PE2#
PE2#sh mpls ldp nei
    Peer LDP Ident: 65.0.0.2:0; Local LDP Ident 65.0.0.3:0
        TCP connection: 65.0.0.2.646 - 65.0.0.3.53359
        State: Oper; Msgs sent/rcvd: 8421/8414; Downstream
        Up time: 5d02h
        LDP discovery sources:
          Ethernet0/0, Src IP addr: 34.34.34.1
        Addresses bound to peer LDP Ident:
          34.34.34.1      23.23.23.2      65.0.0.2

The above outputs give us the basic understanding of the how the unicast routing is happening and the label information of different prefixes from all the nodes. I will explain one of the labeling information between PE1 and PE2 loopback ip address. Now, if we see the show mpls forwarding-table output on PE1, we see that the local label generated for 65.0.0.3/32 prefix is 18, the outgoing label is 17 and the next-hop address is 23.23.23.2 which is the P router. This outgoing label 17, we have received it from the same P router. The label 17 is the local label on P router for the prefix 65.0.0.3/32. It has the outgoing label as the Pop label which is the label value 3 (Implicit-Null Label) which we have received it from PE2 due to the concept of Penultimate-Hop-Popping (PHP). So now, if we do a trace from PE1 to PE2, we see the following output:

PE1#trace 65.0.0.3 sou lo0 numeric 
Type escape sequence to abort.
Tracing the route to 65.0.0.3
VRF info: (vrf in name/id, vrf out name/id)
  1 23.23.23.2 [MPLS: Label 17 Exp 0] 0 msec 0 msec 0 msec  <<<<<<<<
  2 34.34.34.2 0 msec *  1 msec
PE1#

From the above output we see that on the next-hop we see the mpls label as 17 which we saw as the local label for prefix 65.0.0.3/32 on P router. This way with the traceroute command output we also get to know the label information in the path. Now, in order to check for the LSP to be complete, we can use the "ping mpls" command. Lets have a look at it here:

PE1#ping mpls ipv4 65.0.0.3 255.255.255.255           
Sending 5, 100-byte MPLS Echos to 65.0.0.3/32, 
     timeout is 2 seconds, send interval is 0 msec:

Codes: '!' - success, 'Q' - request not sent, '.' - timeout,
  'L' - labeled output interface, 'B' - unlabeled output interface, 
  'D' - DS Map mismatch, 'F' - no FEC mapping, 'f' - FEC mismatch,
  'M' - malformed request, 'm' - unsupported tlvs, 'N' - no label entry, 
  'P' - no rx intf label prot, 'p' - premature termination of LSP, 
  'R' - transit router, 'I' - unknown upstream index,
  'l' - Label switched with FEC change, 'd' - see DDMAP for return code,
  'X' - unknown return code, 'x' - return code 0

Type escape sequence to abort.
!!!!!
Success rate is 100 percent (5/5), round-trip min/avg/max = 1/1/1 ms
 Total Time Elapsed 6 ms

PE1#

The successful ping result confirms that the LSP (labelled switched path) is complete. We try to remove the "mpls ip" config from the interface on P router towards PE2 and we see the following result from the "ping mpls" output:

From P router:
==========

P#conf t
Enter configuration commands, one per line.  End with CNTL/Z.
P(config)#int e0/0
P(config-if)#no mpls ip
P(config-if)#end
P#
*Oct 23 19:42:38.988: %LDP-5-NBRCHG: LDP Neighbor 65.0.0.3:0 (2) is DOWN (LDP disabled on interface)
*Oct 23 19:42:39.910: %SYS-5-CONFIG_I: Configured from console by console
P#

From PE1 router:
============

PE1#ping mpls ipv4 65.0.0.3 255.255.255.255
Sending 5, 100-byte MPLS Echos to 65.0.0.3/32, 
     timeout is 2 seconds, send interval is 0 msec:

Codes: '!' - success, 'Q' - request not sent, '.' - timeout,
  'L' - labeled output interface, 'B' - unlabeled output interface, 
  'D' - DS Map mismatch, 'F' - no FEC mapping, 'f' - FEC mismatch,
  'M' - malformed request, 'm' - unsupported tlvs, 'N' - no label entry, 
  'P' - no rx intf label prot, 'p' - premature termination of LSP, 
  'R' - transit router, 'I' - unknown upstream index,
  'l' - Label switched with FEC change, 'd' - see DDMAP for return code,
  'X' - unknown return code, 'x' - return code 0

Type escape sequence to abort.
BBBBB
Success rate is 0 percent (0/5)
 Total Time Elapsed 6 ms

PE1#

Hope by now, we understand how to troubleshoot the broken LSP Path and how to check for the labels along the path. We shall now configure vrf ABC on both sides i.e. PE1 and PE2 router with the respective interfaces and routing protocols along with the CE1 and CE2 routers.

Config on PE1:
==============
vrf definition ABC
 rd 100:1
 route-target export 100:1
 route-target import 100:1
 !
 address-family ipv4
 exit-address-family
!
!
interface Ethernet0/0
 no ip address
!
interface Ethernet0/0.100
 encapsulation dot1Q 100
 vrf forwarding ABC
 ip address 192.168.10.1 255.255.255.0
!         
interface Loopback1
 vrf forwarding ABC
 ip address 172.65.0.1 255.255.255.255
!
router ospf 200 vrf ABC
 router-id 172.65.0.1
 network 172.65.0.1 0.0.0.0 area 0
 network 192.168.10.1 0.0.0.0 area 0
!


Config on PE2:
==============
vrf definition ABC
 rd 100:1
 route-target export 100:1
 route-target import 100:1
 !
 address-family ipv4
 exit-address-family
!
!
interface Ethernet1/0
 no ip address
!
interface Ethernet1/0.100
 encapsulation dot1Q 100
 vrf forwarding ABC
 ip address 192.168.20.1 255.255.255.0
!
interface Loopback1
 vrf forwarding ABC
 ip address 172.65.0.2 255.255.255.255
!
router ospf 200 vrf ABC
 router-id 172.65.0.2
 network 172.65.0.2 0.0.0.0 area 0
 network 192.168.20.1 0.0.0.0 area 0
!

Config on CE1:
=============
interface Loopback0
 ip address 1.1.1.1 255.255.255.255
!
 interface Ethernet0/0.100
 encapsulation dot1Q 100
 ip address 192.168.10.2 255.255.255.0
!
router ospf 200
 router-id 1.1.1.1
 network 1.1.1.1 0.0.0.0 area 0
 network 192.168.10.2 0.0.0.0 area 0
!

Config on CE2:
=============
interface Loopback0
 ip address 2.2.2.2 255.255.255.255
!
 interface Ethernet0/0.100
 encapsulation dot1Q 100
 ip address 192.168.20.2 255.255.255.0
!
router ospf 200
 router-id 2.2.2.2
 network 2.2.2.2 0.0.0.0 area 0
 network 192.168.20.2 0.0.0.0 area 0
!

From the above config, we see that vrf ABC has been configured with OSPF as the PE-CE protocol. Our next task is to being up the reachability between CE1 and CE2 or between 1.1.1.1/32 and 2.2.2.2/32. If we see the components described above the main component now is the MP-BGP. So we will be configuring vpnv4 address-family under BGP in order to make both the sites communicate with each other.

Config on PE1:
==============
router bgp 100
 bgp router-id 65.0.0.1
 bgp log-neighbor-changes
 no bgp default ipv4-unicast
 neighbor 65.0.0.3 remote-as 100
 neighbor 65.0.0.3 update-source Loopback0
 !
 address-family ipv4
 exit-address-family
 !
 address-family vpnv4
  neighbor 65.0.0.3 activate
  neighbor 65.0.0.3 send-community extended
  neighbor 65.0.0.3 next-hop-self
 exit-address-family
 !
 address-family ipv4 vrf ABC
  redistribute ospf 200 match internal external 1 external 2
 exit-address-family
!
router ospf 200 vrf ABC
redistribute bgp 100 subnets
!


Config on PE2:
==============
router bgp 100
 bgp router-id 65.0.0.3
 bgp log-neighbor-changes
 no bgp default ipv4-unicast
 neighbor 65.0.0.1 remote-as 100
 neighbor 65.0.0.1 update-source Loopback0
 !
 address-family ipv4
 exit-address-family
 !
 address-family vpnv4
  neighbor 65.0.0.1 activate
  neighbor 65.0.0.1 send-community extended
  neighbor 65.0.0.1 next-hop-self
 exit-address-family
 !
 address-family ipv4 vrf ABC
  redistribute ospf 200 match internal external 1 external 2
 exit-address-family
!

router ospf 200 vrf ABC
redistribute bgp 100 subnets
!

In the above config we see that we have done a vpnv4 peering between both PE1 and PE2. We also configure "address-family ipv4 vrf ABC" in order to redistribute the vrf routing table into the BGP which will be carried over to the remote PE router using the vpnv4 address-family. Also, we need to redistribute the prefixes that are learnt via the remote PE router into the local vrf routing table. Lets now check the reachability between CE1 and CE2 router:

Ouput on CE1:
==========
CE1#ping 2.2.2.2 so lo0
Type escape sequence to abort.
Sending 5, 100-byte ICMP Echos to 2.2.2.2, timeout is 2 seconds:
Packet sent with a source address of 1.1.1.1 
!!!!!
Success rate is 100 percent (5/5), round-trip min/avg/max = 1/1/1 ms
CE1#

Ouput on CE2:
==========
CE2#ping 1.1.1.1 sou lo0
Type escape sequence to abort.
Sending 5, 100-byte ICMP Echos to 1.1.1.1, timeout is 2 seconds:
Packet sent with a source address of 2.2.2.2 
!!!!!
Success rate is 100 percent (5/5), round-trip min/avg/max = 1/1/1 ms
CE2#

So we now see that there is a reachability between CE1 and CE2 loopback addresses. We shall now see how is the packet flow happening and what all things we need to look at in a MPLS VPN scenario.

Outputs on PE1:
==============
PE1#sh mpls for vrf ABC
Local      Outgoing   Prefix           Bytes Label   Outgoing   Next Hop    
Label      Label      or Tunnel Id     Switched      interface              
21         No Label   1.1.1.1/32[V]    1180          Et0/0.100  192.168.10.2  <<<<<<<<<<
22         Pop Label  172.65.0.1/32[V] 0             aggregate/ABC 
23         No Label   192.168.10.0/24[V]   \
                                       590           aggregate/ABC 
PE1#sh mpls for vrf ABC 172.65.0.2
Local      Outgoing   Prefix           Bytes Label   Outgoing   Next Hop    
Label      Label      or Tunnel Id     Switched      interface              
None       22         172.65.0.2/32[V] 0             Et1/0      23.23.23.2  

PE1#sh mpls for vrf ABC 2.2.2.2   
Local      Outgoing   Prefix           Bytes Label   Outgoing   Next Hop    
Label      Label      or Tunnel Id     Switched      interface              
None       21         2.2.2.2/32[V]    0             Et1/0      23.23.23.2  <<<<<<<<<<<

PE1#sh mpls for vrf ABC 192.168.20.0 24
Local      Outgoing   Prefix           Bytes Label   Outgoing   Next Hop    
Label      Label      or Tunnel Id     Switched      interface              
None       23         192.168.20.0/24[V]   \
                                       0             Et1/0      23.23.23.2  

PE1#show ip bgp vpnv4 vrf ABC labels 
   Network          Next Hop      In label/Out label
Route Distinguisher: 100:1 (ABC)
   1.1.1.1/32       192.168.10.2    21/nolabel
   2.2.2.2/32       65.0.0.3        nolabel/21    <<<<<<<<<<<<
   172.65.0.1/32    0.0.0.0         22/nolabel(ABC)
   172.65.0.2/32    65.0.0.3        nolabel/22
   192.168.10.0     0.0.0.0         23/nolabel(ABC)
   192.168.20.0     65.0.0.3        nolabel/23

PE1#sh ip cef vrf ABC 1.1.1.1 det
1.1.1.1/32, epoch 0
  local label info: other/21
  nexthop 192.168.10.2 Ethernet0/0.100
PE1#   
   
PE1#sh ip cef vrf ABC 1.1.1.1 internal 
1.1.1.1/32, epoch 0, RIB[I], refcount 5, per-destination sharing
  sources: RIB, LTE 
  feature space:
   IPRM: 0x00028000
   LFD: 1.1.1.1/32 1 local label
   local label info: other/21
        contains path extension list
        disposition chain 0xEFA12F64
  ifnums:
   Ethernet0/0.100(39): 192.168.10.2
  path EEA8C510, path list EEA92A2C, share 1/1, type attached nexthop, for IPv4
    MPLS short path extensions: MOI flags = 0x0 label none
  nexthop 192.168.10.2 Ethernet0/0.100, adjacency IP adj out of Ethernet0/0.100, addr 192.168.10.2 EE740228
  output chain: IP adj out of Ethernet0/0.100, addr 192.168.10.2 EE740228
PE1#

PE1#sh ip cef vrf ABC 2.2.2.2 det
2.2.2.2/32, epoch 0, flags rib defined all labels
  recursive via 65.0.0.3 label 21   <<<<<<<<<<<
    nexthop 23.23.23.2 Ethernet1/0 label 17   <<<<<<<<<<
PE1#

PE1#sh ip cef vrf ABC 2.2.2.2 internal 
2.2.2.2/32, epoch 0, flags rib defined all labels, RIB[B], refcount 5, per-destination sharing
  sources: RIB 
  feature space:
   IPRM: 0x00018000
   LFD: 2.2.2.2/32 0 local labels
        contains path extension list
  ifnums: (none)
  path EEA8C4A0, path list EEA929DC, share 1/1, type recursive, for IPv4, flags must-be-labelled
    MPLS short path extensions: MOI flags = 0x0 label 21    <<<<<<<<<<<<<<
  recursive via 65.0.0.3[IPv4:Default] label 21, fib EDCD1798, 1 terminal fib, v4:Default:65.0.0.3/32
    path EEEB4DA0, path list EE87A99C, share 1/1, type attached nexthop, for IPv4
      MPLS short path extensions: MOI flags = 0x0 label 17    <<<<<<<<<<<<<
    nexthop 23.23.23.2 Ethernet1/0 label 17, adjacency IP adj out of Ethernet1/0, addr 23.23.23.2 EF365EF0
  output chain: label 21 label 17 TAG adj out of Ethernet1/0, addr 23.23.23.2 EF365D90
PE1#

Outputs of PE2:
===============
PE2#sh mpls for vrf ABC
Local      Outgoing   Prefix           Bytes Label   Outgoing   Next Hop    
Label      Label      or Tunnel Id     Switched      interface              
21         No Label   2.2.2.2/32[V]    1770          Et1/0.100  192.168.20.2   <<<<<<<<<<<<
22         Pop Label  172.65.0.2/32[V] 0             aggregate/ABC 
23         No Label   192.168.20.0/24[V]   \
                                       0             aggregate/ABC 
PE2#
PE2#sh ip bgp vpnv4 vrf ABC labels 
   Network          Next Hop      In label/Out label
Route Distinguisher: 100:1 (ABC)
   1.1.1.1/32       65.0.0.1        nolabel/21
   2.2.2.2/32       192.168.20.2    21/nolabel   <<<<<<<<
   172.65.0.1/32    65.0.0.1        nolabel/22
   172.65.0.2/32    0.0.0.0         22/nolabel(ABC)
   192.168.10.0     65.0.0.1        nolabel/23
   192.168.20.0     0.0.0.0         23/nolabel(ABC)

PE2#
PE2#sh ip cef vrf ABC 2.2.2.2 det
2.2.2.2/32, epoch 0
  local label info: other/21
  nexthop 192.168.20.2 Ethernet1/0.100

PE2#sh ip cef vrf ABC 2.2.2.2 internal
2.2.2.2/32, epoch 0, RIB[I], refcount 5, per-destination sharing
  sources: RIB, LTE 
  feature space:
   IPRM: 0x00028000
   LFD: 2.2.2.2/32 1 local label
   local label info: other/21   <<<<<<<<<<<<
        contains path extension list
        disposition chain 0xEFAD2490
  ifnums:
   Ethernet1/0.100(40): 192.168.20.2
  path EEB51750, path list EE4F69AC, share 1/1, type attached nexthop, for IPv4
    MPLS short path extensions: MOI flags = 0x0 label none
  nexthop 192.168.20.2 Ethernet1/0.100, adjacency IP adj out of Ethernet1/0.100, addr 192.168.20.2 EE504D58
  output chain: IP adj out of Ethernet1/0.100, addr 192.168.20.2 EE504D58
PE2#

Lets now try to see the packet flow from the PE router perspective. For every prefix, the local PE router generates a VPN label and sends it over to the remote PE router. In our case, when CE1 loopback 1.1.1.1/32 tries to reach CE2 router at 2.2.2.2/32, it goes to the PE1 router. There it sees that the VPN label is 21 which its showing as the outgoing label in the mpls forwarding table of the vrf ABC. When you do "show mpls forwarding-table vrf <vrf_name>", it actually shows the VPN labels. When we see the detailed output of the same we can also see the tranport label (IGP label) which is used to carry out the prefix in the core. The core actually doesn't needs to know the customer prefix. That's the reason MPLS was invented i.e. for a BGP free core as the customer prefixes will be learnt through BGP from different sites.

PE1#sh mpls for vrf ABC 2.2.2.2 det 
Local      Outgoing   Prefix           Bytes Label   Outgoing   Next Hop    
Label      Label      or Tunnel Id     Switched      interface              
None       21         2.2.2.2/32[V]    0             Et1/0      23.23.23.2  
        MAC/Encaps=14/22, MRU=1496, Label Stack{17 21}
        AABBCC000301AABBCC0002018847 0001100000015000
        VPN route: ABC
        No output feature configured
PE1#

Thus in the above output we see that on PE1, the VPN label is 21 (outgoing label) but the label stack shows 2 labels. One is the transport label and the other is the VPN label. At the top of the stack is the transport label and at the bottom of the stack is the VPN label. Over here the transport label is 17 which was the same outgoing label on PE1 for reaching PE2 loopback address 65.0.0.3/32. Also, if you see the output of "show ip cef vrf ABC 2.2.2.2 internal" you will find that the prefix 2.2.2.2/32 is having a label value of 21 in vrf ABC and is reachable via 65.0.0.3 which is having the label value of 17 which is having 23.23.23.2 as the next-hop. Now since the VPN label is at the bottom of the stack, so only transport or the IGP label will get label swapped in the core and the packet or the label beneath it will remain unchanged and unnoticed. Above we noticed that PE2 gives an implicit-null label for the prefix 65.0.0.3/32 to P router, so when the packet reaches PE2, its only got local label that was assigned to prefix 2.2.2.2/32 in vrf ABC by PE2 router and the router thus knows that since its not got just the VPN label, it now needs to perform a RIB lookup in the vrf routing table from which we see that the data needs to go out via Ethernet1/0.100 interface in order to reach 2.2.2.2/32. Thus the packet flow for the MPLS VPN scenario.

LDP does not assign labels for BGP learnt prefixes:

I would also like to clarify a small concept, that LDP does not generates label for the BGP learnt prefixes. Only BGP itself can generate label for BGP learnt prefixes. Let me show you an example. Lets say on PE1 router we configure a loopback 100 with ip 65.1.0.1/32 and on PE we configure loopback 100 with ip 65.1.0.2/32. We configure iBGP peering between PE1 and PE2 and advertise this prefix through BGP. We now see the following results:

Outputs on PE1:
===============
PE1#sh ip route 65.1.0.2
Routing entry for 65.1.0.2/32
  Known via "bgp 100", distance 200, metric 0, type internal
  Last update from 65.0.0.3 00:00:30 ago
  Routing Descriptor Blocks:
  * 65.0.0.3, from 65.0.0.3, 00:00:30 ago
      Route metric is 0, traffic share count is 1
      AS Hops 0
      MPLS label: none

PE1#sh mpls for 65.1.0.2
Local      Outgoing   Prefix           Bytes Label   Outgoing   Next Hop    
Label      Label      or Tunnel Id     Switched      interface              
None       17         65.1.0.2/32      0             Et1/0      23.23.23.2  

PE1#sh mpls for 65.1.0.1
Local      Outgoing   Prefix           Bytes Label   Outgoing   Next Hop    
Label      Label      or Tunnel Id     Switched      interface              
None       No Label   65.1.0.1/32      0                        

PE1#sh ip cef 65.1.0.2 det
65.1.0.2/32, epoch 0, flags rib only nolabel, rib defined all labels
  recursive via 65.0.0.3
    nexthop 23.23.23.2 Ethernet1/0 label 17
PE1#sh ip cef 65.1.0.2 inter
PE1#sh ip cef 65.1.0.2 internal 
65.1.0.2/32, epoch 0, flags rib only nolabel, rib defined all labels, RIB[B], refcount 5, per-destination sharing
  sources: RIB 
  feature space:
   IPRM: 0x00018000
  ifnums:
   Ethernet1/0(6): 23.23.23.2
  path EEA8C4A0, path list EEA929DC, share 1/1, type recursive, for IPv4
  recursive via 65.0.0.3[IPv4:Default], fib EDCD1798, 1 terminal fib, v4:Default:65.0.0.3/32
    path EEEB4DA0, path list EE87A99C, share 1/1, type attached nexthop, for IPv4
      MPLS short path extensions: MOI flags = 0x0 label 17
    nexthop 23.23.23.2 Ethernet1/0 label 17, adjacency IP adj out of Ethernet1/0, addr 23.23.23.2 EF365EF0
  output chain: label 17 TAG adj out of Ethernet1/0, addr 23.23.23.2 EF365D90
PE1# 
PE1#

Outputs on PE2:
===============
PE2#sh mpls for 65.1.0.2
Local      Outgoing   Prefix           Bytes Label   Outgoing   Next Hop    
Label      Label      or Tunnel Id     Switched      interface              
None       No Label   65.1.0.2/32      0                        
PE2#

From the above output, we see that both PE1 and PE2 have not assigned any label for prefixes 65.1.0.1/32 and 65.1.0.2/32 respectively. Since on PE1, the next-hop is seen as 65.1.0.3/32 which is learnt via IGP, thus we see the outgoing label as 17.

Hope everything that has been explained is clear till now. Lets now see an example of the MPLS VPN scenario having MPLS Traffic-engineering configured in the core.

Config on PE1:
===========
interface Tunnel1
 ip unnumbered Loopback0
 mpls ip
 tunnel mode mpls traffic-eng
 tunnel destination 65.0.0.3
 tunnel mpls traffic-eng autoroute announce
 tunnel mpls traffic-eng path-option 1 dynamic
!
interface Ethernet1/0
 ip address 23.23.23.1 255.255.255.252
 mpls ip
 mpls traffic-eng tunnels
 ip rsvp bandwidth
!
router ospf 100
 router-id 65.0.0.1
 network 23.23.23.1 0.0.0.0 area 0
 network 65.0.0.1 0.0.0.0 area 0
 mpls traffic-eng router-id Loopback0
 mpls traffic-eng area 0
!

Similar interface and ospf config related to mpls traffic-eng has to be done on P and PE2 routers. Once the tunnel comes up, all the traffic sourced from 65.0.0.1 to 65.0.0.3 will show the outgoing interface as Tunnel 1. You must be wondering why "mpls ip" has been configured on the Tunnel interface. I will show you the reason why.

PE1#sh mpls for 65.0.0.3
Local      Outgoing   Prefix           Bytes Label   Outgoing   Next Hop    
Label      Label      or Tunnel Id     Switched      interface              
18    [T]  No Label   65.0.0.3/32      0             Tu1        point2point 

[T]     Forwarding through a LSP tunnel.
        View additional labelling info with the 'detail' option
PE1#

If I remove the "mpls ip" command from the tunnel interface, I dont see any outgoing label here. Actually this is not an expected behavior. The expected behavior is we should be able to see Pop label as the outgoing label but due to IOS defect we are seeing this issue. If we try to run the setup with this issue, the traffic wont pass though as the core will know that the router has the outgoing interface as Tunnel1 but it doesn't know the label through which it has to switch the traffic. In order to fix this issue, I have configured "mpls ip" and on the tunnel tail end I have configured "mpls ldp targeted-hello accept". This will actually fix the issue and we will now be able to look the Pop label as the outgoing label.

PE1#sh mpls for 65.0.0.3
Local      Outgoing   Prefix           Bytes Label   Outgoing   Next Hop    
Label      Label      or Tunnel Id     Switched      interface              
18    [T]  Pop Label  65.0.0.3/32      0             Tu1        point2point 

[T]     Forwarding through a LSP tunnel.
        View additional labelling info with the 'detail' option
PE1#

Now, with mpls traffic-eng configured, we see the following outputs:

Outputs on PE1:
===============
PE1#sh mpls for vrf ABC 2.2.2.2 det
Local      Outgoing   Prefix           Bytes Label   Outgoing   Next Hop    
Label      Label      or Tunnel Id     Switched      interface              
None       21         2.2.2.2/32[V]    0             Tu1        point2point 
        MAC/Encaps=14/22, MRU=1496, Label Stack{18 21}, via Et1/0
        AABBCC000301AABBCC0002018847 0001200000015000
        VPN route: ABC
        No output feature configured
PE1#
PE1#sh mpls traffic-eng tunnels tunnel 1

Name: PE1_t1                              (Tunnel1) Destination: 65.0.0.3
  Status:
    Admin: up         Oper: up     Path: valid       Signalling: connected
    path option 1, type dynamic (Basis for Setup, path weight 20)

  Config Parameters:
    Bandwidth: 0        kbps (Global)  Priority: 7  7   Affinity: 0x0/0xFFFF
    Metric Type: TE (default)
    AutoRoute: enabled  LockDown: disabled Loadshare: 0 [0] bw-based
    auto-bw: disabled
  Active Path Option Parameters:
      State: dynamic path option 1 is active
    BandwidthOverride: disabled  LockDown: disabled  Verbatim: disabled


  InLabel  :  -
  OutLabel : Ethernet1/0, 18  <<<<<<<<<<<<<<
  Next Hop : 23.23.23.2
  RSVP Signalling Info:
       Src 65.0.0.1, Dst 65.0.0.3, Tun_Id 1, Tun_Instance 8
    RSVP Path Info:
      My Address: 23.23.23.1   
      Explicit Route: 23.23.23.2 34.34.34.1 34.34.34.2 65.0.0.3 
      Record   Route:   NONE
      Tspec: ave rate=0 kbits, burst=1000 bytes, peak rate=0 kbits
    RSVP Resv Info:
      Record   Route:   NONE
      Fspec: ave rate=0 kbits, burst=1000 bytes, peak rate=0 kbits
  Shortest Unconstrained Path Info:
    Path Weight: 20 (TE)
    Explicit Route: 23.23.23.1 23.23.23.2 34.34.34.1 34.34.34.2 
                    65.0.0.3 
  History:
    Tunnel:
      Time since created: 1 hours, 21 minutes
      Time since path change: 1 hours, 18 minutes
      Number of LSP IDs (Tun_Instances) used: 8
    Current LSP: [ID: 8]
      Uptime: 1 hours, 18 minutes
PE1#

From the above outputs we see that the label stack shows two labels "{18 21}". The label 18 is the transport label which we have received through TE tunnel and the label 21 is the VPN label. Since RSVP is also a signally protocol and is also capable of generating labels, the label 18 is not the label received or generated via IGP but through RSVP. Rest all the functionality is same as the normal MPLS VPN scenario. I have shown different examples here just to show what all things we can check while we are configuring or troubleshooting MPLS VPNs.

Hope you enjoyed the MPLS VPN session.

Cheers...!!!