Genie's Tech Blog

Where knowledge has no dimensions

Scalable EoMPLS or EoMPLS

Hello Friends,

I am back with yet another interesting topic. Today I am going to discuss on EoMPLS. EoMPLS or Scalable EoMPLS is a L2 point-to-point service between two service instances on different systems. Like Local Connect it doesn't use any MAC learning and it's solely between 2 points. It also doesn't require any global VLAN resource (this applies to scalable EoMPLS only). Before we move further, lets understand what is EVC and Local Connect:

By using the EVC framework you can combine multiple different services from different physical ports, i.e. when using multipoint bridging (aka bridge-domains), in order to put them into the same virtual circuit. Probably the biggest advantage of the EVC framework is the ability to support multiple services per physical port. This means that under a single physical port you can have any of the following mixed together :

- 802.1q trunk

- 802.1q tunnel

- Local connect

- Scalable EoMPLS (EoMPLS xconnect)

- Multipoint Bridging (L2 bridging)

- Multipoint Bridging (VPLS, SVI-based EoMPLS)

- L3 termination

Local connect is a L2 point-to-point service between two service instances on the same system. The service instances can by under the same port (hair-pinning) or under different ports. In contrast with the traditional L2 bridging, this one doesn't use any MAC learning and it's solely between 2 points. Also Local Connect doesn't require any global VLAN resource.

In EoMPLS, you can have any VLAN tag rewrite configuration under the service instances, as long as you keep in mind the following :

a) If both sides are EVC based, then you need to have common VLAN tag rewrite configurations on both sides

b) If one side is not EVC based, then depending on whether it's a physical interface or a subinterface, you'll probably need to remove one tag from the EVC side (subinterfaces remove tags by default)

Note : By default, VC type 5 is used for EoMPLS. In case VC type 4 is negotiated and used, an additional tag will be added after the VLAN tag rewrite configuration and before the data gets EoMPLS encapsulated.

Let's consider the following topology:

 

In the above topology, we have PE1 is the 7600 and the PE2 is the 7200 router. So we are configuring service instance on the 7600 and the other side we will be configuring the xconnect under the sub-interface. Before we move further understanding the configuration of the PE devices, its important  to understand that both the PE devices should have the same MTU value configured and also should have the same VC id configured. if the VC's are not same the l2vpn circuit will not come up. The same hold for the MTU as well.

Below is the configuration for the above setup:

Config on PE1:
==============
pseudowire-class lanos
 encapsulation mpls
 interworking ethernet
!
interface Loopback0
 ip address 1.1.1.1 255.255.255.255
!
interface GigabitEthernet4/1
 mtu 9216
 no ip address
 no keepalive
 service instance 999 ethernet
  encapsulation dot1q 2511
  rewrite ingress tag pop 1 symmetric
  xconnect 2.2.2.2 999 encapsulation mpls pw-class lanos
   mtu 9216
 !
!
interface GigabitEthernet6/1
 mtu 9216
 ip address 12.12.12.1 255.255.255.252
 ip ospf network point-to-point
 mpls ip
!
router ospf 100
 router-id 1.1.1.1
 network 1.1.1.1 0.0.0.0 area 0
 network 12.12.12.1 0.0.0.0 area 0
!


Config on PE2:
==============
pseudowire-class lanos
 encapsulation mpls
 interworking ethernet
!
interface GigabitEthernet0/0
 mtu 9216
 ip address 12.12.12.2 255.255.255.252
 ip ospf network point-to-point
 media-type rj45
 speed auto
 duplex auto
 negotiation auto
 mpls ip
!
interface GigabitEthernet0/1
 mtu 9216
 no ip address
 media-type sfp
 speed 1000
 duplex auto
 negotiation auto
!
interface GigabitEthernet0/1.999
 encapsulation dot1Q 2511
 xconnect 1.1.1.1 999 encapsulation mpls pw-class lanos
  mtu 9216
!
router ospf 100
 router-id 2.2.2.2
 network 2.2.2.2 0.0.0.0 area 0
 network 12.12.12.2 0.0.0.0 area 0
!

Config on CE1:
==============
interface GigabitEthernet1/0/1
 switchport trunk encapsulation dot1q
 switchport mode trunk
!
interface Vlan2511
 ip address 192.168.10.1 255.255.255.0
! 
Config on CE2:
==============
interface GigabitEthernet1/0/1
 switchport trunk encapsulation dot1q
 switchport mode trunk
!
interface Vlan2511
 ip address 192.168.10.2 255.255.255.0
 !
 

In the above configuration we can see that we have used service instance on the 7600 side where as on the other side we have used the sub-interface configuration. Please remember that the 7200 platform doesn't support service instance configuration. This basic configuration should help us to bring up the pseudowire or the xconnect circuit up. Lets now have a look at how it looks on the router when the circuit comes up.

Outputs on PE1:
===============
PE1#sh xconnect all
Legend:    XC ST=Xconnect State  S1=Segment1 State  S2=Segment2 State
  UP=Up       DN=Down            AD=Admin Down      IA=Inactive
  SB=Standby  HS=Hot Standby     RV=Recovering      NH=No Hardware

XC ST  Segment 1                         S1 Segment 2                         S2
------+---------------------------------+--+---------------------------------+--
UP     ac   Gi4/1:2511(Eth VLAN)         UP mpls 2.2.2.2:999                  UP
PE1#

PE1#sh mpls l2 vc 999 det
Local interface: Gi4/1 up, line protocol up, Eth VLAN 2511 up
  Interworking type is Ethernet
  Destination address: 2.2.2.2, VC ID: 999, VC status: up
    Output interface: Gi6/2, imposed label stack {17}
    Preferred path: not configured  
    Default path: active
    Next hop: 12.12.12.2
  Create time: 19:39:08, last status change time: 19:38:44
  Signaling protocol: LDP, peer 12.12.12.2:0 up
    Targeted Hello: 1.1.1.1(LDP Id) -> 2.2.2.2, LDP is UP
    Status TLV support (local/remote)   : enabled/supported
      LDP route watch                   : enabled
      Label/status state machine        : established, LruRru
      Last local dataplane   status rcvd: No fault
      Last local SSS circuit status rcvd: No fault
      Last local SSS circuit status sent: No fault
      Last local  LDP TLV    status sent: No fault
      Last remote LDP TLV    status rcvd: No fault
      Last remote LDP ADJ    status rcvd: No fault
    MPLS VC labels: local 18, remote 17 
    Group ID: local 0, remote 0
    MTU: local 9216, remote 9216
    Remote interface description: 
  Sequencing: receive disabled, send disabled
  Control Word: On (configured: autosense)
  VC statistics:
    transit packet totals: receive 34928, send 17
    transit byte totals:   receive 2514988, send 1514
    transit packet drops:  receive 0, seq error 0, send 0

PE1#sh mpls ldp binding
  lib entry: 1.1.1.1/32, rev 5
        local binding:  label: imp-null
        remote binding: lsr: 12.12.12.2:0, label: 16
  lib entry: 2.2.2.2/32, rev 7
        local binding:  label: 16
        remote binding: lsr: 12.12.12.2:0, label: imp-null
  lib entry: 12.12.12.0/24, rev 2
        local binding:  label: imp-null
        remote binding: lsr: 12.12.12.2:0, label: imp-null
PE1#
PE1#show mpls ldp discovery 
 Local LDP Identifier:
    1.1.1.1:0
    Discovery Sources:
    Interfaces:
        GigabitEthernet6/2 (ldp): xmit/recv
            LDP Id: 12.12.12.2:0; no host route
    Targeted Hellos:
        1.1.1.1 -> 2.2.2.2 (ldp): active, xmit/recv
            LDP Id: 12.12.12.2:0; no host route
PE1#

The same outputs can be seen on the PE router. Now if you notice closely the above output we see that between PE1 and PE2 router, there is not just a regular LDP session but also a Targeted LDP session. Please remember that even if the PE's are not directly connected, the targeted LDP session is formed between them when we bring the MPLS pseudowire. You can also verify that if the pseudowire is working fine or not using the "ping mpls pseudowire" ping method. This is one way of checking if the traffic on the pseudowire works fine or not.

PE1#ping mpls pseudowire 2.2.2.2 999
%Total number of MS-PW segments is less than segment number; Adjusting the segment number to 1
Sending 5, 100-byte MPLS Echos to 2.2.2.2, 
     timeout is 2 seconds, send interval is 0 msec:

Codes: '!' - success, 'Q' - request not sent, '.' - timeout,
  'L' - labeled output interface, 'B' - unlabeled output interface, 
  'D' - DS Map mismatch, 'F' - no FEC mapping, 'f' - FEC mismatch,
  'M' - malformed request, 'm' - unsupported tlvs, 'N' - no label entry, 
  'P' - no rx intf label prot, 'p' - premature termination of LSP, 
  'R' - transit router, 'I' - unknown upstream index,
  'X' - unknown return code, 'x' - return code 0

Type escape sequence to abort.
!!!!!
Success rate is 100 percent (5/5), round-trip min/avg/max = 1/1/1 ms
PE1#

If you see the above output, it clearly shows that the ping is fine that means that the pseudowire is working fine and is capable of carrying the traffic but there may be a twist to it depending on the egree (customer facing Line card) as this ping is initiated from the RP rather than the LC.

Verifying Customer devices:

CE1#ping 192.168.10.2
Type escape sequence to abort.
Sending 5, 100-byte ICMP Echos to 192.168.10.2, timeout is 2 seconds:
!!!!!
Success rate is 100 percent (5/5), round-trip min/avg/max = 1/8/17 ms
CE1#

From the above output, its clear that the customer traffic is flowing properly.

Troubleshooting:

1. MTU Mismatch

Lets start with the troubleshooting with basic stuff like MTU mismatch. On the PE2, i change the mtu 9216 to the default value and lets see what happens

interface GigabitEthernet0/1.999
 encapsulation dot1Q 2511
 xconnect 1.1.1.1 999 encapsulation mpls pw-class lanos
  mtu 9216
end

PE2(config-if)#int gi 0/1.999
PE2(config-subif)# xconnect 1.1.1.1 999 encapsulation mpls pw-class lanos
PE2(config-subif-xconn)#no  mtu 9216
PE2(config-subif-xconn)#end
PE2#
*Jun  9 08:52:21.225: %SYS-5-CONFIG_I: Configured from console by console
PE2#
PE2#sh xconn all
Legend:    XC ST=Xconnect State  S1=Segment1 State  S2=Segment2 State
  UP=Up       DN=Down            AD=Admin Down      IA=Inactive
  SB=Standby  RV=Recovering      NH=No Hardware

XC ST  Segment 1                         S1 Segment 2                         S2
------+---------------------------------+--+---------------------------------+--
DN     ac   Gi0/1.999:2511(Eth VLAN)     UP mpls 1.1.1.1:999                  DN
PE2#

As soon as the mtu value is changed, the xconnect circuit goes down. We can see that in the "show mpls l2transport vc <vcid> detail" output as well

Output from PE1:
=============
PE1#sh mpls l2 vc 999 det
Local interface: Gi4/1 up, line protocol up, Eth VLAN 2511 up
  Interworking type is Ethernet
  Destination address: 2.2.2.2, VC ID: 999, VC status: down
    Output interface: none, imposed label stack {}
    Preferred path: not configured  
    Default path: no route
    No adjacency
  Create time: 21:08:25, last status change time: 00:02:00
  Signaling protocol: LDP, peer 12.12.12.2:0 up
    Targeted Hello: 1.1.1.1(LDP Id) -> 2.2.2.2, LDP is UP
    Status TLV support (local/remote)   : enabled/supported
      LDP route watch                   : enabled
      Label/status state machine        : remote invalid, LruRnd
      Last local dataplane   status rcvd: No fault
      Last local SSS circuit status rcvd: No fault
      Last local SSS circuit status sent: DOWN PW(rx/tx faults)
      Last local  LDP TLV    status sent: No fault
      Last remote LDP TLV    status rcvd: No fault
      Last remote LDP ADJ    status rcvd: No fault
    MPLS VC labels: local 18, remote 17 
    Group ID: local 0, remote 0
    MTU: local 9216, remote 1500
    Remote interface description: 
  Sequencing: receive disabled, send disabled
  Control Word: On (configured: autosense)
  VC statistics:
    transit packet totals: receive 37542, send 24
    transit byte totals:   receive 2703446, send 2320
    transit packet drops:  receive 0, seq error 0, send 0

PE1#

Output from PE2:
============
PE2#sh mpls l2 vc 999 det
Local interface: Gi0/1.999 up, line protocol up, Eth VLAN 2511 up
  MPLS VC type is Eth VLAN, interworking type is Ethernet
  Destination address: 1.1.1.1, VC ID: 999, VC status: down
    Output interface: none, imposed label stack {}
    Preferred path: not configured  
    Default path: no route
    No adjacency
  Create time: 00:01:45, last status change time: 00:01:45
  Signaling protocol: LDP, peer 1.1.1.1:0 up
    Targeted Hello: 12.12.12.2(LDP Id) -> 1.1.1.1
    Status TLV support (local/remote)   : enabled/supported
      Label/status state machine        : remote invalid, LruRru
      Last local dataplane   status rcvd: no fault
      Last local SSS circuit status rcvd: no fault
      Last local SSS circuit status sent: PW DOWN(rx,tx faults)
      Last local  LDP TLV    status sent: no fault
      Last remote LDP TLV    status rcvd: no fault
    MPLS VC labels: local 17, remote 18 
    Group ID: local 0, remote 0
    MTU: local 1500, remote 9216
    Remote interface description: 
  Sequencing: receive disabled, send disabled
  VC statistics:
    packet totals: receive 0, send 0
    byte totals:   receive 0, send 0
    packet drops:  receive 0, seq error 0, send 0

PE2#

As we see that with the mismatch of the MTU, though the interface and line protocol is up, the VC is in down state.

 

2. Verifying hardware programming on RSP720 and ES+ linecard on Cisco 7600 platform:

Sometimes on the 7600 platform, we might see issue with the traffic forwarding on the xconnect circuit even though the circuit is up. This may probably be seen when the hardware is wrong programming with mpls labels or other registers. Lets have a look what we need to check in case we see such issues.

PE1#sho mpls for lab 18 det    
Local      Outgoing   Prefix           Bytes Label   Outgoing   Next Hop    
Label      Label      or Tunnel Id     Switched      interface              
18         No Label   l2ckt(999)       4511052       Gi4/1      point2point 
        MAC/Encaps=0/0, MRU=0, Label Stack{}
        No output feature configured
PE1#show mls cef mpls lab 18 det

Codes: M - mask entry, V - value entry, A - adjacency index, P - FIB Priority
       D - FIB Don't short-cut, m - mod-num, E - ELSP?
Format: MPLS - (b | xtag vpn pi cr mcast label1 exp1 eos1 valid2 label2 exp2 eos
2)
V(2151   ): B | 1 0    0 0 0 18      0 1 0 0       0 0 (A:147456 ,P:0,D:0,m:0 :E
:1)
M(2151   ): F | 1 FFF  0 0 1 FFFFF   0 1 0 0       0 0
PE1#show mls cef adj  
PE1#show mls cef adjacency entry 147456 det

Index: 147456  smac: a100.0003.0003, dmac: 0000.0000.0003
               mtu: 9264, vlan: 1016, dindex: 0xC0, l3rw_vld: 1
               format: MPLS, flags: 0x8600 
               label0: 0, exp: 0, ovr: 0
               label1: 0, exp: 0, ovr: 0
               label2: 3, exp: 0, ovr: 0
               op: REPLACE_LABEL2
               packets: 62668, bytes: 5640084

PE1#
PE1-dfc4>show mpls forwarding-table lab 18 det
Local      Outgoing   Prefix           Bytes Label   Outgoing   Next Hop    
Label      Label      or Tunnel Id     Switched      interface              
18         No Label   l2ckt(999)       4541508       Gi4/1      point2point 
        MAC/Encaps=0/0, MRU=0, Label Stack{}
        No output feature configured
PE1-dfc4>en       
PE1-dfc4#show mls cef mpls lab 18 det
Codes: M - mask entry, V - value entry, A - adjacency index, P - FIB Priority
       D - FIB Don't short-cut, m - mod-num, E - ELSP?
Format: MPLS - (b | xtag vpn pi cr mcast label1 exp1 eos1 valid2 label2 exp2 eos2)
V(2151   ): B | 1 0    0 0 0 18      0 1 0 0       0 0 (A:131072 ,P:0,D:0,m:0 :E:1)
M(2151   ): F | 1 FFF  0 0 1 FFFFF   0 1 0 0       0 0

PE1-dfc4#show mls cef adj entry 131072 det
Index: 131072  smac: a100.0003.0003, dmac: 0000.0000.0003
               mtu: 9264, vlan: 1016, dindex: 0xC0, l3rw_vld: 1
               format: MPLS, flags: 0x8600 
               label0: 0, exp: 0, ovr: 0
               label1: 0, exp: 0, ovr: 0
               label2: 3, exp: 0, ovr: 0
               op: REPLACE_LABEL2
               packets: 0, bytes: 0

PE1-dfc4#

PE1-dfc4#show plat atom imp-tbl 
Pseudo    Dlci    Dest    LTL   # Lbls   Remote   Local         Outgoing        
  IW    Backup    AC segment   Segment
Ckt Idx   or Vcd  Vlanid  Index  Imposed  Label    Label         Interface      
  Type     VC       ssm id    Status
-------   ------  ------  -----  -------  ------  -------  ---------------------
  ----  ------     ---------- -------
      3     999    1018  0xFFFF        1      18       16                  Gi6/2
   ETH    No           4097        UP

PE1-dfc4#show plat atom imp-tbl 
Pseudo    Dlci    Dest    LTL   # Lbls   Remote   Local         Outgoing          IW    Backup    AC segment   Segment
Ckt Idx   or Vcd  Vlanid  Index  Imposed  Label    Label         Interface        Type     VC       ssm id    Status
-------   ------  ------  -----  -------  ------  -------  ---------------------  ----  ------     ---------- -------
      3     999    1018  0xFFFF        1      18       16                  Gi6/2   ETH    No           4097        UP

PE1-dfc4#show plat atom disp-tbl 
Pseudo   Dlci or   Local         Outgoing          IW   Backup
Ckt Idx    Vcd     Label         Interface        Type   VC   
-------  -------  -------  ---------------------  ----  ------
      3     999       18                  Gi4/1   ETH    No 
PE1-dfc4#


PE1#rem comm sw test mcast ltl-info index C0    

index 0xC0 contain ports 4/1,50-64,T1,T2
PE1#


Now we need to enable service internal to look at some internal platform level commands on the 7600.

PE1#show mpls infrastructure lfd pseudowire det
PW ID: 8193 VC ID: 999, Destination address: 2.2.2.2
SSM Class: SSS HW
Imposition details:
 Label stack {18}, Output interface: Gi6/2
 Preferred path: not configured
 Control Word: enabled, Sequencing: disabled
Disposition details:
 Local label: 18
 Control Word: enabled, Sequencing: disabled
 SSS Switch: 3422552066

PE1#

PE1#show mpls for lab 16 det
Local      Outgoing   Prefix           Bytes Label   Outgoing   Next Hop    
Label      Label      or Tunnel Id     Switched      interface              
16         Pop Label  2.2.2.2/32       3630          Gi6/2      12.12.12.2  
        MAC/Encaps=14/14, MRU=1504, Label Stack{}
        CCEF48536A1B0024C4DBAB008847 
        No output feature configured
PE1#

// Please note that the label 16 is the local label that we received from the imposition table.

Outputs from LC:
===============
PE1-dfc4#show mpls infrastructure lfd pseudowire det
PW ID: 8193 VC ID: 999, Destination address: 2.2.2.2
SSM Class: SSS HW
Imposition details:
 Label stack {18}, Output interface: Gi6/2
 Preferred path: not configured
 Control Word: enabled, Sequencing: disabled
Disposition details:
 Local label: 18
 Control Word: enabled, Sequencing: disabled
 SSS Switch: 2281701378

PE1-dfc4#

PE1-dfc4#show mpls for lab 16 det
Local      Outgoing   Prefix           Bytes Label   Outgoing   Next Hop    
Label      Label      or Tunnel Id     Switched      interface              
16         Pop Label  2.2.2.2/32       3630          Gi6/2      12.12.12.2  
        MAC/Encaps=14/14, MRU=1504, Label Stack{}
        CCEF48536A1B0024C4DBAB008847 
        No output feature configured
PE1-dfc4#show mls cef mpls lab 16 det
Codes: M - mask entry, V - value entry, A - adjacency index, P - FIB Priority
       D - FIB Don't short-cut, m - mod-num, E - ELSP?
Format: MPLS - (b | xtag vpn pi cr mcast label1 exp1 eos1 valid2 label2 exp2 eos2)
V(2149   ): B | 1 0    0 0 0 16      0 1 0 0       0 0 (A:98304  ,P:0,D:0,m:0 :E:1)
M(2149   ): F | 1 FFF  0 0 1 FFFFF   0 1 0 0       0 0
V(2150   ): B | 1 0    0 0 0 16      0 0 0 0       0 0 (A:114688 ,P:0,D:0,m:0 :E:1)
M(2150   ): F | 1 FFF  0 0 1 FFFFF   0 1 0 0       0 0
PE1-dfc4#show mls cef adj entry 98304 det
Index: 98304   smac: 0024.c4db.ab00, dmac: ccef.4853.6a1b
               mtu: 1518, vlan: 1018, dindex: 0x0, l3rw_vld: 1
               format: MPLS, flags: 0x1000208408 
               label0: 0, exp: 0, ovr: 0
               label1: 0, exp: 0, ovr: 0
               label2: 0, exp: 0, ovr: 0
               op: POP
               packets: 0, bytes: 0

PE1-dfc4#show mls cef adj entry 114688 det
Index: 114688  smac: 0024.c4db.ab00, dmac: ccef.4853.6a1b
               mtu: 1518, vlan: 1018, dindex: 0x0, l3rw_vld: 1
               format: MPLS, flags: 0x1000208508 
               label0: 0, exp: 0, ovr: 0
               label1: 0, exp: 0, ovr: 0
               label2: 0, exp: 0, ovr: 0
               op: POP
               packets: 0, bytes: 0

PE1-dfc4#

PE1#
PE1#show vlan internal usage | in 1018
1018 GigabitEthernet6/2
PE1#

 If we notice the above output, we try to see that the label is programmed properly on the RP/SP as well as the DFC on the LC. SOmetimes when we see a value mismatch between these two which is basically due to wrong hardware programming, we see the traffic forwarding issue on the 7600 platform.

Hope this information helps. 

Please reach out to me in case you have any queries.

Cheers.. !!!

Comments (4) -

  • rajneesh

    6/11/2013 9:42:02 AM |

    great article.

    many thanx....

  • Genie

    6/19/2013 4:43:59 AM |

    Thanks Rajneesh

  • Guilherme

    4/1/2014 6:04:59 AM |

    very good articles on your blog, I have a problem in a EoMPLS / VPLS on a Cisco 7606.
    like to know if you could help me in throubleshooting I'm really no idea what it could be because I have done everything.

  • Genie

    4/1/2014 6:19:39 AM |

    Hello Guilherme
    This blog is just for discussing technology, we have a separate support forum where in we look at the problems that you are facing.
    Could you please post your questions at http://forums.codergenie.com
    The registration is completely free.

Comments are closed