Genie's Tech Blog

Where knowledge has no dimensions

Multicast Equal Cost Multipath (ECMP)

Hello Friends, 

Its been a long while that I posted something interesting on my blog. Here comes something that might interest you. Today I am going to discuss on Multicast ECMP feature. But before we discuss this feature, lets first define the problem that we are trying to solve so that we know why we need this feature.

Problem Definition:

Currently if there are multiple equal-cost paths towards a PIM source or RP, only one of them will be selected as the RPF path. PIM will send joins out of that path towards the source (of S,G routes) or RP (of *, G routes). All multicast traffic will be received on this RPF interface. Because of this a portion of the network can become very congested with multicast traffic while the rest is idle.

Multipath feature enables PIM to split multicast load among several equal cost paths and theirfore improve overall network throughput. This feature is similar to per-flow load splitting in CEF. Also, this feature is not specific to IPv4 / IPv6, of PIM mode (sparse, bidir, SSM).

 

In the above topology, R1 and R2 are the source, R3 is the RP and R6 is the where the receivers are connected. The complete setup is running OSPF as the IGP and PIM on it.

All the routers are having their loopback defined as x.x.x.x/32 where x is the host number. On the R6 loopback, Ii have joined a group 239.0.0.1 using the command "ip igmp join-group 239.0.0.1". Now, i try to ping from R1 and R2 sourcing loopback, I see the following outputs on R6

R6#sh ip mroute
IP Multicast Routing Table
Flags: D - Dense, S - Sparse, B - Bidir Group, s - SSM Group, C - Connected,
       L - Local, P - Pruned, R - RP-bit set, F - Register flag,
       T - SPT-bit set, J - Join SPT, M - MSDP created entry, E - Extranet,
       X - Proxy Join Timer Running, A - Candidate for MSDP Advertisement,
       U - URD, I - Received Source Specific Host Report, 
       Z - Multicast Tunnel, z - MDT-data group sender, 
       Y - Joined MDT-data group, y - Sending to MDT-data group, 
       V - RD & Vector, v - Vector
Outgoing interface flags: H - Hardware switched, A - Assert winner
 Timers: Uptime/Expires
 Interface state: Interface, Next-Hop or VCD, State/Mode

(*, 239.0.0.1), 00:00:55/stopped, RP 3.3.3.3, flags: SJCL
  Incoming interface: Ethernet1/0, RPF nbr 56.56.56.5
  Outgoing interface list:
    Loopback0, Forward/Sparse, 00:00:55/00:02:19

(1.1.1.1, 239.0.0.1), 00:00:03/00:02:55, flags: LJT
  Incoming interface: Ethernet1/0, RPF nbr 56.56.56.5
  Outgoing interface list:
    Loopback0, Forward/Sparse, 00:00:03/00:02:55

(13.13.13.1, 239.0.0.1), 00:00:03/00:02:55, flags: LJT
  Incoming interface: Ethernet1/0, RPF nbr 56.56.56.5
  Outgoing interface list:
    Loopback0, Forward/Sparse, 00:00:03/00:02:55

(2.2.2.2, 239.0.0.1), 00:00:13/00:02:46, flags: LJT
  Incoming interface: Ethernet1/0, RPF nbr 56.56.56.5
  Outgoing interface list:
    Loopback0, Forward/Sparse, 00:00:13/00:02:46

(23.23.23.2, 239.0.0.1), 00:00:13/00:02:46, flags: LJT
  Incoming interface: Ethernet1/0, RPF nbr 56.56.56.5
  Outgoing interface list:
    Loopback0, Forward/Sparse, 00:00:13/00:02:46

(*, 224.0.1.40), 00:05:42/00:02:23, RP 3.3.3.3, flags: SJCL
  Incoming interface: Ethernet1/0, RPF nbr 56.56.56.5
  Outgoing interface list:
    Loopback0, Forward/Sparse, 00:05:41/00:02:23

From the above output, we clearly see that the incoming interface is Ethernet1/0 and the RPF neighbor has been selected as 56.56.56.5, but there is another link Ethernet 0/0. By default, if there are multiple equal cost paths towards teh RP or a source, router will elect only one path or incoming interface (the highest IP address PIM neighbor) for multicast flow. Here in our case its Ethernet 1/0 as the RPF neighbor is 56.56.56.5 where as the other RPF neighbor is 46.46.46.4 which is having a smaller IP.

Now, when we configure "ip multicast multipath" in the global config and enabled the debugs "debug ip multicast rpf tracked" on router R6 to see the difference before and after enabling the multipath command:

R6#debug ip multicast rpf tracked 
IP multicast RPF track debugging is on
R6#
*Sep 28 01:58:15.291: IPMCAST-RPF RIB_RWATCH: Debugging is ON
R6#
R6#
*Sep 28 02:30:11.243: IPMCAST-RPF RIB_RWATCH: (default:ipv4 multicast:base) W 1.1.1.1/32 c=0x4E9A268 EVENT Track stop
*Sep 28 02:30:11.243: IPMCAST-RPF RIB_RWATCH: (default:ipv4 multicast:base) W 1.1.1.1/32 c=0x4E9A268 Removing
*Sep 28 02:30:11.243: IPMCAST-RPF RIB_RWATCH: (default:ipv4 multicast:base) R 1.1.1.1/32  d=115 p=2 -> Et1/0 56.56.56.5 40, Et0/0 46.46.46.4 40 Scheduled to check for deletion
*Sep 28 02:30:11.243: IPMCAST-RPF RIB_RWATCH: (default:ipv4 multicast:base) R 1.1.1.1/32  d=115 p=2 -> Et1/0 56.56.56.5 40, Et0/0 46.46.46.4 40 Deleting
*Sep 28 02:30:11.343: IPMCAST-RPF RIB_RWATCH: (default:ipv4 multicast:base) W 13.13.13.1/32 c=0x4E9A3E8 EVENT Track stop
R6#
*Sep 28 02:30:11.343: IPMCAST-RPF RIB_RWATCH: (default:ipv4 multicast:base) W 13.13.13.1/32 c=0x4E9A3E8 Removing
*Sep 28 02:30:11.343: IPMCAST-RPF RIB_RWATCH: (default:ipv4 multicast:base) R 13.13.13.0/24  d=115 p=2 -> Et1/0 56.56.56.5 30, Et0/0 46.46.46.4 30 Scheduled to check for deletion
*Sep 28 02:30:11.343: IPMCAST-RPF RIB_RWATCH: (default:ipv4 multicast:base) R 13.13.13.0/24  d=115 p=2 -> Et1/0 56.56.56.5 30, Et0/0 46.46.46.4 30 Deleting
R6#
*Sep 28 02:30:12.543: IPMCAST-RPF RIB_RWATCH: (default:ipv4 multicast:base) W 23.23.23.2/32 c=0x4E9A6E8 EVENT Track stop
*Sep 28 02:30:12.543: IPMCAST-RPF RIB_RWATCH: (default:ipv4 multicast:base) W 23.23.23.2/32 c=0x4E9A6E8 Removing
*Sep 28 02:30:12.543: IPMCAST-RPF RIB_RWATCH: (default:ipv4 multicast:base) R 23.23.23.0/24  d=115 p=2 -> Et1/0 56.56.56.5 30, Et0/0 46.46.46.4 30 Scheduled to check for deletion
*Sep 28 02:30:12.543: IPMCAST-RPF RIB_RWATCH: (default:ipv4 multicast:base) R 23.23.23.0/24  d=115 p=2 -> Et1/0 56.56.56.5 30, Et0/0 46.46.46.4 30 Deleting
R6#
*Sep 28 02:30:14.343: IPMCAST-RPF RIB_RWATCH: (default:ipv4 multicast:base) W 2.2.2.2/32 c=0x4E9A568 EVENT Track stop
*Sep 28 02:30:14.343: IPMCAST-RPF RIB_RWATCH: (default:ipv4 multicast:base) W 2.2.2.2/32 c=0x4E9A568 Removing
*Sep 28 02:30:14.343: IPMCAST-RPF RIB_RWATCH: (default:ipv4 multicast:base) R 2.2.2.2/32  d=115 p=2 -> Et1/0 56.56.56.5 40, Et0/0 46.46.46.4 40 Scheduled to check for deletion
*Sep 28 02:30:14.343: IPMCAST-RPF RIB_RWATCH: (default:ipv4 multicast:base) R 2.2.2.2/32  d=115 p=2 -> Et1/0 56.56.56.5 40, Et0/0 46.46.46.4 40 Deleting
R6#
R6#
R6#conf t
Enter configuration commands, one per line.  End with CNTL/Z.
R6(config)#ip multicast multipath 
R6(config)#
*Sep 28 03:03:51.787: IPMCAST-RPF RIB_RWATCH: (default:ipv4 multicast:base) W 3.3.3.3/32 c=0x4D44E30 EVENT Track stop
*Sep 28 03:03:51.787: IPMCAST-RPF RIB_RWATCH: (default:ipv4 multicast:base) W 3.3.3.3/32 c=0x4D44E30 Removing
*Sep 28 03:03:51.787: IPMCAST-RPF RIB_RWATCH: (default:ipv4 multicast:base) T 3.3.3.3/32 EVENT Track start
*Sep 28 03:03:51.787: IPMCAST-RPF RIB_RWATCH: (default:ipv4 multicast:base) N 3.3.3.3/32 Adding track
*Sep 28 03:03:51.787: IPMCAST-RPF RIB_RWATCH: Adding to client notification queue
*Sep 28 03:03:51.787: IPMCAST-RPF : Received notification from Route Watch, Sending message to process: PIM Process
*Sep 28 03:03:51.787: IPMCAST-RPF RIB_RWATCH: (default:ipv4 multicast:base) W 3.3.3.3/32 c=0x4D44FF0 EVENT Track stop
*Sep 28 03:03:51.787: IPMCAST-RPF RIB_RWATCH: (default:ipv4 multicast:base) W 3.3.3.3/32 c=0x4D44FF0 Removing
*Sep 28 03:03:51.787: IPMCAST-RPF RIB_RWATCH: (default:ipv4 multicast:base) T 3.3.3.3/32 EVENT Track start
*Sep 28 03:03:51.787: IPMCAST-RPF RIB_RWATCH: (default:ipv4 multicast:base) N 3.3.3.3/32 Adding track
*Sep 28 03:03:51.787: IPMCAST-RPF RIB_RWATCH: Adding to client notification queue
*Sep 28 03:03:51.799: IPMCAST-RPF : Processing Route Watch notification
R6(config)#end
R6#

*Sep 28 03:09:24.911: IPMCAST-RPF RIB_RWATCH: (default:ipv4 multicast:base) T 1.1.1.1/32 EVENT Track start HP
*Sep 28 03:09:24.911: IPMCAST-RPF RIB_RWATCH: (default:ipv4 multicast:base) N 1.1.1.1/32 Adding track
*Sep 28 03:09:24.911: IPMCAST-RPF RIB_RWATCH: (default:ipv4 multicast:base) N 1.1.1.1/32 QP Schedule query
*Sep 28 03:09:24.911: IPMCAST-RPF RIB_RWATCH: (default:ipv4 multicast:base) T 1.1.1.1/32 EVENT Query found route
*Sep 28 03:09:24.911: IPMCAST-RPF RIB_RWATCH: (default:ipv4 multicast:base) N 1.1.1.1/32 Adding route
*Sep 28 03:09:24.911: IPMCAST-RPF RIB_RWATCH: (default:ipv4 multicast:base) R 1.1.1.1/32  d=0 p=0 -> Updating
*Sep 28 03:09:24.911: IPMCAST-RPF RIB_RWATCH: (default:ipv4 multicast:base) R 1.1.1.1/32  d=115 p=2 -> Et1/0 56.56.56.5 40, Et0/0 46.46.46.4 40 Notifying
*Sep 28 03:09:24.911: IPMCAST-RPF RIB_RWATCH: Adding to client notification queue (HP)
*Sep 28 03:09:24.911: IPMCAST-RPF : Received notification from Route Watch, Sending message to process: PIM Process
*Sep 28 03:09:24.911: IPMCAST-RPF RIB_RWATCH: (default:ipv4 multicast:base) T 13.13.13.1/32 EVENT Track start HP
*Sep 28 03:09:24.911: IPMCAST-RPF RIB_RWATCH: (default:ipv4 multicast:base) N 13.13.13.1/32 Adding track
*Sep 28 03:09:24.911: IPMCAST-RPF RIB_RWATCH: (default:ipv4 multicast:base) N 13.13.13.1/32 QP Schedule query
*Sep 28 03:09:24.911: IPMCAST-RPF RIB_RWATCH: (default:ipv4 multicast:base) T 13.13.13.1/32 EVENT Query found route
*Sep 28 03:09:24.911: IPMCAST-RPF RIB_RWATCH: (default:ipv4 multicast:base) N 13.13.13.0/24 Adding route
*Sep 28 03:09:24.911: IPMCAST-RPF RIB_RWATCH: (default:ipv4 multicast:base) R 13.13.13.0/24  d=0 p=0 -> Updating
*Sep 28 03:09:24.911: IPMCAST-RPF RIB_RWATCH: (default:ipv4 multicast:base) R 13.13.13.0/24  d=115 p=2 -> Et1/0 56.56.56.5 30, Et0/0 46.46.46.4 30 Notifying
R6#
*Sep 28 03:09:24.911: IPMCAST-RPF RIB_RWATCH: Adding to client notification queue (HP)
*Sep 28 03:09:24.911: IPMCAST-RPF : Processing Route Watch notification
*Sep 28 03:09:24.911: IPMCAST-RPF RIB_RWATCH: (default:ipv4 multicast:base) W 1.1.1.1/32 c=0x4E9A568 Client notified reachable
*Sep 28 03:09:24.911: IPMCAST-RPF RIB_RWATCH: (default:ipv4 multicast:base) W 13.13.13.1/32 c=0x4E9A6E8 Client notified reachable
R6#
*Sep 28 03:09:28.659: IPMCAST-RPF RIB_RWATCH: (default:ipv4 multicast:base) T 23.23.23.2/32 EVENT Track start HP
*Sep 28 03:09:28.659: IPMCAST-RPF RIB_RWATCH: (default:ipv4 multicast:base) N 23.23.23.2/32 Adding track
*Sep 28 03:09:28.659: IPMCAST-RPF RIB_RWATCH: (default:ipv4 multicast:base) N 23.23.23.2/32 QP Schedule query
*Sep 28 03:09:28.659: IPMCAST-RPF RIB_RWATCH: (default:ipv4 multicast:base) T 23.23.23.2/32 EVENT Query found route
*Sep 28 03:09:28.659: IPMCAST-RPF RIB_RWATCH: (default:ipv4 multicast:base) N 23.23.23.0/24 Adding route
*Sep 28 03:09:28.659: IPMCAST-RPF RIB_RWATCH: (default:ipv4 multicast:base) R 23.23.23.0/24  d=0 p=0 -> Updating
*Sep 28 03:09:28.659: IPMCAST-RPF RIB_RWATCH: (default:ipv4 multicast:base) R 23.23.23.0/24  d=115 p=2 -> Et1/0 56.56.56.5 30, Et0/0 46.46.46.4 30 Notifying
*Sep 28 03:09:28.659: IPMCAST-RPF RIB_RWATCH: Adding to client notification queue (HP)
*Sep 28 03:09:28.659: IPMCAST-RPF : Received notification from Route Watch, Sending message to process: PIM Process
*Sep 28 03:09:28.659: IPMCAST-RPF RIB_RWATCH: (default:ipv4 multicast:base) T 2.2.2.2/32 EVENT Track start HP
*Sep 28 03:09:28.659: IPMCAST-RPF RIB_RWATCH: (default:ipv4 multicast:base) N 2.2.2.2/32 Adding track
*Sep 28 03:09:28.659: IPMCAST-RPF RIB_RWATCH: (default:ipv4 multicast:base) N 2.2.2.2/32 QP Schedule query
*Sep 28 03:09:28.659: IPMCAST-RPF RIB_RWATCH: (default:ipv4 multicast:base) T 2.2.2.2/32 EVENT Query found route
*Sep 28 03:09:28.659: IPMCAST-RPF RIB_RWATCH: (default:ipv4 multicast:base) N 2.2.2.2/32 Adding route
*Sep 28 03:09:28.659: IPMCAST-RPF RIB_RWATCH: (default:ipv4 multicast:base) R 2.2.2.2/32  d=0 p=0 -> Updating
*Sep 28 03:09:28.659: IPMCAST-RPF RIB_RWATCH: (default:ipv4 multicast:base) R 2.2.2.2/32  d=115 p=2 -> Et1/0 56.56.56.5 40, Et0/0 46.46.46.4 40 Notifying
*Sep 28 03:09:28.659: IPMCAST-RPF RIB_RWATCH: Adding to client notification queue (HP)
*Sep 28 03:09:28.659: IPMCAST-RPF : Processing Route Watch notification
R6#
*Sep 28 03:09:28.659: IPMCAST-RPF RIB_RWATCH: (default:ipv4 multicast:base) W 23.23.23.2/32 c=0x4E9A3E8 Client notified reachable
*Sep 28 03:09:28.659: IPMCAST-RPF RIB_RWATCH: (default:ipv4 multicast:base) W 2.2.2.2/32 c=0x4E9A268 Client notified reachable
R6#

In the above output, we see that before enabling the command "ip multicast multipath", the link between R4 and R6 is getting deleted as the RPF neighbor. This is the default behavior on the ECMP paths. But once we enable the command, we see that none of the RPF neighbors are getting deleted and the load is actually getting split for different sources. Also, now when we see the show ip mroute from the router R6, wee see the following difference:

R6#sh ip mroute
IP Multicast Routing Table
Flags: D - Dense, S - Sparse, B - Bidir Group, s - SSM Group, C - Connected,
       L - Local, P - Pruned, R - RP-bit set, F - Register flag,
       T - SPT-bit set, J - Join SPT, M - MSDP created entry, E - Extranet,
       X - Proxy Join Timer Running, A - Candidate for MSDP Advertisement,
       U - URD, I - Received Source Specific Host Report, 
       Z - Multicast Tunnel, z - MDT-data group sender, 
       Y - Joined MDT-data group, y - Sending to MDT-data group, 
       V - RD & Vector, v - Vector
Outgoing interface flags: H - Hardware switched, A - Assert winner
 Timers: Uptime/Expires
 Interface state: Interface, Next-Hop or VCD, State/Mode

(*, 239.0.0.1), 01:23:51/stopped, RP 3.3.3.3, flags: SJCL
  Incoming interface: Ethernet1/0, RPF nbr 56.56.56.5
  Outgoing interface list:
    Loopback0, Forward/Sparse, 01:23:50/00:02:09

(2.2.2.2, 239.0.0.1), 00:11:27/00:01:27, flags: LJT
  Incoming interface: Ethernet1/0, RPF nbr 56.56.56.5
  Outgoing interface list:
    Loopback0, Forward/Sparse, 00:11:27/00:02:09

(23.23.23.2, 239.0.0.1), 00:11:27/00:01:29, flags: LJT
  Incoming interface: Ethernet0/0, RPF nbr 46.46.46.4
  Outgoing interface list:
    Loopback0, Forward/Sparse, 00:11:27/00:02:09

(13.13.13.1, 239.0.0.1), 00:11:31/00:01:27, flags: LJT
  Incoming interface: Ethernet1/0, RPF nbr 56.56.56.5
  Outgoing interface list:
    Loopback0, Forward/Sparse, 00:11:31/00:02:09

(1.1.1.1, 239.0.0.1), 00:11:31/00:01:29, flags: LJT
  Incoming interface: Ethernet1/0, RPF nbr 56.56.56.5
  Outgoing interface list:
    Loopback0, Forward/Sparse, 00:11:31/00:02:09

(*, 224.0.1.40), 01:23:51/00:02:17, RP 3.3.3.3, flags: SJCL
  Incoming interface: Ethernet1/0, RPF nbr 56.56.56.5
  Outgoing interface list:
    Loopback0, Forward/Sparse, 01:23:50/00:02:17

R6#

We now see that for Stream coming from R1, R5 is seen as the RPF neighbor and for the stream coming from R2, R4 is seen as the RPF neighbor. We can also see the tracked multicast RPF entries:

R6#sh ip multicast rpf tracked 
RWatch output: Paths=16
Table default:ipv4:base (type=1, size=4):
  Neighbours:
    46.46.46.4       Et0/0      5
    56.56.56.5       Et1/0      5
  Tree nodes:
Table default:ipv4 multicast:base (type=1, size=4):
  Neighbours:
  Tree nodes:
    1.1.1.1/32
      R pdb: 258, distance: 115 paths: 2
        -> (ipv4:base) Et1/0 56.56.56.5 metric: 40
        -> (ipv4:base) Et0/0 46.46.46.4 metric: 40
      W (-> 1.1.1.1/32):
        C 0x4E9A568
    2.2.2.2/32
      R pdb: 258, distance: 115 paths: 2
        -> (ipv4:base) Et1/0 56.56.56.5 metric: 40
        -> (ipv4:base) Et0/0 46.46.46.4 metric: 40
      W (-> 2.2.2.2/32):
        C 0x4E9A268
    3.3.3.3/32
      R pdb: 258, distance: 115 paths: 2
        -> (ipv4:base) Et1/0 56.56.56.5 metric: 30
        -> (ipv4:base) Et0/0 46.46.46.4 metric: 30
      W (-> 3.3.3.3/32):
        C 0x4D44FF0
        C 0x4D44E30
    13.13.13.0/24
      R pdb: 258, distance: 115 paths: 2
        -> (ipv4:base) Et1/0 56.56.56.5 metric: 30
        -> (ipv4:base) Et0/0 46.46.46.4 metric: 30
    13.13.13.1/32
      W (-> 13.13.13.0/24):
        C 0x4E9A6E8
    23.23.23.0/24
      R pdb: 258, distance: 115 paths: 2
        -> (ipv4:base) Et1/0 56.56.56.5 metric: 30
        -> (ipv4:base) Et0/0 46.46.46.4 metric: 30
    23.23.23.2/32
      W (-> 23.23.23.0/24):
        C 0x4E9A3E8
R6#

Hope all has now clarified on how "ip multicast multipath" works.

Feel free to ask any questions on this one.

Cheers...!!!

Comments (1) -

  • Yap Chin Hoong

    10/30/2013 5:05:17 PM |

    ECMP should be Equal-Cost Multipath, and has nothing to do with multicast. :p

Comments are closed