Let's talk about IP routing (IP forwarding) by L3 switch. Please follow us carefully as this process is the most complicated one.

Network Topology

Below is the same network topology we discussed in the previous post, except this time packet delivery is from SVR1 (1.1.1.10) to SVR4 (2.1.1.30).

1. Delivering a Packet from SVR1 to SVR4: R1, Flooding an ARP Request to the Ports that belong to the Destination VLAN, after Receiving an IP Packet

① SVR1 needs to send a packet to the destination, SVR4 with IP address of 2.1.1.30.
② Through routing table lookup, SVR1 finds out the destination is matched to the default route (0.0.0.0/0), and thus the gateway address is 1.1.1.1 (R1) and the OIF is lan1.
③ Then through ARP table lookup, it also finds out the MAC address of the gateway address 1.1.1.1 is a1 (For easier explanation, we assume a1 is listed as the MAC address in SVR1's ARP table)
④ SVR1 sends a packet consisting of the following information to the destination address 2.1.1.30:

[Ethernet Header] Destination MAC address = a1 (R1's MAC address), Source MAC address = m1 (SVR1's MAC address)
[IP Header] Destination IP address = 2.1.1.30 (SVR4's IP address), Source IP address = 1.1.1.10 (SVR1's IP address)

⑤ The line card #1 at R1 keeps the packet received through ge1/1 port in the ingress packet buffer for a while.

Source MAC Learning
⑥ The packet processor at the line card #1 checks its MAC table to see if there is an entry that matches m1, the source MAC address of the received packet, or not. In this example, there is no matching entry.
⑦ So, the processor informs the control module of the event that "m1, the source MAC address, belonging to VLAN 10 is mapped to ge1/1 port".
⑧ The control module records the received information in its MAC table (source MAC learning).
⑨ Now, the control module has to send the learned MAC address information to the line cards. Note that not all the line cards, but only those with the same VLAN need the learned MAC information. So, the control module refers to the VLAN table, and finds out that ge1/1, ge1/2 and ge2/1 ports belong to VLAN 10, and that the line cards #1 and #2 support the VLAN. Note that entries in the VLAN table are created when VLAN values are assigned through CLI.
⑩ The control module forwards the MAC information (VLAN=10, source MAC=m1, port=ge1/1) to the line cards #1 and #2 so that the same information can be copied into their MAC tables (allowing for source MAC learning).

IP Routing or Ethernet Switching
⑪ The packet processor has to decide whether to perform IP routing (delivering packet through FIB lookup) or Ethernet switching (delivering through MAC table lookup). So, it checks the destination MAC address of the received packet.

• If it is a1, the MAC address of L3 switch R1, then IP routing is performed.
• If not, Ethernet switching is performed.

In this example, IP routing is performed because the destination MAC address is a1.

IP Routing based on Destination IP address
⑫ Through FIB lookup (LPM: Longest Prefix Match) for the destination IP address 2.1.1.30, the packet processor finds out the destination of the packet is directly connected to R1, and the OIF is VLAN 20.
⑬ Then through ARP table lookup, it checks if there is an entry that matches the MAC address of the destination 2.1.1.30.
⑭ No matching entry in the ARP table. So, the processor informs the control module of ARP miss event (no MAC address for 2.1.1.30).
⑮ The control module hence checks the VLAN table to identify line cards that belong to VLAN 20. Here the line cards #1 and #2 have the ports that belong to VLAN 20.
⑯ The control module now sends an ARP request (Who has 2.1.1.30? Tell 2.1.1.1) packet to the two line cards.
⑰ Upon receiving the ARP request packet, the two cards refer to their VLAN tables and identify the ports that belong to VLAN 20.
⑱ The ARP Request packet is sent (flooded) to ge1/3 port in the line card #1, and ge2/2 and ge2/3 in the line card #2.

2. Delivering a Packet from SVR1 to SVR4: R1, Receiving an ARP Reply from SVR4

① The ARP request packet is received by SVR4, SVR5 and SVR6. SVR4, of which IP address is 2.1.1.30, responds to the request by sending an ARP reply (2.1.1.30 is at m3) packet to R1.
② Again, the received packet is kept in the ingress packet buffer for a while.
③④⑤⑥⑦ The same source MAC learning explained in the previous section above is performed. As a result, R1 now knows m3, the source MAC address that belongs to VLAN 20, is mapped to ge1/3 port.
⑧ The packet processor notices the destination MAC address of the received packet is a1, R1's MAC address, and thus IP routing should be performed.
⑨ Then, it notices the packet type (EtherType in Ethernet Header) is marked as ARP packet (EtherType = 0x0806). All ARP packets are sent to the control module, instead of being routed, if their destination MAC addresses are matched to the MAC address of the router (i.e. a1 here).
⑩ The control module, upon receiving the ARP reply, records m3, the MAC address of 2.1.1.30, in its ARP table.
⑪ It sends the learned information to the line card #1 that has forwarded the ARP packet, so that the same information can be recorded in the ARP table there as well.

3. Delivering a Packet from SVR1 to SVR4: IP Routing (Forwarding)

R1 is ready to send a packet (routing) to the destination 2.1.1.30.

① SVR1 sends a packet destined for the destination 2.1.1.30 to R1.
② R1 keeps it in the ingress packet buffer for a while.
③ The source MAC address of the received packet is m1, a learned MAC address. So, source MAC learning is skipped.
④ R1 notices the destination MAC address is set a1, which is its own MAC address. So, the packet should be routed.
⑤ The FIB lookup for the destination IP address of the received packet (2.1.1.30) finds the OIF is VLAN 20.
⑥ And the ARP table lookup finds the MAC address for 2.1.1.30 is m3.
⑦ Finally, R1 performs MAC table lookup with m3, and finds out m3 is mapped to port ge1/3.
⑧ Now it moves the packet kept in the ingress packet buffer to the egress packet buffer to deliver.
⑨ The packet is now delivered to ge1/3.

Summary