Building a Highly Adaptive, Resilient,
and Scalable MPLS Backbone
Ning So, Verizon Business Hao-Hsin Huang, WANDL, Inc
Feb 9, 2007 MPLS World Congress 2007 Paris, France
ⓒ 2006 Verizon All Rights Reserved PT00000 00/00/06

About Verizon Business and WANDL

Verizon Business MPLS core network

MPLS TE Fast Reroute (FRR) design on VzB MPLS core
Design challenges and solutions

Solving LSP meshing scalability

Q&A
Verizon Business is the premier communications solutions provider for global businesses, government agencies, and educational institutions

Data and IP services

Security

IT solutions

Managed networks

Premises equipment

Contact centers

Conferencing

Voice
WANDL is the leading supplier of software solutions for advanced network planning, management, design, and optimization

20 years expertise in software development for network optimization, planning, design, OSS/automation

Work with Cisco, Juniper, Tellabs, Alcatel, Nortel, Lucent, Huawei, etc

Manage technologies such as IP, MPLS, VoIP, transport (SONET/SDH), FR/ATM ,TDM

Customers include carriers, ISPs, telcos, PTTs, service providers, enterprise, and government organizations
Verizon Business Private MPLS Core Network

US-based

Backbone trunks are OC48 and OC192 packet-over-SONET

Provide Layer 2 (MPLS-TE) transport services for Verizon Business’ private data networks including IP, Ethernet, and Frame Relay networks
Network Capacity
Provide sufficient backbone capacity to carry traffic from all feeder
networks
Network Resiliency
100% traffic re-route during single hardware and/or transmission facility
failure
Network Performance
Provide the minimal latency routes available between any two network
elements
Network Efficiency
Best practices in network architecture and traffic engineering techniques to optimize network routing and resource usage
Underlying transmission facilities supporting the MPLS core network are gradually shifting from a 1+1 SONET ring-based infrastructure to 1+0 linear-based infrastructure
Transmission equipment moving from conventional DWDM to Ultra Long Haul (ULH) ULH favors linear over ring infrastructures ULH has fewer optical regions compared with conventional systems, resulting in
a less efficient ring design (larger rings) and no impact on linear design ULH has a much higher availability rate, reducing the need for the ring system for transmission facility maintenance and repair work
The core network with linear transmission facility relies more on the Layer 2 reroute for service restoration
MPLS TE FRR is the only proven technology that provides service restoration at a speed comparable to ring-based restoration
FRR design can be highly complicated with implementation and management presenting bigger challenges to service providers
FRR design challenges
1
Bundled FRR design for ease of implementation and management
2
FRR bandwidth and pre-emption design
3
FRR design with built-in facility failures knowledge base
4
“Do not re-route” LSP on a FRR-enabled network environment
Description

When Juniper FRR is engineered and signaled on a per LSP basis, service providers lose the control of FRR route selection

Design and management of individual FRR quickly becomes too time consuming for service providers with a large MPLS core network (such as the Verizon Business MPLS Core network)

Utilize the node/link protection scheme
Solution
Challenge #1 Solution Illustration
NLP Discovery and Design Parameters
Auto FRR design parameters
NLP flag on LSP
Primary tunnel pathinyellow
NLP bypass tunnel in green
Challenge #1 Solution  Bypass Tunnel and
Its Protected Primary Tunnel Paths
Bypass tunnel path Protected path
All tunnels being protected by this bypass tunnel
Description
How to set up the bypass tunnel bandwidth and preemption?
Problems occur if bypass tunnel bandwidth is set up according to the primary LSP bandwidth

Additional bandwidth on the backbone has to be reserved for FRR, causing inefficient use of valuable network resources

FRR bandwidth and preemption design quickly becomes too complicated when multiple FRR paths are set up to account for multiple network failures

Multiple network element failure can cause domino effect on FRR reroute due to preemption which magnifies the problem and causes network instability

Service provider loses performance predictability due to the massive
amount of combinations and permutations of the re-route scenarios
Solution

Set bypass tunnel bandwidth to zero, effectively turning off the preemption

All traffic on the FRR route share the pain during the outages

Rely on the LSP re-signal to restore the LSP primary route, based on the new network topology

Allow queuing at the physical interface level handle the traffic discard during congestion
Alternative Solution

Create multiple bypass tunnels

Set maximum bw and subscription ratio for bypass tunnels

Load balance among all available network resources

Minimize congestion

Prevent losing all tunnels during multiple failure scenarios
Challenge #2 Alternative Solution
Multiple Bypass Tunnels

Full BW for bypass tunnel design

Low RSVP BW to conserve network resource

Set maximum protection BW for bypasstunnels

Set maximum number of bypass tunnels allowed

A possible scenario is to use multiple (4) OC48 trunks to protect a
single OC192 trunk

Load balancing is desired among 4 OC48 trunks

This scheme could preserve 75% of the traffic even when the OC192 and one OC48 are downat the same time
Description Current MPLS network does not have any knowledge of the physical layer topology, so the auto-configured FRR may be put on the same physical transmission facilities as the primary route of the LSP

Manual configuration of individual FRR, even using node/link protection scheme, can be error prone

Admin Group function is the ideal and logical choice for marking the facilities However, existing RSVP TE extension does not include Admin Group
Solution

Use fate sharing function to group trunks based on physical topology

Fate sharing allows for manual categorization of trunks

When constructing bypass tunnels, router will avoid links in the same group as the protected link

Juniper fate-sharing and Cisco Share Risk Link Group (SRLG) features are vendor-specific implementation

Create explicit bypass paths using WANDL tool to ensure facility diversity
Challenge #3 Solution
FRR Design with Facility Diversity
Primary tunnel path in yellow and NLP bypass tunnel in green  With facility diversity


W/o facility diversity

Bypass tunnel uses link inthe same facility due to CSPF
Description
How to design LSP that resists re-routing in a FRR-enabled network environment

Customers with ultra-delay sensitive applications who do not want to be placed on sub-optimal route

Customers prefer to be notified of the failure and switch to alternative network/provider
Solution

Dedicate a set of “do not re-route” LSPs

Disable the FRR at the LSP head end while maintaining thenode/link protection scheme network wide
Challenge #4 Solution
LSP Tunnels Configlet Generation
LSP withand  
Mix of LSPs with NLP  without NLP  
and without NLP  
Link-protection on  
rsvp interfaces  

Designing a large MPLS core network that converges multiple feeder networks with thousands of trunking LSPs for each feeder network

Designing and managing a network with tens of thousands of LSPs is not practical for service providers

Network performance and operator ability to troubleshoot are negatively impacted as the number of LSPs increases

Establish a full mesh of LSPs as the base LSP set in the MPLS core network

Set up feeder network logical trunking mesh as pseudowires within the base LSPs

Establish each set of PW with its own Class of Service queue for traffic management and policing

Build more than one set of base LSPs depending on
special requirements, eg, “do not re-route” LSPs
LSP Scalability illustration
Tunnels routed
thru a core link
Pseudowires riding on a LSP tunnel

One fully meshed set of LSP tunnels

Many PWs riding on a LSP tunnel

All PWs passing thru a link are protected by FRR NLP

FRR is a necessary and powerful tool in the MPLS TE-enabled network; however, it is also highly complicated to implement

MPLS TE is still a rapidly developing and evolving technology with many gaps in the current standards

Although most major service providers have deployed MPLS TE-enabled networks, the learning curve is still steep, and there are many challenges ahead

A comprehensive FRR design and simulation tool like MPLSView is needed to ensure proper BW protection and efficient use of network resources
Thank You!
Ning So  Hao-Hsin Huang  
Verizon Business  WANDL, Inc  
2400 N Glenville  88 Centennial Ave  
Richardson, Texas  Piscataway, NJ  
ningso@verizonbusinesscom  haohsin@wandlcom