Carrier Ethernet Transport in Metro and Core Networks

Tutorial by Claus G. Gruber and Achim
Autenrieth Nokia Siemens Networks

13th International Telecommunications Network Strategy and Planning
Symposium - .Convergence in Progress” Networks 2008

September 28 . October 2, 2008
Budapest, Hungary ⓒ Nokia Siemens Networks

About Us
Dr.-Ing. Claus G. Gruber
Claus Gruber is senior consultant and project manager at Nokia Siemens Networks, Munich,
Germany. Division: Research Technology and Platforms, Network Technology, Network Control and
Transport (RTP NT NCT). His main area of research focuses on next generation packet network
architectures including Carrier Grade Ethernet and IP/MPLS over WDM. He is mainly interested in
networking concepts, total cost of ownership, multilayer traffic engineering and resilience, control
plane, and network management and configuration of ubiquitous communication technologies.
Prior to his work at Nokia Siemens Networks he was a member of the research and teaching staff at
Technische Universitat Munchen (TUM), Germany, where he received his Dr.-Ing. and Dipl.-Ing.
degree in electrical engineering and information technology.
Claus published about 30 articles in journals and conference proceedings and submitted about 20
invention reports in the area of routing, resilience, network planning, optimization and management
that are currently under review at EU and US patent offices.

Dr.-Ing. Achim Autenrieth
Achim Autenrieth is Head of IP Transport R&D Management Innovation (IPT RD Innovation) at Nokia
Siemens Networks, Munich, Germany. Focus areas of his work are multilayer transport networks
(OTN/DWDM, SDH/SONET, Ethernet/MPLS-TP, IP/MPLS), control plane protocols (ASON/GMPLS),

network architecture evaluation, multilayer resilience and multilayer network design, routing and
grooming. Prior to his current responsibility he was working as project manager and senior research scientist in

internal innovation projects and funded research projects at Siemens AG, Corporate Technology and
Siemens AG, Fixed Networks.
Achim studied Electrical Engineering and Information Technology at the Technische Universitat
Munchen (TUM) and received his Dipl.-Ing. and Dr.-Ing. degree in 1996 and 2003, respectively. From
1996 to 2003 he was member of the research and teaching staff at the Institute of Communication
Networks at TUM.

General Information
. Schedule
. 9:00 . 10:30 Tutorial Part I
. Q&A
. After each main section
. 10:30 . 11:00 Coffee Break
. 11:00 . 12:30 Tutorial Part II

. To ensure proper knowledge transfer to the audience, some basic behavior rules should be strictly obeyed during the tutorial

Contents
1. Introduction
2. Operator Requirements for Transport Networks
3. Ethernet Basics
4. Carrier Ethernet Evolution
5. Carrier Ethernet Transport Technologies
6. Carrier Ethernet Transport Network Architecture & Solution
7. Outlook Towards Future Internet Architectures
8. Conclusion

Networks get run over by a huge traffic growth innovation is a must on the way forward

. The fastest and most cost efficient access technologies are not sufficient on their own 5 billion people . Huge traffic volumes have to connected be transported throughout the network
. Data super highways and an optimized end-to-end transport are needed to connect 5bn people

Challenges and Opportunities
Reinventing the Add value beyond bit-pipe connected world
User service 100x traffic experience growth

5 Bn people connected
Environmental Performance
Internet for the next billion

Tomorrow\'s communication world

5 Bn People connected
4 Bn mobile users
2 Bn fixed broadband users
Source: Nokia Siemens Networks estimations based external forecasts (Ovum, Strategy Analytics)
xDSL
FTTx
cable
0.2 Bn
0.4 Bn
0.6 Bn
0.8 Bn
fixed
WiMAX
5 Bn
4 Bn
3 Bn
2 Bn
2 Bn
Mobile Users Worldwide
Fixed Broadband Subscriptions* Worldwide
Main growth in mobile subscriptions from new growth markets

Majority can be always online via mobile high-speed Internet access technologies

Wireline Broadband will facilitate usage of applications like TV and/or video streaming.

2015 2005 2010
Voice and high-speed Internet enabled (EDGE, HSPA, ... , LTE, WiMAX)
Voice and low speed Internet enabled
2005 2010 2015

* Broadband subscriptions are typically shared by 2-3 people

Broadband services drive transport network evolution

Cost of data transport must go down
.Optical Metro
.Rural connectivity
.Photonic core Operators invest into the whole network
Enable next generation of connectivity
Transport investment worldwide
Consumers
Quality of life for citizens
Business
Growth and efficiency
Government
Productivity
Source: Connectivity Scorecard
. Higher network efficiency
.. One technology
. Leased Line OPEX
.. Profitable self built Revenues
Traffic
Voice
Dominant
Data
Dominant
2007 2011
CAGR
7,7%
Broadband enabled network
Demand for fixed broadband will increase over the next years

Total Broadband Access Market World total
6.4%
5.0%
Fiber access
6,2
5,9
5,6
5,3
5,0
DSLAM
Narrowband
2006 2007 2008 2009 2010

. In the year 2012, there will be more than . DSL is the dominant broadband market and will 500 million Broadband subscribers worldwide
remain
. Most subscribers will use a DSL connection
. Driven by high bandwidth demand, fiber based
. Fiber access subscription is expected to grow in access revenue will double in the next 10 years line with IPTV subscription . Narrowband revenue will decrease

2006 2008 2010 2012
100
200
300
400
500
Million 600
subscriptions
(world)
Fiber to the building/home subscription
DSL Subscriptions
IPTV/VoD Subscriptions
Cablemodem Subscriptions
Total Broadband Subscriptions
Source: internal research based on several analyst forecasts
1,1 0,9 0,9 0,8
2,9 2,8 2,9 3 3,1
1,0 1,5 1,8 2,1
2,3
1
Source: internal research based on several analyst forecasts 5 billion people connected
“100x traffic growth within 5 years” means a growing need for scalable networks
Growing # of customers
Business
New services at lower cost
Growing # of services
Consumer
Consumer

Multimedia services drive bandwidth requirement

Triple Play
services require
bandwidth from
25 to 100 Mbit/s
per user!
IPTV: 20-30 Mbps
(multiroom HDTV, VoD)
Internet: 5-10 Mbps
VoIP: 0.1 Mbps
Source: Internet research, 2006

Increasing bandwidth demands require a simplified and more efficient infrastructure
Operators go Ethernet Technology goes highest scalability and flexibility

Level 3:
Up to 100Gbit/s channels “Ethernet is becoming a in the core preferred enabler for leading applications, e.g. Internet, Flexible Gigabit services Source: Conferences; Lightreading 2007 Content delivery, utility multi-Gigabit wavelengtservices, IP video, …”

switching Ethernet switching @ all FT, Telefonica: transport technologie “IP does not scale enough, Microwave Radio, NG SDH Ethernet is an alternative”

DWDM, Carrier Etherne

The broadband telecommunication environment is enabled by next generation connectivity

Megabit applications - Gigabit services Broadband access everywhere Reliable and secure traffic control Flexible bandwidths from access to core
Optimized connectivity in fixed and mobile environment Solutions to balance networks and ensure Quality of Service Carrier Ethernet Transport

Contents
1. Introduction
2. Operator Requirements for Transport Networks
3. Ethernet Basics
4. Carrier Ethernet Evolution
5. Carrier Ethernet Transport Technologies
6. Carrier Ethernet Transport Network Architecture & Solution
7. Outlook Towards Future Internet Architectures
8. Conclusion

What is “Carrier Ethernet Transport” ?

In a sentence . Ethernet with Carrier Grade qualities for Transport Networks But seriously…
. Taking the simple, well known and widely deployed Ethernet service and extending it to the metro and core of public networks thus maintaining the simplicity, flexibility and cost effectiveness of the protocol and components on an end-to-end basis

Carrier Ethernet Transport technology is defined by six key attributes

Resiliency
End to End Ethernet
. Connection Oriented Ethernet
. Seamless Ethernet across portfolio of . 50ms protection IP Transport/Nokia Siemens Network . Resilient IP (ResIP)
. Differentiated service creation certification

Optimized Deployment

Scalability
. Scalable architecture with . Standardized platforms end to end portfolio . Prove worldwide
. Technology agnostic multi- deployment layer optimization
. Over 20,000 service and support personnel

Simple Management
. Automation of network
. Point and click provisioning
. Standard Operation an

Maintenanc
Flexible Solutions

. Integrated Solution for Mobile Backhaul, Business and residential services
. Shared best practices

Carrier Ethernet Transport . Defined
. Architecture Goals and Building Blocks Connection Oriented Packet Based Service Transparent Deterministic Controlled Enable IP Services over a Converged Carrier Class Transport Architecture Add Scalability, Resiliency, and Manageability to Ethernet Multi-Service Convergence Static Managed
Isolated Secure Predictable Protected Guaranteed SLA Point-and-Click Provisioning Carrier Grade OAM High Reliability Stratum Quality Sync Hard QoS High Scalability Integrated TDM

Carrier Ethernet Transport . Defined
. Fundamental Requirements
Unified Architecture for Cost-Effective
Transport of High-Speed Packet Services
Connection
Oriented
. Provisioned
. Deterministic
. Predictable
L3 Service
Transparency
Guaranteed
SLA’s
Carrier Class
Resiliency
Multi-Layer
Service
Management
. L2 Client
Encapsulation
. Secure
Transport
. L3 Proxy
. Provisioned
. Strict QoS
. Connection
Admission
Control
. NE Quality
. SW Stability
. Network
Protection
. End-to-End
. Pt-and-Click
. Control Plane
. Robust OAM
. Reporting
Ethernet
Economics
. Scalable
. Multi-Service
. Single UNI
. Synchronous
. Cost-Effective

Contents
1. Introduction
2. Operator Requirements for Transport Network
3. Ethernet Basics
4. Carrier Ethernet Evolution
5. Carrier Ethernet Transport Technologies
6. Carrier Ethernet Transport Network Architecture & Solution
7. Outlook Towards Future Internet Architectures
8. Conclusion

Contents
1. Introduction
2. Operator Requirements for Transport Network
3. Ethernet Basics
. Network Basics
4. Carrier Ethernet Evolution
5. Carrier Ethernet Transport Technologies
6. Carrier Ethernet Transport Network Architecture & Solution
7. Outlook Towards Future Internet Architectures
8. Conclusion

Going Back to Where It Began
. We have to go back to 1984

Network Hierarchy Concept

The OSI Reference Model
Layer
Layer
n-1 Layer
n
n+1
The concept of layers

. It is a simple and efficient way of communication Provides services to higher layers with standardized interfaces
Uses services of lower layers with standardized interfaces

Network Hierarchy Concept
The OSI Reference Model

Application
Presentation
Session
Transport
Network
Data Link
1 Physical
2
3
4
5
6
7
The OSI reference model provides:
. Standardized interfaces (compatibility, interoperability and competition)
. Simplifies network technology development considerably (just trust and use the functionality of the lower layer)

Why seven layers?
. Is an often discussed question (e.g. “Three layer approach of Future Internet projects)

Network Hierarchy Concept
The OSI Reference Model
What we call application

e.g. email client such as Thunderbird
Real 8
Application 9
The User Application
Presentation 5 Session
6
7
Network service part of applications
Provides network services to applications
(e.g. protocols to applications such as snmp)
Data presentation
Presents data in the right format to the application layer
(includes encryption, reformating, restructuring of data)

Interapplication communication
Maintains sessions between applications

Transport
Network
Data Link
1 Physical
2
3
4
End-to-end connection

Ensures data transport reliability, information flow (includes maintaining of virtual circuits between hosts)

Data delivery
Provides routes between two host systems (might be at different locations)
(includes network discovery and routing decision)

Access to media
Defines the data format and how the access to the media is controlled (includes bit-error correction)

Binary transmission on a physical link
Electrical, mechanical, procedural, and functional specification

Network Hierarchy Concept
Data Encapsulation
Application
Presentation
Session
Transport
Network
Data Link
1 Physical
2
3
4
5
6
7
Header Header Header Header Header Header
Header Header Header Header Header
Header Header Header Header
Header Header Header
Header Header
Header Header Data
Header Data
Header Data
Header Data
Header Data
Header Data
Header Data

Network Hierarchy Concept


Communication

G
F
E
D
C
B1
1 A1
2
3
4
5
6
7 G
F
E
D
C
B2
1 A2
2
3
4
5
6
7
C
B1
1 A1
2
3
B2
A2
Only instances
of the same layer
can talk to each other!
Intermediate
System (IS)
End System 1 End System 2

Sample OSI Layer Protocols and Service
OSI Layer End System Transit System End System Equipment
Specification
Protocols Services
7
Application
6
Presentation
5
Session
4
Transport
3
Network
2
Data Link
1
Physical
Packets
Packets
Messages / Data
Datagram
Datagram
Packets
Gateway
Gateway
Gateway
Gateway
Router
Bridge
Switch
Transceiver
Repeater
Hub, Cable
Frames
Frames
Bits
Bits
Information unit Information unit

Network Hierarchy
According to OSI Reference Model
NSN Location
Munich
NSN Location
Espoo
Routers are used to connect networks
Switches are used to connect hosts
Backbone Network A
Backbone Network B

Fixed Transport Network Structure
IP Fixed Access
Aggregation Core
Edge
Services
Residential
Layer 2 VPN,
Ethernet /TDM
Leased
Line
Business
Layer 1
Optical/
Wavelength
Leased Line
Business
Voice,
Video,
HSI
L3 VPN
CES
CLS
MSAN
L2 switch
COS
CIS
CIS
OTN/DWDM
Metro
BRAS
Carrier Ethernet /
SDH/SONET
Optical
Transport
Routing
Applications
IP/MPLS
Core
OTN/DW
DM Core
Server
IMS VoIP, VoD, IPTV,…
L2
Transport
Carrier Ethernet /
SDH/SONET
HSI: High Speed Internet CIS: Customer IP service CES: Customer Ethernet Service COS: Customer Optical Service
MSAN: Multiservice access node (PON, DSLAM) CLS: Customer Legacy Services

Contents
1. Introduction
2. Operator Requirements for Transport Network
3. Ethernet Basics
. Ethernet Standards
4. Carrier Ethernet Evolution
5. Carrier Ethernet Transport Technologies
6. Carrier Ethernet Transport Network Architecture & Solution
7. Outlook Towards Future Internet Architectures
8. Conclusion

The original Ethernet by Bob Metcalf

Bob Metcalf, 1973
The original format for Ethernet was developed in Xerox Palo Alto Research Centre (PARC), California in 1972
and called Alto Aloha. Using Carrier Sense Multiple Access with Collision Detection (CSMA/CD) it had a
transmission rate of 2.94Mb/s and could support 256 devices over cable stretching for 1km. The two inventors
were Robert Metcalf and David Boggs

Advantages of Packet and Ethernet Networks

. Packet
. Almost 100% of traffic generated by applications is packet based
. Multiplex gain
. Control plane often deployed in combination with packet services (restoration)
. Advantages of Ethernet
. Widely deployed
. The standard for LAN equipment (10M, 100M, 1G, 10G, 100G)) available in almost every computing device
. Chipsets are very cheap and high numbers
. Plug and play
. Very simple technology to operate
. Combines data link layer and switching layer
. Drawbacks of Ethernet:
. MAC addressing scheme
. Different protocols (STP, RSTP, MSTP)
. Limited traffic-engineering and slow failure recovery
. Operation Administration and Maintenance

IEEE 802 Standards
. IEEE 802.1 . Architecture, management, switching
. 802.1D MAC layer bridges
. 802.1Q Virtual LANs
. 802.1p Quality-of-Service & Multicast support
. 802.1d Spanning Tree Protocol (STP)
. 802.1s/w Multiple STP / Rapid STP
. IEEE 802.3 . CSMA/CD (Ethernet) standards
. 802.3u Fast Ethernet (100Base-TX, 100Base-FX)
. 802.3x Full-duplex Ethernet over LAN
. 802.3z Gigabit Ethernet over fiber (1000Base-X)
. 802.3ab Gigabit Ethernet over copper (1000Base-T)
. 802.3ad Aggregation of multiple link segments (LAG)

Ethernet Basics
IEEE 802.3 Ethernet Interfaces

Older Ethernet Implementations:
10 Base 5 “yellow cable” / 10 Base 2 “cheapernet” Typical Implementation: Busses / Segments Disadvantage: R Collision
multiply when data loa Increase

Application
Current Implementations with electrical Interfaces: Presentation
10 Base T
100 Base T “Fast Ethernet” Session
1000 Base T “Gigabit Ethernet” Transport

Current Implementations with optical Interfaces:
Network
Typical Implementation: 100 Base FX “Fast Ethernet” Point-to-Point
Data Link Advantage: 1000 Base SX “Gigabit Ethernet”

Collisions can be minimized 1000 Base LX with a switch Physical
10 Gigabit-Ethernet In optical Ethernets, Collision detection is not possible

Ethernet Basics
IEEE 802.3 Ethernet Frames and MAC Addressing

MAC-Address: (Media Access Control)
Address on Layer 2 most commonly used on Ethernet, 6 Bytes long,
linked to Hardware, worldwide unique

Ethernet Frame

Data Link
Network
Transport
Session
Presentation
Application
Physical
Destination
MAC
Source
MAC
Type
Field Data of Layers 3 to 7 Check
sum

6 Bytes 6 Bytes 2 By up to 1500 Bytes 4 Bytes
The Type Field: specifies, which Layer 3 Protocol is contained
The Checksum (CRC) secures both addresses, type field and data
Minimum length 64 bytes, maximum length 1518 bytes

MAC-Broadcast addresses all stations on a LAN (Address = ff:ff:ff:ff:ff:ff)
MAC-Multicast addresses all stations with a particular property
e.g. all switches supporting a particular protocol

Ethernet’s timeline
10 000
1 000
100
10
1
1970 1975 1980 1985 1990 1995 2000 2005

Ethernet Basics
Ethernet Switching (1)

A B C D E F
1 2 3 4 5 6
Address C F
MAC- Port 1 Port 2 Port 3 Port 4 Port 5 Port 6
Table
A .. C
C ?
A
MAC-Learning

Ethernet Basics
Ethernet Switching (2)

A B C D E F
1 2 3 4 5 6
Address C F
MAC- Port 1 Port 2 Port 3 Port 4 Port 5 Port 6
Table
A .. C
D ?
A
F .. D
Flooding

Ethernet Basics
802.1d . Spanning Tree (1)

Path 1 (working)
active links

blocked links

In Ethernet networks loops are strictly forbidden because otherwise broadcast storms would
bring down the network performance. With Spanning tree protocol loops are avoided in an
Ethernet network: All links that would built up a loop are blocked by the Switches. So STP can
be used for protection: If the working link fails, the protection link (i.e. a blocked link) is activated.

Ethernet Basics
802.1d . Spanning Tree (2)
Path 1 (broken)
Path 2 (unblocked)

If the working link fails, the protection link (i.e. a blocked link) is activated. RSTP (Rapid
spanning tree protocoll) improves the switching time from several seconds to approximately
one second.

802.1w . Rapid Spanning Tree Protocol (RSTP
. Spanning Tree was designed for Enterprise. Recovery Time is not acceptable for Carrier Grade.
. Rapid Spanning Tree Protocol is identical to STP, except:
.STP . Learns the backup route after failure
.RSTP . Learns the backup route before failure
. The convergence time is significantly shortened: Timing STP RSTP Worst Case ~60s 1s

802.1s . Multiple Spanning Tree Protocol (MSTP
. MSTP enables the use of SW 1 SW 3 different paths for different
VLAN 20
VLANs (or groups of VLANs)

. Traffic can be organized to use all possible links, optimising
VLAN 10 traffic distribution

. If a link fails, only the MSTIs (MSTP Instances . individual trees) using that link are affected Advantages
. MSTP only works together with
. Efficient VLAN Paths RSTP (e.g. SW 1 => SW 4)
. Up to 32+1 instances per node . Load-sharing SW 2 SW 4

Ethernet Basics
802.1Q . VLAN support

IEEE 802.3 Frame without VLAN Tag Header

Destination
address
Source
address
Type /
Length Data CRC

IEEE 802.3 with 802.1Q 4-Byte VLAN Tag Header
Destination
address
Source
address
Type/
8100 Data CRC
4 bytes
TPID
TAG Protocol Identifier
TCI
Tag Control Identifier
2 bytes 2 bytes

TAG Protocol Identifier TPID
0x8100 Priority
C
F
I VLAN ID

16 bit 3 bit 1 bit 12 bit

802.1Q Highlights
Customer separation by VLAN
VLAN Functionality Highlights

. Up to 4096 VLAN
. Priority 802.1p associated with VLAN
. VLAN-based priority take precedence
. Allows Spanning Tree per VLAN
. Allows overlapping VLANs

VLAN Advantages
. Better security
. Solve the broadcast problem
. Solve the physical location issue

Physical view
Logical view
S S
R S
R

Ethernet Basics
Ethernet VLANs (1)
A B C D E F
A .. D
B .. D
A
VLAN 1 B D E
VLAN 2 C E F
MAC- Port 1 Port 2 Port 3 Port 4 Port 5 Port 6
Table
1 2 3 4 5 6
?

Ethernet Basics
Ethernet VLANs (2)
A B C D E F
1 2 3 4 5 6
VLAN 1 B D E
VLAN 2 C E F
MAC- Port 1 Port 2 Port 3 Port 4 Port 5 Port 6
Table
A
X
A .. D
B .. D
..

Contents
1. Introduction
2. Operator Requirements for Transport Network
3. Ethernet Basics
4. Carrier Ethernet Evolution
5. Carrier Ethernet Transport Technologies
6. Carrier Ethernet Transport Network Architecture & Solution
7. Outlook Towards Future Internet Architectures
8. Conclusion

Evolution of Ethernet Hierarchy
802.1D 802.1Q 802.1ad 802.1ah

Customer MAC
VLAN
SA: Source MAC Address
DA: Destination MAC Address
VID: VLAN ID
C-VID: Customer VID
S-VID: Service VID
VID: VLAN ID
B-SA: Backbone SA
B-DA: Backbone DA
B-VID: Backbone VID
B-TAG: a Provider Bridge S-TAG
I-SID: 24 bit Service ID
Networks 2008 - Carrier Ethernet Transport in Metro and Core Networks

I-TAG: allocated for 802.1Q service instance
B-VID VLAN identifies per destination alternate path
B-DA MAC identifies destination node
B-SA MAC identifies source node
Q-in-Q
Mac-in-Mac
Service ID
Backbone VID
Backbone MAC
Provider
Backbone
Bridges
PBB
PBB-TE
Provider
Bridges
VLAN XC:
based on
VLAN ID
DA
Pay
load
S-VID
B-DA
B-VID
DA
SA
Pay
load
S-VID
C-VID
SA
C-VID
B-SA
I-SID
Standard

Contains IP packet
Ethernet Frame
“Inner” VLAN ID acc.
“Outer” VLAN ID IEE 802.3
802.1ad Provider Bridge (Q-in-Q)

The Concept
. Adding another layer of 802.1Q
. The purpose - expanding the VLAN space by tagging the tagged packets
. The expanded VLAN space allows the service provider to provide certain services, such as
Internet access on specific VLANs for specific customers, and yet still allows the service
provider to provide other types of services for their other customers on other VLANs.
Destination Source Type /
address address Length Data CRC

Destination
address
Source
address C-VLAN
Type /
Length Data CRC

Frame with double VLAN tag header 802.1ad

Destination
address
Source
address S-VLAN C-VLAN
Type /
Length Data CRC

Support of 4K S-VLAN x 4K C-VLAN = theoretical 16 Mill VLAN
Transport of Ethernet Services
Issues with Flat Ethernet Architecture
Transport Network Transport Network
C-DA
S-TAG C-TAG

802.1ad
Frame C-DA C-SA TP
ID
SVID
TPI
D
SVID
L/T User Data FCS

6 octets 6 octets 2 2 2 2 2 46 . 1500 octets 4 octets

. Full transparency ?
. Use of client information as forwarding decision ?
. Learning of all client MAC addresses in all transport nodes ?
. Known
isues with STP
issues with STP (resilience and traffic engineering)