Page 1 of 64
Netmanias White paper
Analyze Trends, Technologies and Market
NETMANIAS WHITE PAPER
Optical Ethernet Network
for startup service providers and legacy carriers
by Jangwoo Son
06/28/2001
Jangwoo Son
son@netmanias.com
www.netmanias.com
016-460-5747
NETMANIAS-SON-37Page 2 of 64
Netmanias White paper
Optical Ethernet Network
for startup service providers
and legacy carriers
Presented at KRNET 2001, Seoul (06/27/2001)Page 3 of 64
Netmanias White paper
Contents
• Metro Bottleneck: Problems are …
• Ethernet Invasion
• Optical Ethernet Network for Startup SP
• Optical Ethernet Network for Legacy
Carriers
• ConclusionPage 4 of 64
Netmanias White paper
Internet Big Picture
CO Metro POP
Metro Core
(Interoffice)
Access
(“last mile”)
Backbone
(Long-haul)
전화선, HFC, FTTX
TDM(copper, T1/T3)
SONET
CWDM/DWDM
Wireless
Free space optic
Single-E
User Contents
Customer
Premise
분산화
MDU(Apart)
MDU:
T1/T3, GE, POS, ATM
PPP,FR/HDLC(T1/T3)
xDSL
POS(S)
ATM(T/S)
MPLS
GE/10GE(T/S)
RPR(S)
Multiservice SONET
POS
ATM
MPLS
GE/10GE
RPR
Multiservice SONET
POS
ATM
MPLS
10GBE
MPλS
Area
Transport
Network
Data
Network
Enterprise
Home
MTU
Residential xDSL
CM
Ether.
T1/E1
xDSL
Ether.
Ether.
Res.: Dialup, xDSL  
CM, GE, xPON
Ether.
xDSL
Ether.
SONET
CWDM
Metro DWDM
SONET
Long-haul DWDM
LAN
MTU 구내배선:
Copper
UTP(10%)
Fiber(5%)
Edge
Backbone
(InterCity)
Metro Core
(interoffice)
Metro  POP
CO CO
Business
Residential
IP access IP aggregation IP Core IP aggregation IP access
Residential
Global Internet
or Other ISP
Business
Metro  POP
SDU
Gear:
Edge Router
ATM Switch
MPLS Router
10GBE Switch
RPR node
MSPP node
Gear:
Core Router
ATM Switch
MPLS Router
10GBE Switch
MPλS node
FLC
DSLAM
L2/3.
DSLAM
L2/3.
(FLC)
Copper/fiber(10%, MTU) fiber fiber
Metro Core
(interoffice)
In-building
In-building
Function
DSLAMPage 5 of 64
Netmanias White paper
Internet Performance  ?
Backbone Metro
CO CO
LAN
Residentia
l
Metro
- nDS0, T1, T3 Leased line
(64K, 256K, 516K, 1.544M,
2.048M, 45M)
- TDM-based  SONET
Enterprise
GE  Fast (Gigabit)
backbone
IP/DWDM, TSR
OADM, OXC, OPS
Residential  Fast (Gigabit/Terabit)
Cable, xDSL, and
Broadband Wireless
Fast (Mbps/user)
Contents
User User
Global Internet
or Other ISP
Metro  
POP
Metro  
POPPage 6 of 64
Netmanias White paper
MAN service
Backbone Metro
CO CO
LAN
Residential
Metro
Contents
(localization)
Web server, Co-location,
ASP, CSP, DBSP
Global Internet
or Other ISP
Metro  
POP
Metro  
POP
- nDS0, T1, T3 Leased line
(64K, 256K, 516K, 1.544M,
2.048M, 45M)
- TDM-based  SONET
LAN-to-LAN
interconnection
Internet Access
xSP AccessPage 7 of 64
Netmanias White paper
,  MAN ?
• 현재, Business Internet access, private line은 SONET/SDH 기반 전송
망을 이용하는 데, 이 SONET/SDH망의 문제는
– Circuit switching/TDM, BW allocation flexibility, Expensive
– Upgrade ?, Long lead time
OC3/12
access
ring
OC3/12
access
ring
OC12/48
Metro Core Ring
OC3/12
access
ring OC12/48
Metro Core Ring
OC3/12
access
ring OC3/12
access
ring
OC12/48
Metro Core Ring
OC3/12
access
ring
OC3/12
access
ring
Long-haul DWDM backbone
(OC48/192)
T1
T3
ISP
POP
Co-location
CO
CO
CO
CO
CO
CO
CO CO
POP
POP
Internet
B-to-B
512K
OC3/12
access
ring
OC3/12
access
ring
OC12/48
Metro Core Ring
OC3/12
access
ring OC12/48
Metro Core Ring
OC3/12
access
ring OC3/12
access
ring
OC12/48
Metro Core Ring
OC3/12
access
ring
OC3/12
access
ring
Long-haul DWDM backbone
(OC48/192)
T1
T3
ISP
POP
Co-location
CO
CO
CO
CO
CO
CO
CO CO
POP
POP
Internet
B-to-B
512KPage 8 of 64
Netmanias White paper
Trends
TDM-based SONET
Ethernet over dark fiber
Ethernet & TDM over SONET
Ethernet over SONET
Ethernet over MPLS
IP
Ethernet
Pkt muxing
TDM muxing
Voice
SONET framing
λ
IP
PPP/
HDLC/FR
TDM muxing (VT)
Voice
SONET framing
λ
IP
Ethernet/VLAN
Dark fiber/λ
IP
Ethernet
MPLS
Ethernet/VLAN
Dark fiber/λ
IP
Ethernet
GFP, LAPS, PPP/HDLC, SDL, BCP
SONET framing
λ
Appian
Lucent
Native networks
Cisco
Redback
Ciena
Riverstone
Extreme
Foundry
Cisco
Nortel
RPR
IP
RPR
TDM muxing
Voice
SONET framing
λ
EtherNet Phy
Metro Optical Ethernet
Next Generation SONETPage 9 of 64
Netmanias White paper
Ethernet vs. SONET
• Scalable in increments from  
1M bit/sec to 1G bit/sec and
soon to 10G bit/sec.
• Lower equipment costs.
• Versatile — can support IP  
services.
• QoS issues exist.
• Ethernet is not optimized for
voice traffic.
• Is not quickly restored in the
event of a failure.
Ethernet
• Survivable — 50 msec
restoration.
• 99.999% reliability.
• Optimized for voice traffic.
• Billions of dollars worth of
deployed infrastructure.
• Expensive to deploy.
• Is not proficient in carrying
data traffic.
SONETPage 10 of 64
Netmanias White paper
Ethernet Invasion
• LAN = Ethernet
• Customer Premise and Metro optical
Ethernet access
– Legacy expensive Router with TDM
WAN interface(T1/T3)
– CSU/DCU
– SONET  ADM
• Metro Core
• National-wide ConsiderationsPage 11 of 64
Netmanias White paper
Ethernet Invasion
LAN
Metro Core
LAN
LAN
LAN
LAN
LAN
WAN
Metro access
Metro access
Metro Core
LAN
Metro Core
LAN
LAN
LAN
LAN
LAN
WAN
Metro access
Metro access
Metro Core
LAN
Metro Core
LAN
LAN
LAN
LAN
LAN
WAN
Metro access
Metro access
Metro Core
LAN
Metro Core
LAN
LAN
LAN
LAN
LAN
WAN
Metro access
Metro access
Metro Core
Ethernet
WDMPage 12 of 64
Netmanias White paper
Metro Optical Ethernet
• Ethernet over Dark Fiber
• Startup Ethernet Service Provider
• Concept
• Evolution
• ESPs : Telseon, Yipes, Intellispace,…
• Vendors : Riverstone, Extreme, Foundry,
Cisco, Nortel,…Page 13 of 64
Netmanias White paper
Metro Optical Ethernet
• Metro Optical Ethernet
– No ATM and/or SONET gear required !
– Optical IP/Ethernet-based network (Fiber to the MTU/Customer)
– Ethernet packet-switching Network (L2/L3)
– Rate-limiting/guaranteeing service: 1M,2M,…,100M,110,…1000Mbps
– Dynamic, Real-time, web-based and self-BW provisioning
– Cost-effective (망구축 및 운영 비용)
– Low service charge: $1000/100Mbps/Month (Cogent)
– Target service: LAN-interconnection, Internet access, xSP access
• Advantages
– Cheap
– Easy to maintain and operate
– Statistical multiplexing (cf. SONET)
– Seamless Internetworking (No protocol conversion)
– Scalability (10GbE)
– Fast/Dynamic Provisioning
– BW Granularity(++1M)Page 14 of 64
Netmanias White paper
Metro optical Ethernet: Evolution
Native Ethernet
(LAN)
• No BW guarantee(CL)
The 1
st
Generation
Metro Optical Ethernet
The 2nd
Generation
Metro Optical Ethernet
• No CoS/QoS
• Resiliency
STP (30sec)
• P-to-P, Mesh topologies
• Short range Optics
1000BaseLX (5Km)
• Geographic limitation
Ethernet LAN
• BW guarantee(CO)
= Ingress rate-limiting (UPC)
+ 802.1q VLAN (Connection)
+ 802.1p priority queueing
• Switch architecture
H/W-based wire-speed L2/L3
forwarding, queueing and scheduling
• CoS/QoS
Diff-serv to 802.1p mapping
• Resiliency/Restoration
RSTP/EAPS (1sec)
• Ring topology
• Long range optics (70~150Km)
BGIC/CWDM
• Geographic limitation
Ethernet MAN
• 1 Gigabit Ethernet/4GbE
• Real-time provisioning
• Usage-based billing
• End-to-end BW guarantee(CO)
= Ingress rate-limiting (UPC)
+ MPLS LSP
(Per-LSP BW reservation)
• Switch architecture
H/W-based wire-speed MPLS
forwarding, queueing and scheduling
• MPLS CoS/QoS
802.1p/TOS/Diff-serv/port #
to Exp/CoS or LSP
• Resiliency/Restoration
MPLS fast reroute (100msec)
• Ring: RPR (802.17)
- SRP
- Restoration (<50msec)
- Fair ring-access
- Statistical BW sharing
• Long range optics
BGIC/CWDM/DWDM
• Geographic limitation
Ethernet WAN
• 10 Gigabit Ethernet/40GbE
• Real-time/Web-based/
Self-provisioning
• VoIP
• Usage-based billing
2000
2001-Page 15 of 64
Netmanias White paper
Concept of BW guarantee
of VLAN-based Ethernet VPN (TLS)
1 7 A B
A B
A B
802.1p/q
End-to-end connection: 802.1q VLAN
Priority queueing: 802.1p
Multi-Field Packet Classification
(L2/L3/L4 header)
⇒Ingress rate limiting/guaranteeing
(ex. 20Mbps ingress rate)
MFPC (L2/L3/L4 header)
⇒802.1p mapping
8B 6B 6B 2B 1B 4B 4B 1B 2B 2B 4B
Ethernet(L2) IP (L3) TDP/UDP (L4)
Pre
dst
MAC
src
MAC
Type
0x0800
CRC
dst
IP
Protocol
ID
src port
number
User
data
TOS
src
IP
dst port
number
8B 6B 6B 2B 1B 4B 4B 1B 2B 2B 4B
Pre
dst
MAC
src
MAC
Type
0x0800
CRC
dst
IP
Protocol
ID
src port
number
User
data
TOS
src
IP
dst port
number
802.1
p/q tag
2B
Type
0x8100
2BPage 16 of 64
Netmanias White paper
End-to-end Connection: 802.1q
MTU
VLAN ID 7
VLAN ID 5
7 A B
A B
C D
5 C D
802.1q tag (VLAN ID)
7 A B
A B
C D
Forwarding based-on {dst MAC addr. + VLAN ID}
5 C D
•One VLAN per customer
•VLAN across MAN
- Statically pre-configured, or
- Automatically provisioned
using GARP/GVRP
MAC   VLAN ID  port
B          7             x
D          5             y
7 A B
5 C D
MTU
MTU
CPE
CPE
CPE
CPE
CPE
CPE
CPE
MAC   VLAN ID  port
B          7             a
D          5             a
MAC   VLAN ID  port
B          7             c
D          5             c
MAC   VLAN ID  port
B          7            z
MAC   VLAN ID  port
B          7            2
2
c
MAC   VLAN ID  port
D          5             k
a
x
kPage 17 of 64
Netmanias White paper
Packet Classification
Control packets
Customer A + Voice flows
Customer B + FTP flows
Customer A + E-mail flows
Multi-Field Packet
Classification
Policy priority
queues
8B 6B 6B 2B 1B 4B 4B 1B 2B 2B 4B
Ethernet(L2) IP (L3) TDP/UDP (L4)
Pre
dst
MAC
src
MAC
Type
0x0800
CRC
dst
IP
Protocol
ID
src port
number
User
data
TOS
src
IP
dst port
number
Packet FilterPage 18 of 64
Netmanias White paper
Multi-Field Flow Classification
MAC
IP
0800
IPX
8138
RARP
8035
ARP
0806
VLAN Tag
1800
UDP
17
TCP
06
RSVP
46
IGMP
02
ICMP
01
Telnet
23
HTTP
80
FTP
20
SMTP
25
DNS
53
RTP
5004
! #
8B 6B 6B 2B 1B 4B 4B 1B 2B 2B 4B
Ethernet(L2) IP (L3) TDP/UDP (L4)
Pre
dst
MAC
src
MAC
Type
0x0800
CRC
dst
IP
Protocol
ID
src port
number
User
data
TOS
src
IP
dst port
number
#Page 19 of 64
Netmanias White paper
Ingress Rate Limiting
Physical port speed 100Mbps
Up
5 Mbps
5 Mbps
Down
Customer
A
5 Mbps
15 Mbps
Customer
B
15 Mbps
15 Mbps
Customer
C
MAN port
GbE
PRL (Port Rate Limiting)
Customer에게 물리적인 port FE GE port를 주고 customer가 사
BW를 임의로 임의로 조절해줄 조절해줄 수수 있는있는 기능 (H/W: rate limiting
H/W로 구현함으로써 performance degradation이 없음을 없음을 강조)Page 20 of 64
Netmanias White paper
Ingress Rate Limiting
ARL (Aggregate Rate Limiting)
customer로부터 온( customer로 갈) aggregate traffic rate을 제한( )
customer의 트래픽 트래픽 중중 특정특정 응용의 aggregate traffic rate을 제한( ) ( ,
A로부터 유입되는 VoIP aggregate traffic= 1Mbps로 제한/ )
Packet Filter
10
Mbps
1Mbps
( )
Traffics from
Customer A
( 15Mbps) Voice aggregate traffic
Shaped Traffics from
Medialincs
(Total: 10Mbps
VoIP: 1Mbps)
8B 6B 6B 2B 1B 4B 4B 1B 2B 2B 4B
Ethernet(L2) IP (L3) TDP/UDP (L4)
Pre
dst
MAC
src
MAC
Type
0x0800
CRC
dst
IP
Protocol
ID
src port
number
User
data
TOS
src
IP
dst port
numberPage 21 of 64
Netmanias White paper
Mapping to 802.1p
8B 6B 6B 2B 1B 4B 4B 1B 2B 2B 4B
Ethernet(L2) IP (L3) TDP/UDP (L4)
Pre
dst
MAC
src
MAC
Type
0x0800
CRC
dst
IP
Protocol
ID
src port
number
User
data
TOS
src
IP
dst port
number
8B 6B 6B 2B 1B 4B 4B 1B 2B 2B 4B
Pre
dst
MAC
src
MAC
Type
0x0800
CRC
dst
IP
Protocol
ID
src port
number
User
data
TOS
src
IP
dst port
number
802.1
p/q tag
2B
Type
0x8100
2B
Policy
TPI
0x8100
Priority CFI VLAN ID
16 b 3 b 1 b 12 b
Port srcMAC dstMAC TOS srcIP dstIP srcPort# dstPort# VLAN ID priority
#1 mac_a mac_b 7 165.132/16 165.154/16 x y 7 1
802.1p/q header
Classification
At Ingress Point,Page 22 of 64
Netmanias White paper
Priority queueing and scheduling
MTU
VLAN ID 7
VLAN ID 5
A B
CD
802.1q tag (VLAN ID)
A B
CD
MTU
MTU
CPE
CPE
CPE
CPE
CPE
CPE
CPE
1 7 A B
802.1p tag (priority)
3 5 CD
1 7 A B
1 7 A B
3 5 CD
3 5 CD
Priority
Field
체크
Highest Priority
Lowest Priority
Priority Field(802.p bits)
1 7 A B
3 5 CD
1 7 A B
3 5 CD 3 5 CD
1 7 A BPage 23 of 64
Netmanias White paper
Concept of BW guarantee
of VLAN-based Ethernet VPN (TLS)
1 7 A B
A B
A B
802.1p/q
End-to-end connection: 802.1q VLAN
Priority queueing: 802.1p
Multi-Field Packet Classification
(L2/L3/L4 header)
⇒Ingress rate limiting/guaranteeing
(ex. 20Mbps ingress rate)
MFPC (L2/L3/L4 header)
⇒802.1p mapping (L4 bridging)
8B 6B 6B 2B 1B 4B 4B 1B 2B 2B 4B
Ethernet(L2) IP (L3) TDP/UDP (L4)
Pre
dst
MAC
src
MAC
Type
0x0800
CRC
dst
IP
Protocol
ID
src port
number
User
data
TOS
src
IP
dst port
number
8B 6B 6B 2B 1B 4B 4B 1B 2B 2B 4B
Pre
dst
MAC
src
MAC
Type
0x0800
CRC
dst
IP
Protocol
ID
src port
number
User
data
TOS
src
IP
dst port
number
802.1
p/q tag
2B
Type
0x8100
2BPage 24 of 64
Netmanias White paper
L4 prioritization
VLAN ID 7
VLAN ID 5
A B
CD
Ethernet frame
MTU Last mile Metro core MTU Last mile
802.1p/q Network
Lookup: {dst MAC addr. + VLAN ID}
Queueing: check 802.1p tag
Output link scheduling:
Priority scheduling based on 802.1p tag
1 7 A B
2 5 CD
A B
CD
1 7 A B
2 5 CD
1 7 A B
2 5 CD
1 7 A B
2 5 CD
Classification:
port , L2 MAC addr., IP addr., TOS/DSCP,
PID, TCP/UDP Port number
Policy/ACL
Rate-limiting
802.1q VLAN tagging
802.1p priority marking based-on TOS/DSCP, Port#
1 7 A B
2 5 CD
802.1q (VLAN ID)
802.1p (priority)
8B 6B 6B 2B 1B 4B 4B 1B 2B 2B 4B
Ethernet(L2) IP (L3) TDP/UDP (L4)
Pre
dst
MAC
src
MAC
Type
0x0800
CRC
dst
IP
Protocol
ID
src port
number
User
data
TOS
src
IP
dst port
number
8B 6B 6B 2B 1B 4B 4B 1B 2B 2B 4B
Pre
dst
MAC
src
MAC
Type
0x0800
CRC
dst
IP
Protocol
ID
src port
number
User
data
TOS
src
IP
dst port
number
802.1
p/q tag
2B
Type
0x8100
2B
A lication A lication A lication A lication 서비스 차별화 가능능Page 25 of 64
Netmanias White paper
Problem of VLAN-based TLS service
• VLAN space limitation: 4096 VLANs (VLAN
ID=12bits)
• MAC address limitation
• No Bandwidth reservation in metro core (end-toend)
• No traffic engineering in metro core (STP Path)
• Slow restoration time(STP, RSTP, EAPS)Page 26 of 64
Netmanias White paper
VLAN space limitation ~ 4096
802.1q
MTU
MTU
MTU
MTU
MTU
MTU MTU
MTU
MTU
MTU
MTU
MTU
802.1q
802.1q 802.1q
802.1q
802.1q
802.1q
802.1q
VLAN 1
VLAN 4096Page 27 of 64
Netmanias White paper
Supported MAC address limitation
802.1q
MTU
MTU
MTU
MTU
MTU
MTU MTU
MTU
MTU
MTU
MTU
MTU
802.1q
802.1q 802.1q
802.1q
802.1q
802.1q
802.1q
Metro Core Ethernet Switches must learn
MAC addresses of
1) Customer router, if  CPE = L3
2) Individual machine, if CPE = L2Page 28 of 64
Netmanias White paper
No Traffic Engineering
802.1q
MTU
MTU
MTU
MTU
MTU
MTU MTU
MTU
MTU
MTU
MTU
MTU
802.1q
802.1q 802.1q
802.1q
802.1q
802.1q
802.1q
Not Traffic-Engineered Path
(Path determined by STP)
VLAN 100’s PathPage 29 of 64
Netmanias White paper
No end-to-end BW guarantee
802.1q
MTU
MTU
MTU
MTU
MTU
MTU MTU
MTU
MTU
MTU
MTU
MTU
802.1q
802.1q 802.1q
802.1q
802.1q
802.1q
802.1q
No BW reservation for VLAN’s connection in Metro Core
No BW guarantee in Metro Core
Only provide priority queueing (802.1p)
VLAN 100’s PathPage 30 of 64
Netmanias White paper
No End-to-end BW reservation
BW shared by 3 VLAN traffics:
No BW guarantee in metro core trunk link
Priority queueing: 802.1p
200M
500M
500M
700M
500M
500M
1200M
VLAN A
VLAN B
VLAN C
Ingress rate limitingPage 31 of 64
Netmanias White paper
MPLS in Metro Optical Ethernet Network
MTU
MTU
MTU
MTU
MTU
MTU MTU
MTU
MTU
MTU
MTU
MTU
802.1q
802.1q 802.1q
802.1q
802.1q
802.1q
802.1q
LER
LER
MPLS
LSR
LSR
MPLS tunnelPage 32 of 64
Netmanias White paper
Ethernet over MPLS: Martini draft
• Martini Internet draft:
– Encapsulation Methods for Transport of Layer 2 Frames Over MPLS
(draft-martini-l2circuit-encap-mpls-01.txt )
– Transport of Layer 2 Frames Over MPLS
(draft-martini-l2circuit-trans-mpls-05.txt)
• Layer 2 PDU (frame relay, ATM AAL5, Ethernet)를 MPLS Network
을 통해 전달하는 절차(Encapsulation, Level distribution)를 기술
• No IP routing in Provider’s Network (LER/PE and LSR/P)
• Service Provider provides L2 connectivity only
• MPLS-based L2 circuit
• Metro Ethernet에 Martini draft를 적용
Ethernet over MPLS (MPLS over Ethernet) network
(Tunnel LSP, VC LSP, Encapsulation)Page 33 of 64
Netmanias White paper
Tunnel LSP
MTU
LER, R1
LSR
LER, R2
LSR
Suppose it is desired to transport layer 2 PDUs from
ingress LSR R1 to egress LSR R2, across an
intervening MPLS network. We assume that there is an
LSP from R1 to R2. That is, we assume that R1 can
cause a packet to be delivered to R2 by pushing some
label onto the packet and sending the result to one of its
adjacencies. Call this label the \"tunnel label\", and the
corresponding LSP the \"tunnel LSP\".
tunnel LSP 1
tunnel LSP: LER(PE)간에 설정
- LER로 들어오는 들어오는 목적지 LER이 같은
VLAN traffic
Aggregation  
tunnel LSP 2
tunnel LSP 3Page 34 of 64
Netmanias White paper
VC LSP
MTU
LER, R1
LSR
LER, R2
LSR
VLAN
100
VLAN
100
VLAN
50
VLAN
50
VLAN
200 VLAN
200
Egress LER, R2
Must know how to treat the received frame
=> Define “VC label (VC LSP)”
R2 can infer form the VC label
the outgoing interface and perhaps the VLAN ID
DA SA 0x8847 S2 S1 DA SA E-type L3 PDU
VC LSP
VC LSP = per VLAN LSP
- Egress LER에서 각각의 VLAN
Label값만 보고보고 알알 수수 있게있게 해준다.Page 35 of 64
Netmanias White paper
Frame flow over MPLS Network
MTU
LER
LSR
LER
LSR
MAC
A
1
2
3
4
1
2
3
4
1
2
3
4
1
2
3
4
MAC
BPage 36 of 64
Netmanias White paper
MTU Access Switch
Ethernet Type
0x0800: IPv4
0x0806: arp
0x8035: rarp
0x8100: vlan
IPX
IPv6
Rfc1700,
LER
LSR
LER
LSR
MAC
A 1
2
3
4
1
2
3
4
1
2
3
4
1
2
3
4
MAC
B
MTU
MAC  VLAN ID  outgoing port
B         100 3
L3 PDU
(IP packet)
DA
B
SA
A
E-type
(0x0800)
1 2
3
802.1p/q
(2/100)
DA
B
SA
A
E-type
(0x8100)
E-type
(0x0800)
L3 PDU
(IP packet)
VLAN taggingPage 37 of 64
Netmanias White paper
802.1p/q
(2/100)
DA
B
SA
A
E-type
(0x8100)
E-type
(0x0800)
L3 PDU
(IP packet)
Ingress LER
LER
LSR
LER
LSR
MAC
A 1
2
3
4
1
2
3
4
1
2
3
4
1
2
3
4
MAC
B
MTU
1 2
3
MAC  VLAN ID  outgoing port    Tunnel Label VC label Label
A         100 4              5       8
1) VLAN-to-LSP mapping (Label push)
- Tunnel LSP: aggregation LSP
- VC LSP: per-VLAN LSP
2) CoS(Ext) field marking
Classification (802.1p/MFPC)
3) Outer MAC header (MACs of LSRs)
Outer
Ethernet
Header
802.1p/q
(2/100)
DA
B
SA
A
E-type
(0x8100)
E-type
(0x0800)
L3 PDU
(IP packet)
DA
b
SA
a
E-type
(0x8847)
Tunnel
label(5)
VC
label(8)
Label stack Original 802.1q frame
a
bPage 38 of 64
Netmanias White paper
LSP
LER
LSR
LER
LSR
MAC
A 1
2
3
4
1
2
3
4
1
2
3
4
1
2
3
4
MAC
B
MTU
1 2
3
802.1p/q
(2/100)
DA
B
SA
A
E-type
(0x8100)
E-type
(0x0800)
L3 PDU
(IP packet)
DA
d
SA
c
E-type
(0x8847)
Tunnel
label (30)
VC
label (8)
802.1p/q
(2/100)
DA
B
SA
A
E-type
(0x8100)
E-type
(0x0800)
L3 PDU
(IP packet)
DA
b
SA
a
E-type
(0x8847)
Tunnel
label (5)
VC
label (8)
1) Label Swapping: Tunnel Label
2) Rewrite MAC header
Incoming port  Tunnel Label outgoing port  Tunnel Label
1          5              3           30
c
dPage 39 of 64
Netmanias White paper
LSP
LER
LSR
LER
LSR
MAC
A 1
2
3
4
1
2
3
4
1
2
3
4
1
2
3
4
MAC
B
MTU
1 2
3
802.1p/q
(2/100)
DA
B
SA
A
E-type
(0x8100)
E-type
(0x0800)
L3 PDU
(IP packet)
DA
f
SA
e
E-type
(0x8847)
VC
label (8)
802.1p/q
(2/100)
DA
B
SA
A
E-type
(0x8100)
E-type
(0x0800)
L3 PDU
(IP packet)
DA
d
SA
c
E-type
(0x8847)
Tunnel
label (5)
VC
label (8)
1) Pop Tunnel label
2) Rewrite MAC header
Incoming port  Tunnel Label outgoing port
1          5              3
e
fPage 40 of 64
Netmanias White paper
Egress LER
LER
LSR
LER
LSR
MAC
A 1
2
3
4
1
2
3
4
1
2
3
4
1
2
3
4
MAC
B
MTU
1 2
3
802.1p/q
(2/100)
DA
B
SA
A
E-type
(0x8100)
E-type
(0x0800)
L3 PDU
(IP packet)
802.1p/q
(2/100)
DA
B
SA
A
E-type
(0x8100)
E-type
(0x0800)
L3 PDU
(IP packet)
DA
f
SA
e
E-type
(0x8847)
VC
label (8)
1) Pop VC label
2) Infer from VC label how to process
the original frame (output port)
Incoming port   VC Label outgoing port
1          8                          4
e
f
Original 802.1q framePage 41 of 64
Netmanias White paper
MTU Distribution Switch
LER
LSR
LER
LSR
MAC
A 1
2
3
4
1
2
3
4
1
2
3
4
1
2
3
4
MAC
B
MTU
1 2
3
DA
B
SA
A
E-type
(0x0800)
L3 PDU
(IP packet)
802.1p/q
(2/100)
DA
B
SA
A
E-type
(0x8100)
E-type
(0x0800)
L3 PDU
(IP packet)
e
f
MAC  VLAN ID  outgoing port
B         100 1
1 2
3Page 42 of 64
Netmanias White paper
Good of MPLS
• VLAN  space 문제 해결 (VLAN ID Reuse)
• Metro Core에서 MAC주소수 문제 해결
• Traffic Engineering in Metro Core
• Protection: MPLS Fast Reroute (<100msec)
• MPLS QoS: LSP별로 차별화 또는 Ext/CoS비
트
• MPLS-based Layer 2 VPN
• Scalability
• SecurityPage 43 of 64
Netmanias White paper
VLAN ID Reuse
Egress LER VLAN ID overwrite하지 않는않는 경우
MTU
MTU
MTU
MTU
MTU
MTU MTU
MTU
MTU
MTU
MTU
MTU
LER
LER
MPLS
LSR
LSR
MPLS tunnel
12
13
10
11
VLAN ID
10
11
14
15
16
17
12
13
14
15
10
11
12
13
14
15 16
17
VLAN ID Reuse
Reuse not allowedPage 44 of 64
Netmanias White paper
MAC address learning
MTU
MTU
MTU
MTU
MTU
MTU MTU
MTU
MTU
MTU
MTU
MTU
802.1q
802.1q 802.1q
802.1q
802.1q
802.1q
LER 802.1q
MPLS
MAC address learning
Metro core customer MAC address VLAN ID가 노출되지
.
, MAC address learning할 필요가 필요가 없다.
LER LER에 연결된 연결된 지역의 customer MAC learning하면 된다.
LER
MAC address
learning %Page 45 of 64
Netmanias White paper
Traffic Engineering in Metro Core
MTU
MTU
MTU
MTU
MTU
MTU MTU
MTU
MTU
MTU
MTU
MTU
802.1q
802.1q 802.1q
802.1q
802.1q
802.1q
802.1q
LER
LER
MPLS
LSR
MAC address learning LSRPage 46 of 64
Netmanias White paper
Good of MPLS
MTU
LER/PE
LSR/P
LER/PE
LSR/P
VLAN space 문제 해결 (VLAN ID Reuse)
Metro Core , 학습해야 할
MAC address수 문제문제 해결
VLAN
VLAN
VLAN
MPLS
• Traffic Engineering
in Metro Core
• MPLS restoration
- pre-established standby LSP
- Fast  Reroute
• MPLS QoS
LSP1
LSP2 LSP3Page 47 of 64
Netmanias White paper
Ethernet over MPLS Solutions
• Riverstone
• Extreme
• Foundry
• Cisco
• NortelPage 48 of 64
Netmanias White paper
Ethernet over SONET
• ESP가 inter-city Ethernet VPN service를 제공
하려는 경우
• Metro 사업자가 legacy SONET infra를 가지고
있는 경우
• Protocol conversion없이 Ethernet connectivity
를 제공
• Eliminate extensive IP Address Administration
• Eliminate IP hop to reduce latency
• Ethernet over SONET using
PPP(rfc1638/2878): Foundry, Extreme,
RiverstonePage 49 of 64
Netmanias White paper
Ethernet over SONET
Metro/Wide Area
SONET backbone
Ethernet
Network
Customer 1
(VLAN 100)
Customer 2
(VLAN 200)
Ethernet
Network
Customer 1
(VLAN 100)
Customer 2
(VLAN 200)
Ethernet
Network
Customer 1
(VLAN 100)
Customer 2
(VLAN 200)
Point-to-point
SONET Circuit
POS interface
Ethernet
IP Routing NetworkPage 50 of 64
Netmanias White paper
Ethernet switching Network
Ethernet over SONET
Metro/Wide Area
SONET backbone
Ethernet
Network
Customer 1
(VLAN 100)
Customer 2
(VLAN 200)
Ethernet
Network
Customer 1
(VLAN 100)
Customer 2
(VLAN 200)
Ethernet
Network
Customer 1
(VLAN 100)
Customer 2
(VLAN 200)
Point-to-point
SONET Circuit
EOS interface Page 51 of 64
Netmanias White paper
Next Generation SONET
Ethernet & TDM over SONET
IP
Ethernet
Pkt muxing
TDM muxing
Voice
SONET framing
λ
Appian OSAP4800
Lucent DMX
Native networks
Cisco ONS15454
Redback SE800
Ciena K2
SONET guy data interface(Ethernet)
SONET gear에 흡수하고 statistical multiplexing
지원함으로써 기존의 TDM service와 더불어
IP data service를 지원함.Page 52 of 64
Netmanias White paper
Cisco ONS 15454
ONS15454
= SONET ADM + DCS +
Ethernet switch + ATM switch
+ DWDM OADM
Customer Interface
- 10/100, GbE
- TDM, ATM, OC-n
Network Interface
- OC12/48/192
Network-side BW
- STS-1,  STS-3c,
STS-6c, STS-12c
L2
sw
L2
sw
TDM
STS-n mux/demux
EoS EoS
STS-nc STS-nc
10/100
GbE
TDM
ATM
OC48/192
SONET RingPage 53 of 64
Netmanias White paper
Point-to-point circuit vs. shared circuit
ONS15454
L2/L3,
Router
10/100 or GbE
STS-1
GbE
ONS15454
L2/L3,
Router
10/100 or GbE
STS-3c
shared circuit
GbE
Dedicated point-to-point circuit Shared packet ring circuit (L2 SW)
HUB site HUB site
OC12
(OC48/OC192)
STS-3c: Shared circuit
STS-3 (Ethernet)
STS-3
STS-3
OC12
(OC48/OC192)
STS-3
STS-3
STS-3
STS-3
STS-1(52M): Ethernet
VLAN
Other service: TDM, ATM
B-A, C-A, D-A
Other service: TDM, ATM
STS-1
STS-1
VLANPage 54 of 64
Netmanias White paper
Rate-limiting
ONS15454
L3 SW,
Router
(2948G)
125M
Customer A (25M)
Customer B (100M)
Customer C (15M)
Customer D (35M)
Customer E (12M) Customer F (60M)
50M
72M
(1) Rate-limiting: per-node
(2) Rate-limiting: per-customer
HUB site
L3 SW,
Router
(2948G)
A Shared STS-6c (310Mbps) circuitPage 55 of 64
Netmanias White paper
Lucent solution
POP
Core router
Ethernet
E1
Transport Network
(SONET and/or DWDM)
POP
Core router
Ethernet (10/100)
Virtual Concatenation
(Link Capacity Adjustment Scheme)
2,4,6,8,10,50, 100
E1
E1
No WAN Router and DSU/CSU
Transport Network
(SONET and/or DWDM)
GbE
GbE
POP
Core router
Ethernet GbE
Ethernet
E1
4*E1
5*E1
Statistical multiplexing
2M-100MbE
EOSPage 56 of 64
Netmanias White paper
Differences
E1
POP
Aggregation
router
E1
POP
Aggregation
router
E1
IMA or ML-PPP
Ethernet
Ingress rate-limiting
POP
Core router
Ethernet
Virtual Concatenation
(Link Capacity Adjustment Scheme)
Transport Network
(SONET and/or DWDM)
E1 E1
Transport Network
(SONET and/or DWDM)
Ethernet Transport Network
E1
E1
POP
Core router
TDM-based SONET
Ethernet over dark fiber
Ethernet over SONET
E1
E1
Transport Network
(SONET and/or DWDM)Page 57 of 64
Netmanias White paper
Appian solution
100M
100M
NG-SONET
ADM
DCS ADM
Local
Access
Metro
Core
DCS
POP
access
POP
Aggregation
router
Multi-Tenant Unit
(a) Legacy TDM leased line
(b) Next-Generation SONET approach
Local
Access
SONET circuit
STS-1 (45Mbps)
Ethernet over
SONET mapping
DCS ADM
Local
Access
Metro
Core
DCS
POP
access
POP
Aggregation
router
Multi-Tenant Unit Local
Access
SONET circuit
STS-1 (45Mbps)
T3
CSU
100M
100M
LAN
switch
WAN
router
T3
100M T3
100M
Company A
Company B
Company C
Multi-Tenant Unit
T3
CSU
LAN
switch
WAN
router
T3
T3 100M
100M
Company A
Company A
Company B
Company C
Multi-Tenant Unit
100M
100M
Company A
100M
100M
802.3x flow control
802.3x flow control
1. 전용회선 서비스Page 58 of 64
Netmanias White paper
Appian solution
CSU
ADM DCS ADM
Local
Access
Metro
Core
DCS
POP
access
256 Kbps
NG-SONET
ADM
DCS ADM
Local Access
OC3/12/48
Metro
Core
DCS
POP
access
10/100
Dedicated T1 circuit is provisioned for a customer.
TDM multiplexing at MTU legacy SONET ADM
Dedicated OC3 (STS-3c) circuit is provisioned for a MTU.
Statistical multiplexing at MTU NG-SONET ADM
STS-3c BW is shared among customers in a building.
Note: The BW cannot be shared among different buildings.
POP
Aggregation
router
POP
Aggregation
router
Multi-Tenant Unit
Multi-Tenant Unit
(a) Legacy TDM leased line
(b) Next-Generation SONET approach
512 Kbps
T1
10/100
1 GbE
Company A
Company B
Company C
Company A
Company B
Company C
가입자
가입자
Router
PBX
T1
1. MTU 인터넷 액세스 서비스Page 59 of 64
Netmanias White paper
Current typical Metro topology: Ring
• Ethernet: Ring에 취약
– Fairness, Restoration (RSTP, EAPS)
• NG SONET:
– SONET  circuit  (STS-n)을 data traffic용
과 Voice traffic용으로 구분하고 Data
traffic을 통계적 다중화함으로써 SONET
용량을 효율적으로 사용할 수 있게 해줌.
– TDM/SONET의 한계는 여전히 존재
• RPR (Resilient Packet Ring)Page 60 of 64
Netmanias White paper
RPR (Resilient Packet Ring)
• IEEE 802.17 RPRWG(dec. 2000), 표준화 완료:
2003 Q1
• Resilient Packet Ring Access Protocol for use in
LAN, MAN and WAN for transfer data packets at
rates scalable to many gigabits per second.
• RPR MAC over Ethernet phy, SONET, WDM
• Motivation
• Ethernet : Restoration, No BW allocation for
nodes on the Ring
• SONET: Static point–to-point circuit between ring
nodes  ~ inefficiency, High-costPage 61 of 64
Netmanias White paper
RPR features
• Dual counter rotating ring:
– both ring carry traffic all of the time
– no reserved protection BW.
• Media Independency:
– OC48c/OC192c SONET/SDH
– 1GbE/10GbE Ethernet
• Plug and play
• Destination-striping of variable length unicast
packets (Spatial reuse)
• Distributed BW management and congestion
control
• Protection mechanism
– < 50msec
– Warring
• COSPage 62 of 64
Netmanias White paper
RPR features
Rx
Tx
Ethernet Switch
or IP Router
RPR node
SONET
RING
Standard Ethernet frame
RPR packet
RPR
node
RPR
node
RPR
nodePage 63 of 64
Netmanias White paper
RPR Network architecture
Traditional
TDM
customer
RPR
RPR
RPR
RPR
OC3/OC48
OC12/OC192
STS-3Xc
RPR virtual Ring
EOS
EOS
Ethernet
Ethernet
Ethernet
Ethernet
Ethernet
POSPage 64 of 64
Netmanias White paper
Conclusion
• Metro Bottleneck: Problems are….
• Ethernet Invasion
• Optical Ethernet Network for Startup SP
– Ethernet Over Dark Fiber
– MPLS
• Optical Ethernet Network for Legacy Carriers
– Next-Generation SONET
– Switching vendors (EOS)
• Customer WAN port: TDM => Ethernet
• Metro Access: Optical Access(Ethernet or SONET)
• Metro Core:
– Current Solution: Ethernet over Dark Fiber or Ethernet over SONET
– Soon,… RPR
• MAN, WAN: 10 Gigabit Ethernet (LAN/WAN Phy)
• Global end-to-end Ethernet