Metro Ethernet

A Metro Ethernet is a computer network that covers a metropolitan area and that is based on the Ethernet standard. It is commonly used as a metropolitan access network to connect subscribers and businesses to a larger service network or the Internet. Businesses can also use Metro Ethernet to connect branch offices to their Intranet.

Ethernet has been a well known technology for decades. An Ethernet interface is much less expensive than a SONET/SDH or PDH interface of the same bandwidth. Ethernet also supports high bandwidths with fine granularity,^{[clarification needed]} which is not available with traditional SDH connections. Another distinct advantage of an Ethernet-based access network is that it can be easily connected to the customer network, due to the prevalent use of Ethernet in corporate and, more recently, residential networks. Therefore, bringing Ethernet in to the Metropolitan Area Network (MAN) introduces a lot of advantages to both the service provider and the customer (corporate and residential).^{[citation needed]}

Metro Ethernet system

A typical service provider Metro Ethernet network is a collection of Layer 2 or/and Layer 3 switches or/and routers connected through optical fiber. The topology could be a ring, hub-and-spoke (star), or full or partial mesh. The network will also have a hierarchy: core, distribution (aggregation) and access. The core in most cases is an existing IP/MPLS backbone, but may migrate to newer forms of Ethernet Transport in the form of 10Gbit/s, 40Gbit/s or 100Gbit/s speeds.

Ethernet on the MAN can be used as pure Ethernet, Ethernet over SDH, Ethernet over MPLS or Ethernet over DWDM. Pure Ethernet-based deployments are cheap but less reliable and scalable, and thus are usually limited to small scale or experimental deployments. SDH-based deployments are useful when there is an existing SDH infrastructure already in place, its main shortcoming being the loss of flexibility in bandwidth management due to the rigid hierarchy imposed by the SDH network. MPLS based deployments are costly but highly reliable and scalable, and are typically used by large service providers.

Metro Area Network topology

Familiar network domains are likely to exist regardless of the transport technology chosen to implement Metro Area Networks: Access, Aggregation/Distribution, Metro, and Core.^[1]

• Access devices normally exist at a customer's premises, unit, or wireless base station. This is the network that connects customer equipment, and may include ONT and/or Residential gateway, or office router.
• Aggregation occurs on a distribution network such as an ODN segment. Often Passive Optical Network or Digital Subscriber Line technologies are employed, but some using point-to-point Ethernet over "home-run" direct fibre. This part of the network includes nodes such as Multi Tenanted Unit switches, Optical line terminals in an outside plant or central office cabinet, Ethernet in the First Mile equipment, or provider bridges.
• A Metro Area Network may include transport technologies MPLS, PBB-TE and T-MPLS, each with its own resiliency and management solutions.
• A Core Network often uses IP-MPLS to connect different Metro networks together.

Much of the functionality of Metro Ethernet such as Virtual Private Line or Virtual Private LAN is implemented by the use of Ethernet VLAN tags that allow differentiation of each part of the network. Logical differentiation of the physical network helps to identify the rights that the traffic has and to ease the management of hosts' access rights with respect to other users and networks.

Pure Ethernet MANs

A pure Ethernet MAN uses only layer 2 switches for all of its internal structure. This allows for a very simple and cheap design, and also for a relatively simple initial configuration. The original Ethernet technology was not well suited for service provider applications; as a shared-media network, it was impossible to keep traffic isolated, which made implementation of private circuits impossible. Ethernet MANs became feasible in the late 90s due to the development of new techniques to allow transparent tunneling of traffic through the use of Virtual LANs as "point to point" or "multipoint to multipoint" circuits. Combined with new features such as VLAN Stacking (also known as VLAN Tunneling), and VLAN Translation, it became possible to isolate the customers' traffic from each other and from the core network internal signaling traffic. However, Ethernet is constantly evolving and has now carrier class features with the recent addition of IEEE 802.1ad (Provider Bridges)(also known as QinQ or stacked VLANs) and IEEE 802.1ah (Provider Backbone Bridges) (also known as MAC in MAC or PBB) and IEEE 802.1Qay (Provider Backbone Transport) (also known as PBT or PBB-TE). Spanning-tree, broadcast packets and dynamic MAC learning are disabled and sub 50ms failover features are introduced.

There are three main shortcomings with a pure non PBT/PBB enabled Ethernet MAN approach:

• By design, layer 2 switches use fixed tables to direct traffic based on the MAC address of the endpoints. As the network gets larger, the number of MAC address transiting through the network may grow beyond the capacity of the core switches. If the core table gets full, the result is a catastrophic loss of performance due to the flooding of packets over the entire network structure. This can be overcome to some degree by smart network design and keeping your network segments and rings small enough to support the MAC table limitations of the equipment. In a pure ethernet network, the network should be designed in a modular grouping where your less expensive, smaller MAC table devices are in small geographically significant segments connected by larger aggregation devices which are interconnected that support two tag manipulation and very large MAC tables. This design keeps locally geographically significant segments interconnected with less expensive equipment, and larger geographically connected areas interconnected with more expensive, more feature laden equipment. This keeps the MAC tables small and helps keep the pure ethernet network scalable.
• Network stability is relatively fragile, especially if compared to the more advanced SDH and MPLS approaches. The recovery time for the standard spanning tree protocol is in the range of tens of seconds, much higher than what can be obtained in the alternative networks (usually a fraction of second). There are a number of optimizations, some standardized through the IEEE, and others vendor-specific, that seek to alleviate this problem. The clever use of such features allow the network to achieve good stability and resilience, at the cost of a more complex configuration and possible use of non-standard, vendor-specific, mechanisms. Some vendor's implementations of RSTP achieve sub 50ms convergence in typical sized rings. RSTP also provides for easy deployment of complex designs such as multi-ring, figure eight, etc. If designed appropriately, in many networks the fragility in this network design can be overcome without the additional expense of MPLS.
• Traffic engineering is very limited. There are few tools to manage the topology of the network; also, the fact that forwarding is done hop-by-hop, added to the possibility of broadcasts even for unicast packets (for instance, while learning new addresses), makes predicting the real traffic pattern very difficult for a networking novice. Custom tools, such as topology maps that outline where blocking ports occur in the network during normal and backup conditions may need to be built to fully understand and troubleshoot the network quickly.

Despite these shortcomings, non PBT/PBB enabled Ethernet-based MANs are used for two primary purposes:

• For small scale deployments (under a few hundred customers), a pure Ethernet MAN can be highly cost-effective. It also has the advantage of not requiring advanced knowledge of IP and related protocols, such as BGP and MPLS, which are necessary for an MPLS-based deployment. Even for larger scale deployments for thousands and thousands of customers can be achieved if careful network design rules are followed. In order to do this effectively skilled networking professionals need to be utilized.
• In large scale Metro Ethernets, it is common for the access part of the network to use a pure layer 2 design. At this level, the pure layer 2 design is deemed to be cheaper while still operating under its design limitations. From the distribution layer and above, traffic is aggregated and routed using an MPLS-based Metro Ethernet design. In very large networks MPLS may be unavoidable, but with careful network design, the use of both PBT and MPLS and their associated cost and complexity can be postponed if not eliminated entirely by careful network planning and design.

Myths regarding Pure Ethernet:

The biggest myth being propagated regarding pure metro-ethernet or carrier ethernet is that there are 4094 VLANs available network wide for a provider network. This is simply not true. There are 4094 VLANs available on each switched path. So the VID(vlan id) cannot be reused along the path from point a to point z, but can be reused anywhere else in the network as long as the paths are separated. Larger pure ethernet aggregation devices allow for traffic classification up to two tags deep. This allows for up to 16.7 million paths on a device of this nature, which should be used to aggregate devices that can only classify traffic based on 4094 VLAN ids. So with proper network design, in most networks VLAN exhaustion is not an issue if the network is designed appropriately. The network should be designed so that devices supporting large MAC tables and traffic classification of two tags are interconnected, and they act as an aggregator for less expensive, smaller mac table, one tag switches in attached rings and segments. Attaching these devices to interconnect larger areas provides for the theoretic possibility of up to 16.7 million unique paths between these devices, limited only by the device processing and memory capabilities. In a properly designed geographically significant modular network, more expensive services such as MPLS and PBT can be postponed or eliminated entirely. VLANs are locally significant only.

Another myth is that RSTP convergence takes many seconds. In certain situations and with some equipment this may be true. However, some vendors are offering devices that will converge RSTP in sub-50ms with little to no planning or effort. Advanced network planning may be required to achieve these speeds in certain situations, but it is possible with certain vendor's RSTP deployment. Problems with spanning tree in many instances arise from poor planning, design, and deployment. Spanning tree should be segmented and designed in small domains to be successful. A spanning tree domain is an area in which BPDUs will propagate. While advanced features of MSTP can be utilized, so can building manual spanning tree domains with legacy RSTP by disabling or blocking BPDUs on certain planned segments. In this way you create domains of segments and rings where spanning-tree is enabled, and keep the segments manageable. It is also essential to chose a root bridge and backup root bridge carefully. Path-costs should be modified so that the network administrator knows exactly what will happen to the traffic in the event of a failed segment anywhere in the network.

Another myth is that L2 metro-ethernet connections remove the need for using L3 routers or L3 switches. This is also not true. While equipment will operate just fine over your new metro-ethernet gear on L2 without a router. The whole point is to provide low latency transport. Why send unnecessary broadcast traffic over a metro-ethernet connection that you are probably paying for by Mbps? In most situations routing over your metro-ethernet connection will keep your broadcast traffic down to a bare minimum and help utilize your connection's bandwidth for real traffic, not superfluous packets. This is especially important with more and more nodes on each end of the connection. Routers are not very expensive. If you are paying out hundreds or thousands monthly for a metro-ethernet connection, spend the extra money and get a good router.

SONET/SDH-based Ethernet MANs

A SONET/SDH based Ethernet MAN is usually used as an intermediate step in the transition from a traditional, time-division based network, to a modern statistical network (such as Ethernet). In this model, the existing SDH infrastructure is used to transport high-speed Ethernet connections. The main advantage of this approach is the high level of reliability, achieved through the use of the native SDH protection mechanisms, which present a typical recovery time of 50 ms for severe failures. On the other hand, an SDH-based Ethernet MAN is usually more expensive, due to costs associated with the SDH equipment that is necessary for its implementation. Traffic engineering also tends to be very limited. Hybrid designs use conventional Ethernet switches at the edge of the core SDH ring to alleviate some of these issues, allowing for more control over the traffic pattern and also for a slight reduction in cost.

MPLS-based Ethernet MANs

An MPLS based Metro Ethernet network uses MPLS in the Service Provider's Network. The subscriber will get an Ethernet interface on Copper (ex:-100BASE-TX) or fiber (ex:-100BASE-FX). The customer's Ethernet packet is transported over MPLS and the service provider network uses Ethernet again as the underlying technology to transport MPLS. So, it is Ethernet over MPLS over Ethernet.

Here, Label Distribution Protocol (LDP) signaling is used as site to site signaling for the inner label (VC label) and Resource reSerVation Protocol-Traffic Engineering (RSVP-TE) or LDP may be used as Network signaling for the outer label.

One of the restoration mechanisms used in an MPLS based Metro Ethernet Networks is Fast ReRoute (FRR) to achieve sub-50ms convergence of MPLS local protection. For each deployment situation the benefit versus cost of MPLS must be weighed carefully, so if not implemented on a carrier's distribution network there might be more benefit for MPLS the core network. In some situations the cost may not warrant the benefits, particularly if sub 50ms convergence time is already being achieved with pure Ethernet.

Maturity of Metro Ethernet

A comparison of MPLS-based Metro Ethernet against a pure Ethernet MAN:

• Scalability: In a properly designed Ethernet VLAN network, each switched path can have 4094 single tag VLANs. Some aggregation and core switches can classify traffic by two VLANs using IEEE 802.1ad VLAN stacking, so with such aggregation devices properly placed in the center of a network, end segments and rings of single tag devices can receive only the traffic that they need. When using MPLS, Ethernet VLANs have local meaning only (like Frame Relay PVC). Same scalability considerations apply to the MAC addresses where in a pure Layer 2 Ethernet MAN all MAC addresses are being shared across the network, although this issue can be managed by smart network design and choosing switches with MAC tables sufficient for the size of network segments.
• Resiliency: pure Ethernet network resiliency relies on Spanning Tree Protocols STP, IEEE 802.1w RSTP or IEEE 802.1s MSTP (30 to sub 50ms sec convergence depending on network design) while MPLS-based MANs use mechanisms such as MPLS Fast Reroute to achieve SDH-like (50 msecs) convergence times. Metro Ethernet can also utilise Link aggregation or Resilient Packet Ring where appropriate to add link redundancy and recovery in distribution networks. Some Ethernet vendors' RSTP convergence is also sub-50ms, but this convergence time may vary from vendor to vendor. Ethernet protection switching is also standardised in (ITU G.8031).
• Multiprotocol convergence: with the maturity on pseudowires standards (ATM Virtual Leased Line VLL, FR VLL, etc.) an MPLS-based Metro Ethernet can backhaul IP/Ethernet traffic together with virtually any type of traffic coming from customer or other access networks (i.e. ATM aggregation for UMTS or TDM aggregation for GSM), while this could be more challenging in a pure Ethernet scenario.
• End to End OAM: MPLS-based MAN offers a wide set of troubleshooting and OAM MPLS-based tools which enrich Service Providers ability to effectively troubleshoot and diagnose network problems. These include for example, MAC ping, MAC traceroute, LSP ping etc. However there are now Ethernet OAM tools defined in IEEE 802.1ab]], IEEE 802.1ag^[2] and Ethernet in the First Mile (IEEE 802.3ah^[3]) for monitoring and troubleshooting Ethernet networks. EOAM (Ethernet Operations, Administration, and Maintenance) is a protocol for installing, monitoring, and troubleshooting MANs and WANs.

The Metro Ethernet Forum (MEF) has defined three types of services that can be delivered through Metro Ethernet:

• E-Line or Ethernet Virtual Private Line (EPVL), a Point-to-Point Ethernet Virtual Connection — equivalent of Virtual Private Wire Service (VPWS), Virtual Leased Line (VLL).
• E-LAN or Ethernet Virtual Private LAN (EVP-LAN), a MultiPoint-to-MultiPoint Ethernet Virtual Connection — equivalent of Virtual Private LAN Services (VPLS), Transparent LAN Services.
• E-TREE or Ethernet Virtual Private Tree (EVPT), a Rooted-MultiPoint Ethernet Virtual Connection for multicast domains.

Additionally, various access services can be provided with Metro Ethernet including; High Speed Internet access and IP/VPN access.

There are lot of vendors supplying equipment for Metro Ethernet deployments. They include ADTRAN, ADVA Optical Networking, Alcatel-Lucent, C-COR, Fujitsu Network Communications (FNC), Ciena, Cisco, Creanord, cyaninc.com, DATACOM, Dahili Network, Ericsson, Extreme Networks, Foundry Networks, Hatteras Networks, Huawei, IPITEK, Juniper Networks, MAIPU, MRV, Nortel Networks, RAD Data Communications, Redback Networks an Ericsson Company, Tejas Networks, Tellabs, ZTE and many more.

In June 2002, HKBN built the largest Metro Ethernet IP network in the world, covering 1.62 million homes in Hong Kong. and it will continue to expand towards the 2.0 million target by 2010.

In late September 2007 Verizon Business announced that it is implementing a Metro Ethernet solution across Asia-Pacific including Australia, Singapore, Japan and Hong Kong using Nortel equipment.[1]

Africa's largest and most developed privately owned MPLS Based Metro Ethernet Network is in Kenya. Reaching more than 5000 corporate entities, Kenya Data Networks is providing High End Services using Alcatel Core and Siemens Access equipment. KDN is now moving into FTTH projects and intends to cover more than 100 000 buildings in East Africa within the next 3 years.

References

1. ^ EANTC. "Carrier Ethernet Services - The Future". EANTC. http://www.eantc.com/fileadmin/eantc/downloads/events/2007-2010/CEWC08/EANTC-CEWC08-WhitePaper_v1.1.pdf. Retrieved 29 May 2011. 
2. ^ 802.1ag - Connectivity Fault Management
3. ^ IEEE 802.3ah EFM Standard Ratified

Further reading

• Halabi, Sam (2003). Metro Ethernet. Cisco Press. ISBN 1-58705-096-X. 
• MPLS network over a Metro Ethernet network

External links

• VPLS.ORG
• Metro Ethernet Forum (MEF)
• The Challenges of Ethernet Access
• "Carrier Ethernet Forum". discussion group web site. http://www.carrierethernettechnology.com/forums/. Retrieved August 6, 2011. 
• VPLS.US