Ethernet flow control

WireShark screenshot of an ethernet "Pause" frame

Ethernet flow control is a mechanism for temporarily stopping the transmission of data on Ethernet family computer networks.

Description

Ethernet is a popular family of computer network protocols. Flow control can be implemented at the data link layer. A sending station (computer or network switch) may be transmitting data faster than the other end of the link can accept it. The first flow control mechanism, the PAUSE frame, was defined by the Institute of Electrical and Electronics Engineers (IEEE) task force that also defined full duplex Ethernet link segments. The IEEE standard 802.3x was issued in 1997.^[1]

Pause frame

The overwhelmed network element can send a PAUSE frame, which halts the transmission of the sender for a specified period of time. Media Access Control (MAC) frames to carry the PAUSE commands using Control opcode 0x0001 (hexadecimal). Only stations configured for full-duplex operation may send PAUSE frames. When a station wishes to pause the other end of a link, it sends a MAC Control frame to the 48-bit destination reserved multicast address of 01-80-C2-00-00-01. The use of a well-known address makes it unnecessary for a station to discover and store the address of the station at the other end of the link.

Another advantage of using this multicast address arises from the use of flow control between network switches. The particular multicast address used is selected from a range of address which have been reserved by the IEEE 802.1D standard which specifies the operation of switches used for bridging. Normally, a frame with a multicast destination sent to a switch will be forwarded out all other ports of the switch. However, this range of multicast address is special and will not be forwarded by an 802.1D-compliant switch. Instead, frames sent to this range are understood to be frames meant to be acted upon only within the switch.

A PAUSE frame includes the period of pause time being requested, in the form of two byte unsigned integer (0 through 65535). This number is the requested duration of the pause. The pause time is measured in units of pause "quanta", where each unit is equal to 512 bit times.

By 1999, several vendors supported receiving pause frames, but fewer implemented sending them.^[2][3] Pause frames have several disadvantages.

Issues

One original motivation for the pause frame was to handle network interface controllers (NICs) that did not have enough buffering to handle full-speed reception. This problem is not as common with advances in bus speeds and memory sizes. A more likely scenario is network congestion within a switch. For example, a flow can come into a switch on a higher speed link than the one it goes out, or several flows can come in over two or more links that total more than an output link's bandwidth. These will eventually exhaust any amount of buffering in the switch. However, blocking the sending link will cause all flows over that link to be delayed, even those that are not causing any congestion. This situation is a case of head-of-line blocking, and can happen more often in core network switches due to the large numbers of flows generally being aggregated. Many switches use a technique called Virtual Output Queues to eliminate the HOL blocking internally, so will never send pause frames.^[3]

Subsequent efforts

Congestion management

Another effort was begun in March 2004, and in May 2004 became the IEEE P802.3ar Congestion Management Task Force. In May 2006 the objectives of the task force were revised to specify a mechanism to limit the transmitted data rate at about 1% granularity. The request was withdrawn and task force disbanded in 2008.^[4]

Priority flow control

Ethernet Flow control disturbs the Ethernet class of service (defined in IEEE 802.1p), as the data of all priorities are stopped to clear the existing buffers which might also consist of low priority data. As a remedy to this problem, Cisco Systems came up with its own priority flow control extension of the standard protocol. This mechanism uses 14 bytes of the 42-byte stuffing in a regular pause frame. The MAC control opcode for a Priority pause frame is 0x0101. Unlike the original pause, Priority pause indicates the pause time in quanta for each of eight priority classes separately.^[5] The Priority-based Flow Control (PFC) project was authorized on March 27, 2008 as IEEE 802.1Qbb. Draft 2.3 was proposed on June 7, 2010. Claudio DeSanti of Cisco was editor.^[6] The effort was part of the data center bridging task group, which developed Fibre Channel over Ethernet.^[7]

References

1. ^ IEEE Standards for Local and Metropolitan Area Networks: Supplements to Carrier Sense Multiple Access With Collision Detection (CSMA/CD) Access Method and Physical Layer Specifications - Specification for 802.3 Full Duplex Operation and Physical Layer Specification for 100 Mb/s Operation on Two Pairs of Category 3 Or Better Balanced Twisted Pair Cable (100BASE-T2). Institute of Electrical and Electronics Engineers. 1997. doi:10.1109/IEEESTD.1997.95611. ISBN 1-55937-905-7. http://ieeexplore.ieee.org/servlet/opac?punumber=9536. 
2. ^ Ann Sullivan; Greg Kilmartin; Scott Hamilton (September 13, 1999). "Switch Vendors pass interoperability tests". Network World: pp. 81–82. http://books.google.com/books?id=0Q4EAAAAMBAJ&pg=PA82. Retrieved May 10, 2011. 
3. ^ ^{a b} "Vendors on flow control". Network World Fusion. September 13, 1999. http://www.networkworld.com/netresources/0913flow2.html. Retrieved May 10, 2011.  Vendor comments on flow control in the 1999 test.
4. ^ "IEEE P802.3ar Congestion Management Task Force". December 18, 2008. http://grouper.ieee.org/groups/802/3/ar/. Retrieved May 10, 2011. 
5. ^ "Priority Flow Control: Build Reliable Layer 2 Infrastructure". White Paper. Cisco Systems. June 2009. http://www.cisco.com/en/US/prod/collateral/switches/ps9441/ps9670/white_paper_c11-542809.pdf. Retrieved May 10, 2011. 
6. ^ "IEEE 802.1Q Priority-based Flow Control". Institute of Electrical and Electronics Engineers. June 7, 2010. http://www.ieee802.org/1/pages/802.1bb.html. Retrieved May 10, 2011. 
7. ^ "Data Center Bridging Task Group". Institute of Electrical and Electronics Engineers. June 7, 2010. http://www.ieee802.org/1/pages/dcbridges.html. Retrieved May 10, 2011. 

External links

• "Ethernet Media Access Control - PAUSE Frames". TechFest Ethernet Technical Summary. 1999. http://www.techfest.com/networking/lan/ethernet3.htm#3.2.1. Retrieved May 10, 2011. 
• Tim Higgins (November 7, 2007). "When Flow Control is not a Good Thing". Small Net Builder. http://www.smallnetbuilder.com/index2.php?option=com_content&task=view&id=30212&pop=1&page=0&Itemid=54. Retrieved May 10, 2011. 
• Linux Tool for generating flow control PAUSE frames
• "Ethernet Flow Control". Topics in High-Performance Messaging. http://www.29west.com/docs/THPM/thpm.html#ETHERNET-FLOW-CONTROL. Retrieved November 10, 2007.