IWARP

The Internet Wide Area RDMA Protocol (iWARP) is an Internet Engineering Task Force (IETF) update of the [http://www.rdmaconsortium.org/ RDMA Consortium's] RDMA over TCP standard. iWARP is a superset of the Virtual Interface Architecture that permits zero-copy transmission over legacy TCP. It may be thought of as the features of InfiniBand (IB) applied to Ethernet.

Because a kernel implementation of the TCP stack is seen as a bottleneck, a few vendors now implement TCP in hardware. As simple data losses are rare in tightly coupled network environments, the error-correction mechanisms of TCP may be performed by software while the more frequently performed communications are handled strictly by logic embedded on the network interface controller. This additional hardware is known as the TCP offload engine (TOE).

TOE itself does not prevent copying on the receive side, and must be combined with RDMA hardware for zero-copy results. The RDMA / TCP specification is a set of different wire protocols intended to be implemented in hardware (though it seems feasible to emulate it in software for compatibility but without the performance benefits).

The main component is the Data Direct Protocol (DDP), which permits the actual zero-copy transmission. DDP itself does not perform the transmission; TCP does.

However, TCP does not respect message boundaries; it sends data as a sequence of bytes without regard to protocol data units (PDU). In this regard, DDP itself may be better suited for SCTP, and indeed a focus of the IETF has been to standardize RDMA over SCTP. To run DDP over TCP requires a tweak known as marker PDU aligned (MPA) framing so as to guarantee boundaries of messages.

Furthermore, DDP is not intended to be accessed directly. Instead, a separate RDMA protocol (RDMAP) provides the services to read and write data. Therefore, the entire RDMA over TCP specification is really RDMAP over DDP over MPA over TCP. All of these protocols are intended to be implemented in hardware.

Like InfiniBand, iWARP does not have a standard programming interface, only a set of verbs. Unlike IB, iWARP only has reliable connected communication as this is the only service that TCP and SCTP provide. The iWARP specification also omits many of the special features of IB, such as atomic remote operations.

Networking protocols for iWARP include protocols offered in the OpenFabrics Enterprise Distribution (OFED) by the OpenFabrics Alliance for Linux operating systems, and the Winsock Direct protocol for Microsoft Windows.

Protocols carried over iWARP

*NFS over RDMA
*Sockets Direct Protocol (SDP)
*iSCSI Extensions for RDMA (iSER)
*SCSI RDMA Protocol (SRP)

External links

* [http://www.iol.unh.edu/services/testing/iwarp/ iWARP Consortium] at the University of New Hampshire InterOperability Laboratory -- Testing on iWARP devices
* [http://www.eetimes.com/story/OEG20011207S0091 EE Times] article (dated 12/07/2001) on iWARP
* [http://www.networkworld.com/buzz/2005/092605-iwarp.html Network World] article (dated 09/26/05) on iWARP and competing technologies
* [http://www.ietf.org/html.charters/rddp-charter.html Remote Direct Data Placement Charter] links to IETF specifications on iWARP
* [http://www.neteffect.com/ NetEffect] -- producers of iWARP equipment
* [http://www.ivivity.com/ iVivity] -- producers of iWARP equipment
* [http://www.chelsio.com/ Chelsio Communications] -- producers of iWARP equipment
* [http://www.lampreynetworks.com/Content/iWARP/iWARP_products.html Lamprey Networks] - Producers of iWARP conformance test suite.
* [http://www.c3.lanl.gov/cac2008/presentations/penoff.ppt iWARP over SCTP] -- [http://www.cs.ubc.ca/labs/dsg/mpi-sctp/ UBC group's] slides from their CAC 2008 paper detailing a design of iWARP over SCTP