ATA over Ethernet

ATA over Ethernet (AoE) is a network protocol developed by the Brantley Coile Company [http://www.coraid.com/documents/AoEr10.txt] , designed for simple, high-performance access of SATA storage devices over Ethernet networks. It gives the possibility to build SANs with low-cost, standard technologies.

AoE does not rely on network layers above Ethernet, such as IP, UDP, TCP, etc. While this makes AoE potentially faster than iSCSI, it also means that AoE is not routable across LANs and is intended for local small SANs only. In this regard it is more comparable to Fibre Channel over Ethernet than iSCSI. An alternative to iSCSI, the AoE specification is 8 pages compared with iSCSI's 257 pages. Fact|date=April 2008

Operating system support

The following operating systems provide ATA over Ethernet support:

Hardware Support

The Coraid [http://www.coraid.com/] company offers an array of AoE SAN appliances under the EtherDrive brand.

In 2007 LayerWalker [http://www.layerwalker.com/] announced the world's first single-chip AoE hardware solution called [http://www.layerwalker.com/download/miniSAN-brief-1.0.pdf miniSAN] running at both Fast and Gigabit Ethernet grades. The miniSAN product family offers standard AoE server functions plus other management features that targets PC, consumer and SMB markets.

A commodity Linux server can function as an AoE target using vblade [http://sourceforge.net/projects/aoetools/vblade] . Two independent implementations of vblade exist: A userspace one (part of aoetools package) and another one implemented as a Linux kernel module.

Protocol Description

ATA Encapsulation

SATA (and older PATA) hard drives use the Advanced Technology Attachment (ATA) protocol to issue commands, such as read, write, and status. AoE encapsulates those commands inside Ethernet frames and lets them travel over an Ethernet network instead of a SATA or 40-pin ribbon cable. By using an AoE driver, the host operating system is able to access a remote disk as if it were directly attached.

The encapsulation of ATA provided by AoE is simple and low-level, allowing the translation to happen either at high performance or inside a small, embedded device, or both.

Routability

AoE runs directly on top of Ethernet, instead of an intermediate protocol such as TCP/IP. This reduces the significant CPU overhead of TCP/IP. However, this means that routers cannot be used to route a packet across disparate networks (such as the Internet). Instead, AoE packets can travel within a single local Ethernet storage area network (such as one created by a switch or VLAN).

Security

AoE does not provide native security features. The non-routability of AoE is one source of inherent security (ie: outside users cannot connect to it). However, there are no AoE-specific mechanisms for password verification or encryption. Security may be implemented at the file-system level. Storage is accessible by every machine connected to the switch.

Config String

The AoE protocol provides a mechanism for host-based cooperative locking. When more than one AoE initiator is using an AoE target, they must communicate. The hosts need a way to avoid interfering with one another as they use and modify the data on the shared AoE device.

One option provided by AoE is to use the storage device itself as the mechanism for determining the access of particular hosts. The AoE protocol includes a "config string" feature. The config string can record who is using the device. (It can also record any other information.) If more than one host tries to set the config string simultaneously, only one succeeds. The other host is informed of the conflict.

Related Concepts

Although AoE is a simple network protocol, it opens up a complex realm of storage possibilities. To understand and evaluate these storage scenarios, it helps to be familiar with a few concepts.

Storage Area Networks

The purpose of Storage Area Networks (SANs) is often not to make files and directories available to multiple users, as is the purpose of Network Attached Storage (NAS). Instead, a SAN allows the physical hard drive to be removed from the server that uses it, and placed on the network. A SAN interface is thus similar in principle to non-networked interfaces such as SATA or SCSI. Most users will not use a SAN interface directly. Instead, they will connect to a server that uses a SAN disk instead of a local disk. Direct connection, however, can also be used.

When using a SAN network to access storage, there are several potential advantages over a local disk:
* It is easier to add storage capacity and the amount of storage is practically unlimited.
* It is easier to reallocate storage capacity.
* Data may be shared.
* Additionally, compared to other forms of networked storage, SANs are low-level and high performance

Utilizing Storage Area Networks

To utilize a SAN disk, the host must format it with a filesystem. However, unlike a SATA or SCSI disk a SAN hard drive may be accessed by multiple machines. This is a source of both danger and opportunity.

Traditional filesystems (such as FAT or ext3) are designed to be accessed by a single host, and will cause unpredictable behavior if accessed by multiple machines. Such filesystems may be used, and AoE provides mechanisms whereby an AoE target can be guarded against simultaneous access (see: Config String).

Shared disk filesystems allow multiple machines to use a single hard disk safely by coordinating simultaneous access to individual files. They are slightly similar to network filesystems. These filesystems can be used to allow multiple machines access to the same AoE target without an intermediate server or filesytem (and at higher performance). Examples of shared disk filesystems are GFS, GPFS, MetaSAN, and OCFS2.

See also

* HyperSCSI
* iSCSI
* Fibre Channel over Ethernet

References

*
*

External links

* Articles:
** [http://www.linuxjournal.com/article/8149 ATA Over Ethernet: Putting Hard Drives on the LAN] — Linux Journal (28 April 2005)
** [http://linuxdevices.com/news/NS3189760067.html ATA-over-Ethernet enables low-cost Linux-oriented SAN] — LinuxDevices.com (23 June 2004)
** [http://www.linux-mag.com/id/2028/ The ATA over Ethernet (AoE) Protocol] — Linux Magazine (June 15 2005)
* Protocol:
** [http://www.coraid.com/site/co-pdfs/AoEr10.pdf AoE protocol description] (PDF file)
** [http://www.coraid.com/s.nl/sc.12/category.1120/.f AoE protocol specification]
* Emulator and Tools:
** [http://sourceforge.net/projects/aoetools/ ATA Over Ethernet Tools and vblade] vblade is an AoE target that runs in userspace, and kvblade is an alpha in-kernel AoE target
** [http://lpk.com.price.ru/~lelik/AoE/ vblade, implemented as a kernel module] this is faster than the userspace vblade
** [http://pi.nxs.se/~wowie/qaoed.tgz/ qaoed - Mulithreaded ATA over Ethernet storage target] Runs in userspace
** [http://code.google.com/p/aoeserver/ Aoeserver - Mulithreaded ATA over Ethernet storage target] Implemented as a kernel module
* Live CD:
** [http://www.lbserver.org/aoe/ Slax Frodo with vblade and ATA Over Ethernet Tools] this is useful for testing and backup
* HowTos:
** [http://www.howtoforge.com/ata_over_ethernet_debian_etch Using ATA Over Ethernet On Debian Etch]