Metalink

Metalink

Filename extension	.meta4 .metalink
Internet media type	application/metalink4+xml application/metalink+xml
Type of format	File distribution
Extended from	XML, HTTP

This article is about the open standard. For the telecommunication company, see Metalink Broadband.

Metalink is a cross-platform and cross-application Internet standard/framework/file format for programs that download, including download managers, BitTorrent clients, Web browsers, FTP clients, and P2P programs. For increased convenience, the multiple download locations for a file (FTP/HTTP/P2P) and other metadata in HTTP header fields and/or an XML file with the extension .meta4 or .metalink. The duplicate download locations provide reliability in case one method fails. Some clients also achieve faster download speeds by allowing different chunks/segments of each file to be downloaded from multiple resources at the same time (segmented downloading).

Other features include automatic hash verification of finished downloads, automatic repair of downloads with errors, listing multiple files to be added to a download queue, and automatic OS/language selection.

Metalink is extensible, and supports listing multiple partial and full file hashes along with PGP signatures. Most clients only support verifying MD5, SHA-1, & SHA-256, however. Besides FTP and HTTP mirror locations and rsync, it also supports listing the P2P methods BitTorrent, ed2k, magnet link or any other that uses a URI.

Development History

Metalink 3.0 was publicly released in 2005. It was designed to aid in downloading Linux ISO images and other large files on release day, when servers would be overloaded (each server would have to be tried manually) and to repair large downloads by replacing only the parts with errors instead of fully re-downloading them. It was initially adopted by download managers, and was used by open source projects such as OpenOffice.org and Linux distributions. A community developed around it, more download programs supported it (including proprietary ones) and it saw commercial adoption. In 2008, the community took their work to the Internet Engineering Task Force which resulted in Metalink 4.0 in 2010, described in a Standards Track RFC. Metalink 3.0 (with the extension .metalink) and Metalink 4.0 (with the extension .meta4) are incompatible because they have a slightly different format. In 2011, another Standards Track RFC described Metalink in HTTP header fields.

Client programs

• aria2 (GPL) is a Unix/Windows command line download utility that supports BitTorrent and Metalink (HTTP/FTP/BitTorrent)
• DownThemAll (GPL), a Firefox extension and download manager that has basic Metalink support.
• FlashGot (GPL), a Firefox extension for integrating with many Download Managers, supports Metalink.
• Free Download Manager (GPL), a Windows Download Manager, supports Metalink (HTTP/FTP) in version 2.5.
• GetRight 6, a Windows Download Manager, supports Metalink (HTTP/FTP/BitTorrent).
• jDownloader is an open source download manager written in Java.
• KGet (GPL), KDE's native download manager, supports Metalink in KDE 4.
• Net Transport, a Windows Download Manager, supports Metalink.
• Orbit Downloader, a Windows Download Manager, supports Metalink (HTTP/FTP).
• Phex (GPL), a Mac/Unix/Windows gnutella client, supports Metalink downloads along with exporting a library to Metalink for sharing between users.
• Retriever is a closed source cross platform Java download manager that supports Metalink and BitTorrent.
• SmartFTP, a Windows GUI FTP client, supports Metalink for adding files to an FTP transfer queue and hash verification.
• VeryCD's easyMule (version 1 is GPLed, ver.2 may be GPL-Violation) is a version of eMule with Metalink support.
• wxDownload Fast (GPL) is a Mac/Unix/Windows download manager that supports Metalink (HTTP/FTP).
• yum, an open source command line package management utility for RPM-compatible Linux operating systems, uses Metalink internally, invisible to the user.
• ZYpp, the package management engine that powers YaST2 and Zypper, supports Metalink for downloading. It uses aria2 internally.

See also: Comparison of download managers

In use

Mandriva Linux has integrated Metalink into package management with urpmi and aria2. Fedora has integrated Metalink into package management with yum. openSUSE has integrated Metalink into package management with ZYpp and aria2.

Wubi, the Windows-based Ubuntu installer, uses Metadl (LGPL) to download Ubuntu ISO images and takes advantage of Metalink's features of higher availability and increased reliability. If there are errors in the download, they are repaired, instead of restarting the large download.

Appupdater (GPL) for Windows "is similar to apt-get or yum on Linux. It automates the process of installing and maintaining up to date versions of programs."

Currently, OpenOffice.org uses Metalinks to distribute their free office suite. cURL offers Metalinks of their downloads. UniProt Consortium also provides a Metalink for their large protein database. Dofus, a Flash MMORPG uses Metalinks for downloads and so does Sage, the open-source mathematical software.

The software hosting and collaboration platform Origo generates Metalinks for all hosted releases.

The following Linux distributions use Metalink for ISO image distribution: 64 Studio, Adios, Bayanihan Linux, BeleniX, Berry Linux, BLAG Linux and GNU, Bluewhite64 Linux, Damn Small Linux, Fedora, GoboLinux, Granular Linux, KateOS, Linux Mint, openSUSE, Pardus Linux, PCLinuxOS, PuppyLinux, Sabayon Linux, StartCom Enterprise Linux, SUSE Linux, Ubuntu.

The following FreeBSD based distributions use Metalink for ISO image distribution: DesktopBSD, MidnightBSD, PC-BSD and TrueBSD.

File based search engine findthatfile provides search for Metalinks.

Metalink client feature comparison

A guide to which clients support which Metalink features:

	aria2	DownThemAll	Free Download Manager	GetRight	KGet	Metadl	Metalink Checker	Orbit Downloader	Phex	Retriever	SmartFTP	Speed Download	TheWorld	wxDownload Fast
License	Free Software (GNU General Public License)	Free Software (GNU General Public License)	Free Software (GNU General Public License)	Proprietary / $29.95 for Standard, $49.95 for Pro	Free Software (GNU General Public License)	Free Software (GNU Lesser General Public License)	Free Software (GNU General Public License)	Proprietary / Freeware	Free Software (GNU General Public License)	Proprietary / Freeware	Proprietary / Freeware for personal, $36.95 commercial	Proprietary / $25	Proprietary / Freeware	Free Software (GNU General Public License)
Interface	CLI, XML-RPC	GUI	GUI	GUI	GUI	GUI	CLI	GUI	GUI	GUI	GUI	GUI	GUI	GUI
Metalink 4.0	Yes	Yes	No	No	Yes	No	Yes	No	No	Yes	Yes	No	No	No
Runs on Linux	Yes	Yes	No	No	Yes	No	Yes	Yes	Yes	Yes	No	No	No	Yes
Runs on Windows	Yes	Yes	Yes	Yes	No	Yes	Yes	Yes	Yes	Yes	Yes	No	Yes	Yes
Runs on Mac OS X	Yes	Yes	No	No	No	No	Yes	No	Yes	Yes	No	Yes	No	No
Resume Downloads	Yes	Yes	Yes	Yes	Yes	Yes	Yes	Yes	Yes	Yes	Yes	Yes	Yes	Yes
Segmented Downloads	Yes	Yes	Yes	Yes	Yes	No	Yes	Yes	Yes	Yes	No	Yes	Yes	Yes
Multiple File support	Yes	Yes	Yes	Yes	Yes	Yes	Yes	Yes	Yes	Yes	Yes	Yes	Yes	Yes
"location"	Yes	Yes	No	No	No	No	Yes	No	No	Yes	No	No	No	No
OS	Yes	Yes	Yes	No	No	No	Yes	No	No	Yes	No	Yes	No	No
Language	Yes	Yes	No	No	No	No	Yes	Yes	No	Yes	No	Yes	No	No
HTTP with Transparent Negotiation	Yes	Yes	No	No	No	No	Yes	No	No	Yes	No	No	No	No
HTTP	Yes	Yes	Yes	Yes	Yes	Yes	Yes	Yes	Yes	Yes	No	Yes	Yes	Yes
FTP	Yes	Yes	Yes	Yes	Yes	Yes	Yes	Yes	Yes	Yes	Yes	Yes	No	Yes
BitTorrent	Yes	No	Yes	Yes	Yes	No	No	No	No	Yes	No	No	No	No
magnet	Yes	No	No	No	No	No	No	Yes	Yes	No	No	No	No	No
MD5 Hashes	Yes	Yes	Yes	Yes	Yes	Yes	Yes	Yes	?	Yes	Yes	Yes	No	Yes
SHA-1 Hashes	Yes	Yes	Yes	Yes	Yes	Yes	Yes	Yes	Yes	Yes	Yes	Yes	No	Yes
SHA-256 Hashes	Yes	Yes	Yes	No	No	No	Yes	No	No	Yes	No	No	No	No
Piece Hashes	Yes	No	No	No	No	Yes	Yes	No	No	Yes	No	No	No	No
PGP Signatures	No	No	No	No	No	No	Yes	No	No	No	No	No	No	No

No clients support rsync at this time.

Metalink generation

Metalink is XML so they can be hand crafted or XML tools can be used for automation. These tools are specifically for Metalink.

• Bouncer is "database driven mirror management app that has three main components - a bounce script, a sentry script to check mirrors and files for integrity and a set of web-based administration tools." Bouncer supports Metalink via a patch that has not yet been integrated upstream.
• Geo McFly (GPL) is an on the fly generator of metalinks based on the downloader's geographical location, written in Python.
• Metalink Editor (GPL) is a Cross Platform GUI Metalink editor written in Python which allows you to create and edit Metalinks.
• Metalink Library (GPL) is a Cross Platform Metalink library for Python and PHP based on Metalink Editor with many automatic features and an easy command-line interface to generate Metalink and Torrent files.
• Metalink Generator can also be used to create Metalinks by filling out a Web form.
• Metalink tools (GPL) Unix/Windows command line tools for generating Metalinks with hashes and mirror/p2p lists.
• Dynamic online metalinks based on metadata. Automatically created metalinks based on metadata published by each mirror.
• MirrorManager (MIT X11 license) is used by the Fedora Project for dynamically listing mirrors.
• The MirrorBrain (GPL, Apache License) is a real-time Metalink generator and download redirector. It can either return Metalinks, or automatically redirect non-Metalink clients (per HTTP redirection) to a mirror server near them. Advanced features, as transparent negotiation of Metalinks, or embedding of PGP signatures, are fully supported. It is used by openSUSE, OpenOffice.org, and other projects.
• Origo is a new and easy to use software development and hosting platform for open and closed source projects. Among many features, it has easy release management and mirroring with automatic metalink generation.

Example Metalink 4.0 .meta4 file

Metalink 4.0 files have the extension .meta4 and are XML text files. They are served with the application/metalink4+xml Internet media type.

 <?xml version="1.0" encoding="UTF-8"?>
 <metalink xmlns="urn:ietf:params:xml:ns:metalink">
   <published>2009-05-15T12:23:23Z</published>
   <file name="example.ext">
     <size>14471447</size>
     <identity>Example</identity>
     <version>1.0</version>
     <language>en</language>
     <description>
     A description of the example file for download.
     </description>
     <hash type="sha-256">3d6fece8033d146d8611eab4f032df738c8c1283620fd02a1f2bfec6e27d590d</hash>
     <url location="de" priority="1">ftp://ftp.example.com/example.ext</url>
     <url location="fr" priority="1">http://example.com/example.ext</url>
     <metaurl mediatype="torrent" priority="2">http://example.com/example.ext.torrent</metaurl>
   </file>
 </metalink>

Example Metalink/HTTP header fields

Metalink in HTTP header fields makes use of existing standard HTTP header fields such as ETags, Link header fields (for mirrors and P2P), and Instance Digests (for hashes).

Etag: "thvDyvhfIqlvFe+A9MYgxAfm1q5="
Link: <http://www2.example.com/example.ext>; rel=duplicate
Link: <ftp://ftp.example.com/example.ext>; rel=duplicate
Link: <http://example.com/example.ext.torrent>; rel=describedby; type="application/x-bittorrent"
Link: <http://example.com/example.ext.meta4>; rel=describedby; type="application/metalink4+xml"
Link: <http://example.com/example.ext.asc>; rel=describedby; type="application/pgp-signature"
Digest: SHA-256=MWVkMWQxYTRiMzk5MDQ0MzI3NGU5NDEyZTk5OWY1ZGFmNzgyZTJlODYzYjRjYzFhOTlmNTQwYzI2M2QwM2U2MQ==

Example Metalink 3.0 .metalink file

Metalink 3.0 files have the extension .metalink and are XML text files.

 <?xml version="1.0" encoding="UTF-8"?>
 <metalink version="3.0" xmlns="http://www.metalinker.org">
   <files>
     <file name="example.ext">
       <verification>
         <hash type="md5">example-md5-hash</hash>
         <hash type="sha1">example-sha1-hash</hash>
         <signature type="pgp"/>
       </verification>
       <resources>
         <url type="ftp" location="us" preference="90">ftp://ftp.example.com/example.ext</url>
         <url type="ftp" location="uk" preference="90">ftp://ftp.example.net/example.ext</url>
         <url type="http" location="us" preference="90">http://example.com/example.ext</url> 
         <url type="http" location="de" preference="90">http://example.net/example.ext</url>
         <url type="bittorrent" preference="100">http://example.org/example.ext.torrent</url>
         <url type="rsync"/>
         <url type="magnet"/>
         <url type="ed2k"/>
       </resources>
     </file>
   </files>
 </metalink>

See also

• Magnet URI scheme

References

External links

• Metalink project official homepage
• Making Metalinks article

Metalink standard

• RFC 5854 - "The Metalink Download Description Format"
• RFC 6249 - "Metalink/HTTP: Mirrors and Hashes"
• Metalink 3.0 Specification