History of web syndication technology

This article is specifically dedicated to the history of web syndication technology and, more generally, to the history of technical innovation on many dialects of web syndication feeds such as RSS and Atom, as well as earlier variants such as CDF and more recent innovations like GData.

Pre-RSS Formats

Before RSS, several similar formats already existed for syndication, but none achieved widespread popularity or are still in common use today, as most were envisioned to work only with a single service. These originated from "push" and "pull" technologies. Two of the earliest examples are Backweb and Pointcast.Fact|date=February 2007

Between 1995 and 1997, Ramanathan V. Guha and others at Apple Computer's Advanced Technology Group developed the Meta Content Framework (MCF) [cite web |url=http://news.com.com/2100-1001-203893.html |title=W3C takes first step toward RDF spec |author=Lash, Alex |publisher=CNET |date=1997-10-03 |accessdate=2007-02-16 ] . MCF was a specification for structuring metadata information about web sites and other data, and the basis of Project X (aka Hot Sauce), a 3D flythrough visualizer for the web. When the research project was discontinued, Guha left Apple for Netscape.

According to Netscape's Marc Andreessen, writing in 1999, "Guha came to Netscape in 1997, and after meeting consultant Tim Bray, who was working on XML, he decided to turn MCF into an XML application." [cite web |url=http://wp.netscape.com/columns/techvision/innovators_rg.html |title=Innovators of the Net: R.V. Guha and RDF |author=Andreessen, Marc |date=1999-01-08 |accessdate=2007-02-16 ] . Guha and Bray adapted MCF to use XML, submitting a spec to W3C in June 1997 [http://www.w3.org/TR/NOTE-MCF-XML/] . This combination of MCF with XML later gave rise to RDF. [http://www.w3.org/TR/WD-rdf-syntax-971002/ 1997] [http://www.w3.org/TR/1998/WD-rdf-schema/ 1998] [http://www.w3.org/TR/1999/REC-rdf-syntax-19990222/ 1999] (Tim Bray's own account of this work gives generous credit to Guha. [cite web |url=http://www.tbray.org/ongoing/When/200x/2003/05/21/RDFNet |title=The RDF.net Challenge |author=Bray, Tim |date=2003-05-21 |accessdate=2007-01-31 ] )

In March of 1997, Microsoft submitted a detailed spec for Channel Definition Format (CDF) to w3c [http://www.w3.org/TR/NOTE-CDFsubmit.html] .This format was designed for the Active Channel feature of Internet Explorer 4.0. CDF never became popular, perhaps because of the extensive resources it required at a time when people were mostly on dial-up. Backweb and Pointcast were geared towards news, much like a personal API feed. Backweb later morphed into providing software updates, a precursor to the push update features used by various companies now.

In September of 1997, Netscape previewed a new, competing technology "Aurora," said to be based on Resource Description Framework (RDF), which CNET stated was based on XML. [cite web |url=http://news.com.com/2100-1001-203685.html |title=Netscape previews Aurora |author=Clark, Tim |date=1997-09-29 |accessdate=2007-02-17 ] A slightly later CNET article [cite web |url=http://news.com.com/2100-1001-203893.html |title=W3C takes first step toward RDF spec |author=Lash, Alex |date=1997-10-03 |publisher=CNET |accessdate=2007-02-17 ] describes the October, 1997, submission to W3C of a draft for RDF [http://www.w3.org/TR/WD-rdf-syntax-971002/] , by a working group that included members from many different companies, including R.V. Guha of Netscape.

In December 1997, Dave Winer designed his own XML syndication format for use on his Scripting News weblog. [cite web |url=http://davenet.scripting.com/1997/12/15/scriptingNewsInXML |title=Scripting News in XML |author=Winer, Dave |date=1997-12-15 |publisher=Scripting News |accessdate=2006-10-31 ]

RSS Creation

RDF Site Summary, the first web syndication format to be called "RSS", was offered by Netscape in March 1999 for use on the My Netscape portal. This version became known as RSS 0.9. [cite web |url=http://my.netscape.com/publish/help/quickstart.html |title=My Netscape Network: Quick Start |archiveurl=http://web.archive.org/web/20001208063100/http://my.netscape.com/publish/help/quickstart.html |archivedate=2000-12-08 |publisher=Netscape Communications |accessdate=2006-10-31 ] .

In July 1999, responding to comments and suggestions, Dan Libby produced a prototype tentatively named RSS 0.91 [cite web |url= http://web.archive.org/web/20001204093600/my.netscape.com/publish/formats/rss-spec-0.91.html |title=RSS 0.91 Spec, revision 3 |author=Libby, Dan |date=1999-07-10 |publisher=Netscape Communications |accessdate=2007-02-14 ] (RSS standing for Rich Site Summary), that simplified the format and incorporated parts of Winer's scripting news format. This they considered an interim measure, with Libby suggesting an RSS 1.0-like format through the so-called Futures Document. [cite web |url=http://my.netscape.com/publish/help/futures.html |title=MNN Future Directions |archiveurl=http://web.archive.org/web/20001204123600/http://my.netscape.com/publish/help/futures.html |archivedate=2000-12-04 |publisher=Netscape Communications |accessdate=2006-10-31 ]

In April 2001, in the midst of AOL's acquisition and subsequent restructuring of Netscape properties, a re-design of the My Netscape portal removed RSS/XML support. The RSS 0.91 DTD was removed during this re-design, but in response to feedback, Dan Libby was able to restore the DTD, but not the RSS validator previously in place. In response to comments within the RSS community at the time, Lars Marius Garshol, to whom (co?)authorship of the original 0.9 DTD is sometimes attributed, commented, "What I don't understand is all this fuss over Netscape removing the DTD. A well-designed RSS tool, whether it validates or not, would not use the DTD at Netscape's site in any case. There are several mechanisms which can be used to control the dereferencing of references from XML documents to their DTDs. These should be used. If not the result will be as described in the article." [cite web |url=http://www.webreference.com/authoring/languages/xml/rss/1/ |title=The Evolution of RSS |author=Andrew King |date=2003-04-13 |accessdate=2007-01-17 ]

Effectively, this left the format without an owner, just as it was becoming widely used.

RSS (2000 - 2003)

A working group and mailing list, RSS-DEV, was set up by various users and XML notables to continue its development. At the same time, Winer unilaterally posted a modified version of the RSS 0.91 specification to the Userland website, since it was already in use in their products. He claimed the RSS 0.91 specification was the property of his company, UserLand Software. [cite web |url=http://backend.userland.com/rss091#copyrightAndDisclaimer |title=RSS 0.91: Copyright and Disclaimer |author=Winer, Dave |date=2000-06-04 |publisher=UserLand Software |accessdate=2006-10-31 ]

Since neither side had any official claim on the name or the format, arguments raged whenever either side claimed RSS as its own, creating what became known as the RSS fork.

The RSS-DEV group went on to produce RSS 1.0 in December 2000. [cite web |url=http://web.resource.org/rss/1.0/spec |title=RDF Site Summary (RSS) 1.0 |author=RSS-DEV Working Group |date=2000-12-09 |accessdate=2006-10-31 ] Like RSS 0.9 (but not 0.91) this was based on the RDF specifications, but was more modular, with many of the terms coming from standard metadata vocabularies such as Dublin Core.

Nineteen days later, Winer released by himself RSS 0.92, [cite web |url=http://backend.userland.com/rss092 |title=RSS 0.92 Specification |author=Winer, Dave |date=2000-12-25 |publisher=UserLand Software |accessdate=2006-10-31 ] a minor and supposedly compatible set of changes to RSS 0.91 based on the same proposal. In April 2001, he published a draft of RSS 0.93 which was almost identical to 0.92. [cite web |url=http://backend.userland.com/rss093 |title=RSS 0.93 Specification |author=Winer, Dave |date=2001-04-20 |publisher=UserLand Software |accessdate=2006-10-31 ] A draft RSS 0.94 surfaced in August, reverting the changes made in 0.93, and adding a "type" attribute to the "description" element.

In September 2002, Winer released a final successor to RSS 0.92, known as "RSS 2.0" and emphasizing "Really Simple Syndication" as the meaning of the three-letter abbreviation. The RSS 2.0 spec removed the "type" attribute added in RSS 0.94 and allowed people to add extension elements using XML namespaces. Several versions of RSS 2.0 were released, but the version number of the document model was not changed.

In November 2002, "The New York Times" began offering its readers the ability to subscribe to RSS news feeds related to various topics. In January, 2003, Winer called the New York Times' adoption of RSS the "tipping point" in driving the RSS format's becoming a de facto standard.

In July 2003, Winer and Userland Software assigned ownership of the RSS 2.0 specification to his then workplace, Harvard's Berkman Center for the Internet & Society. [cite web |url=http://blogs.law.harvard.edu/tech/announceRss2 |title=RSS 2.0 Specification moves to Berkman |date=2003-07-15 |publisher=Berkman Center for the Internet & Society |accessdate=2006-10-31 ]

Development of Atom (2003)

As of 2003, the primary method of web content syndication was the RSS family of formats. Members of the community who felt there were significant deficiencies with this family of formats were unable to make changes directly to RSS 2.0 because it was copyrighted by Harvard University and "frozen" by the official specification document, which stated that "no significant changes can be made and it is intended that future work be done under a different name". [http://blogs.law.harvard.edu/tech/rss] )

In June 2003, Sam Ruby set up a wiki to discuss what makes [http://www.intertwingly.net/blog/1472.html "a well-formed log entry"] . This initial posting acted as a rallying point. [http://www.tbray.org/ongoing/When/200x/2003/06/23/SamsPie] People quickly started using the wiki to discuss a new syndication format to address the shortcomings of RSS. It also became clear that the new format could also form the basis of a more robust replacement for blog editing protocols such as Blogger API and LiveJournal XML-RPC Client/Server Protocol.

The project aimed to develop a web syndication format that was: [http://www.intertwingly.net/wiki/pie/RoadMap]
* "100% vendor neutral,"
* "implemented by everybody,"
* "freely extensible by anybody, and"
* "cleanly and thoroughly specified."

In short order, a [http://www.intertwingly.net/wiki/pie/RoadMap project road map] was built. The effort quickly attracted [http://www.intertwingly.net/wiki/pie/RoadMap#head-74b5d46318b48115b07ab1a2e77cb22df987c284 more than 150 supporters] including David Sifry of Technorati, Mena Trott of Six Apart, Brad Fitzpatrick of LiveJournal, Jason Shellen of Blogger, Jeremy Zawodny of Yahoo, Timothy Appnel of the O'Reilly Network, Glenn Otis Brown of Creative Commons and Lawrence Lessig. Other notables supporting Atom include Mark Pilgrim, Tim Bray, Aaron Swartz, Joi Ito, and Jack Park. [http://www.intertwingly.net/wiki/pie/RoadMap#head-74b5d46318b48115b07ab1a2e77cb22df987c284] Also, Dave Winer, the key figure behind RSS 2.0, gave tentative support to the Atom endeavor (which at the time was called Echo.) [http://backend.userland.com/2003/06/26]

After this point, discussion became chaotic, due to the lack of a decision-making process. The project also lacked a name, tentatively using "Pie," "Echo," and "Necho" before settling on Atom. After releasing a project snapshot known as "Atom 0.2" in early July 2003, discussion was shifted off the wiki.

The discussion then moved to a newly set up mailing list. The next and final snapshot during this phase was "Atom 0.3", released in December 2003. This version gained widespread adoption in syndication tools, and in particular it was added to several Google-related services, such as Blogger, Google News, and Gmail. Google's Data APIs (Beta) GData are based on Atom 1.0 and RSS 2.0.

Atom 1.0 and IETF Standardization

In 2004, discussions began about moving the Atom project to a standards body such as the World Wide Web Consortium or the Internet Engineering Task Force (IETF). The group eventually chose the IETF and the [http://www.ietf.org/html.charters/atompub-charter.html Atompub working group] was formally set up in June 2004, finally giving the project a charter and process. The Atompub working group is co-chaired by Tim Bray (the co-editor of the XML specification) and Paul Hoffman. Initial development was focused on the syndication format.

The final draft of "Atom 1.0" was published in July 2005 and was accepted by the IETF as a "proposed standard" in August of 2005. Work then continued on the further development of the publishing protocol and various extensions to the syndication format.

The Atom Syndication Format was issued as a proposed "internet official protocol standard" in IETF RFC 4287 in December 2005. [http://www.ietf.org/rfc/rfc4287.txt] The co-editors of RFC 4287 were Mark Nottingham and Robert Sayre.

Recent technical developments related to web syndication

In January 2005, Sean B. Palmer, Christopher Schmidt, and Cody Woodard produced a preliminary draft of RSS 1.1. [cite web |url=http://inamidst.com/rss1.1/ |title=RSS 1.1: RDF Site Summary |author=Palmer, Sean B. and Christopher Schmidt |date=2005-01-23 |accessdate=2006-10-31 ] It was intended as a bugfix for 1.0, removing little-used features, simplifying the syntax and improving the specification based on the more recent RDF specifications. As of July 2005, RSS 1.1 had amounted to little more than an academic exercise.

In April 2005, Apple Computer released Safari 2.0 with RSS Feed capabilities built in. Safari delivered the ability to read RSS feeds, and bookmark them, with built-in search features. Safari's RSS button is a blue rounded rectangle with RSS written inside in white, . The favicon displayed defaults to a newspaper icon .

In November 2005, Microsoft proposed its Simple Sharing Extensions to RSS. [cite web |url=http://msdn.microsoft.com/xml/rss/sse/ |title=Simple Sharing Extensions for RSS and OPML |date=2006-01-12 |publisher=Microsoft |accessdate=2006-10-31 ]

In December 2005, the [http://blogs.msdn.com/rssteam/archive/2005/12/14/503778.aspx Microsoft IE team] and [http://blogs.msdn.com/michael_affronti/archive/2005/12/15/504316.aspx Outlook team] announced in their blogs that they will be adopting the [http://feedicons.com/ feed icon] first used in the Mozilla Firefox browser 9 release.

In January 2006, Rogers Cadenhead [http://www.rssboard.org/news/15/rss-advisory-board-goes-public relaunched] the RSS Advisory Board in order to move the RSS format forward.

In January 2007, as part of a revitalization of Netscape by AOL, the FQDN for my.netscape.com was redirected to a holding page in preparation for an impending relaunch, and as a result some news feeders using RSS 0.91 [http://news.bbc.co.uk/2/hi/technology/6263501.stm stopped working] . The DTD has again been restored.

References

External links

Early RSS history from several different personal points of view

* [http://goatee.net/2003/rss-history.html History of RSS compiled in 2003 by Joseph Reagle]
* [http://blogs.law.harvard.edu/tech/rssVersionHistory History of RSS compiled in 2004 by Dave Winer]
* [http://diveintomark.org/archives/2002/09/06/history_of_the_rss_fork History of the RSS fork compiled in 2002 by Mark Pilgrim]