Amazon S3

Amazon S3 (Simple Storage Service) is an online storage web service offered by Amazon Web Services. Amazon S3 provides unlimited storage through a simple web services interface. Amazon launched S3, its first publicly-available web service, in the United States in March 2006 [cite press release
publisher = Amazon.com
date = 2006-03-14
title = Amazon Web Services Launches "Amazon S3"
url = http://phx.corporate-ir.net/phoenix.zhtml?c=176060&p=irol-newsArticle&ID=830815] and in Europe in November 2007.cite news
work = Dr. Dobb's Portal
last = Dorsey
first = John
title = Amazon S3 Storage Now Available in Europe
url = http://www.ddj.com/architect/202803151
date = 2007-11-06
accessdate = 2008-03-26] Since its inception, Amazon has charged end users $0.15 per gigabyte-month, with additional charges for bandwidth used in sending and receiving data. As of November 1, 2008, pricing will move to tiers where end users storing more the 50 terabytes per month will receive discounted pricing. [cite offical blog
publisher = Amazon.com
date = 2008-10-08
title = Amazon S3 - Busier Than Ever
url = http://aws.typepad.com/aws/2008/10/amazon-s3---now.html] Amazon claims that S3 uses the same scalable storage infrastructure that Amazon.com uses to run its own global e-commerce network [ [http://www.amazon.com/gp/browse.html?node=16427261 The same data storage infrastructure that Amazon uses to run its own global network of web sites] ] . Amazon S3 is reported to store more than 29 billion objects as of October 2008. [cite offical blog
publisher = Amazon.com
date = 2008-10-08
title = Amazon S3 - Busier Than Ever
url = http://aws.typepad.com/aws/2008/10/amazon-s3---now.html] This is up from 14 billion objects as of January 2008, and from 10 billion in October of 2007.cite web
url=http://www.allthingsdistributed.com/2008/03/happy_birthday_amazon_s3.html
author=Vogels, Werner
work=All Things Distributed
date = 2008-03-19
title=Happy Birthday, Amazon S3!] S3 uses include web hosting, image hosting, and a back-up system. S3 comes with no guarantee that customer data will not be lost.

Design

S3's design aims to provide scalability, high availability, and low latency at commodity costs.

S3 stores arbitrary objects up to 5 gigabytes in size, each accompanied by up to 2 kilobytes of metadata. Objects are organized into "buckets" (each owned by an AWS account), and identified within each bucket by a unique, user-assigned key.

Buckets and objects can be created, listed, and retrieved using either a REST-style HTTP interface or a SOAP interface. Additionally, objects can be downloaded using the HTTP GET interface and the BitTorrent protocol.

Requests are authorized using an access control list associated with each bucket and object.

Bucket names and keys are chosen so that objects are addressable using HTTP URLs:

* http://s3.amazonaws.com/"bucket"/"key"
* http://"bucket".s3.amazonaws.com/"key"
* http://"bucket"/"key" (where "bucket" is a DNS CNAME record pointing to s3.amazonaws.com)

Because objects are accessible by unmodified HTTP clients, S3 can be used to replace significant existing web hosting infrastructure [http://www.bucketexplorer.com/documentation/amazon-s3--how-to-use-Amazon-s3-for-web-hosting.html How to use Amazon S3 for Web Hosting] ] . The Amazon AWS Authentication mechanism allows the bucket owner to create an authenticated URL with time-bounded validity. That is, someone can construct a URL that can be handed off to a third-party for access for a period such as the next thirty minutes, or the next twenty-four hours.

Every item in a bucket can also be served up as a BitTorrent feed, so the S3 store can act as a seed host for a torrent, and any BitTorrent client can retrieve the file, drastically reducing the bandwidth costs for the download. The bandwidth and storage reduction at Amazon S3 can also be augmented by using deduplication and single-instance storage. Amazon does not provide deduplication but many vendors provide this capability as a differentiator.

A bucket can be configured to save HTTP log information to a sibling bucket; this can be used in later data mining operations.

Notable uses

Photo hosting service SmugMug has used S3 since April 2006. In November 2006, they claimed to be saving US$500,000 per year by using S3 instead of their own dedicated storage servers.cite web|url=http://blogs.smugmug.com/don/2006/11/10/amazon-s3-show-me-the-money/|work=SmugMug Blog|publisher=SmugMug|date=November 10, 2006|title=Amazon S3: Show Me the Money] SmugMug has noted outages, slowdowns, and data loss with S3cite web|url=http://blogs.smugmug.com/don/2007/01/30/amazon-s3-outages-slowdowns-and-problems/|work=SmugBlog |title=Amazon S3 Outages, Slowdowns, and Problems|publisher=SmugMug|date=January 30, 2007] but continued to use the service. After seven months of using S3, Smugmug claimed to have saved almost $1 million in storage costs, though some commentors questioned SmugMugs claims, pointing out that S3 charges per month. SmugMug responded that they had been spending over $6,000 per terabyte of space.

There is a User Mode File System for Linux that lets EC2-hosted Xen images mount an S3 bucket as a file system. Note that as the semantics of the S3 file system are not that of a Posix file system, the file system may not behave entirely as expected.

Apache Hadoop file systems can be hosted on S3, as its requirements of a file system are met by S3. As a result, Hadoop can be used to run MapReduce algorithms on EC2 servers, reading data and writing results back to S3.

OpenSimulator is a OpenSource server that is used to host virtual worlds very similar to Second Life. Many OpenSimulator grids use Amazon S3 as a asset database storage because of very cheap costs.

References

* cite web
title = Amazon S3 Developer Guide
url = http://docs.amazonwebservices.com/AmazonS3/2006-03-01/
date = 2006-03-01

External links

* [http://aws.amazon.com/s3 Amazon S3] (official site)