MongoDB

MongoDB

Developer(s)	10gen
Initial release	2009
Stable release	2.0.1 / October 22, 2011; 31 days ago (2011-10-22)
Development status	Active
Written in	C++
Operating system	Cross-platform
Available in	English
Type	Document-oriented database
License	GNU AGPL v3.0 (drivers: Apache license)
Website	www.mongodb.org

MongoDB (from "humongous") is an open source, high-performance, schema-free, document-oriented database written in the C++ programming language.^[1] It manages collections of BSON documents that can be nested in complex hierarchies and still be easy to query and index, which allows many applications to store data in a natural way that matches their native data types and structures.

Development of MongoDB began in October 2007 by 10gen. The first public release was in February 2009.^[2]

Features

Among the features are:

• Consistent UTF-8 encoding. Non-UTF-8 data can be saved, queried, and retrieved with a special binary data type.
• Cross-platform support: binaries are available for Windows, Linux, OS X, and Solaris. MongoDB can be compiled on almost any little-endian system.
• Type-rich: supports dates, regular expressions, code, binary data, and more (all BSON types)
• Cursors for query results

More features:

Ad hoc queries

In MongoDB, any field can be queried at any time. MongoDB supports range queries, regular expression searches, and other special types of queries in addition to exactly matching fields. Queries can also include user-defined JavaScript functions (if the function returns true, the document matches).

Queries can return specific fields of documents (instead of the entire document), as well as sorting, skipping, and limiting results.

Querying nested fields

Queries can "reach into" embedded objects and arrays. If the following object is inserted into the users collection:

{
    "username" : "bob",
    "address" : {
        "street" : "123 Main Street",
        "city" : "Springfield",
        "state" : "NY"
    }
}

We can query for this document (and all documents with an address in New York) with:

> db.users.find({"address.state" : "NY"})

Array elements can also be queried:

> db.food.insert({"fruit" : ["peach", "plum", "pear"]})
> db.food.find({"fruit" : "pear"})

Indexing

The software supports secondary indexes, including single-key, compound, unique, non-unique, and geospatial^[3] indexes. Nested fields (as described above in the ad hoc query section) can also be indexed and indexing an array type will index each element of the array.

MongoDB's query optimizer will try a number of different query plans when a query is run and select the fastest, periodically resampling. Developers can see the index being used with the `explain` function and choose a different index with the `hint` function.

Indexes can be created or removed at any time.

Aggregation

In addition to ad hoc queries, the database supports a couple of tools for aggregation, including MapReduce^[4] and a group function similar to SQL's GROUP BY.

File storage

The software implements a protocol called GridFS^[5] that is used to store and retrieve files from the database. This file storage mechanism has been used in plugins for NGINX^[6] and lighttpd.^[7]

Server-side JavaScript execution

JavaScript is the lingua franca of MongoDB and can be used in queries, aggregation functions (such as MapReduce), and sent directly to the database to be executed.

Example of JavaScript in a query:

> db.foo.find({$where : function() { return this.x == this.y; }})

Example of code sent to the database to be executed:

> db.eval(function(name) { return "Hello, "+name; }, ["Joe"])

This returns "Hello, Joe".

JavaScript variables can also be stored in the database and used by any other JavaScript as a global variable. Any legal JavaScript type, including functions and objects, can be stored in MongoDB so that JavaScript can be used to write "stored procedures."

Capped collections

MongoDB supports fixed-size collections called capped collections.^[8] A capped collection is created with a set size and, optionally, number of elements. Capped collections are the only type of collection that maintains insertion order: once the specified size has been reached, a capped collection behaves like a circular queue.

A special type of cursor, called a tailable cursor,^[9] can be used with capped collections. This cursor was named after the `tail -f` command, and does not close when it finishes returning results but continues to wait for more to be returned, returning new results as they are inserted into the capped collection.

Deployment

MongoDB can be built and installed from source, but it is more commonly installed from a binary package. Many Linux package management systems now include a MongoDB package, including CentOS and Fedora,^[10] Debian and Ubuntu,^[11] Gentoo^[12] and Arch Linux.^[13] It can also be acquired through the official website.^[14]

MongoDB uses memory-mapped files, limiting data size to 2GB on 32-bit machines (64-bit systems have a much larger data size).^[15] The MongoDB server can only be used on little-endian systems, although most of the drivers work on both little-endian and big-endian systems.

Language support

MongoDB has official drivers for:

• C^[16]
• C++^[17]
• C#^[18]
• Erlang^[19]
• Haskell^[20]
• Java^[21]
• JavaScript^[22]
• Lisp^[23]
• Perl^[24]
• PHP^[25]
• Python^[26]
• Ruby^[27]
• Scala^[28]

There are also a large number of unofficial drivers, for C# and .NET,^[18] ColdFusion,^[29] Delphi,^[30] Erlang,^[31][32] Factor,^[33] Fantom,^[34] Go,^[35] JVM languages (Clojure, Groovy,^[36] Scala, etc.),^[37] Lua,^[38] node.js,^[39] HTTP REST,^[40] Ruby,^[41] Racket,^[42] and Smalltalk.^[43]

Replication

MongoDB supports master-slave replication. A master can perform reads and writes. A slave copies data from the master and can only be used for reads or backup (not writes).

MongoDB allows developers to guarantee that an operation has been replicated to at least N servers on a per-operation basis.

Master-slave

As operations are performed on the master, the slave will replicate any changes to the data.

Example: starting a master/slave pair locally:

$ mkdir -p ~/dbs/master ~/dbs/slave
$ ./mongod --master --port 10000 --dbpath ~/dbs/master
$ ./mongod --slave --port 10001 --dbpath ~/dbs/slave --source localhost:10000

Replica sets

Replica sets are similar to master-slave, but they incorporate the ability for the slaves to elect a new master if the current one goes down.

Sharding

MongoDB scales horizontally using a system called sharding^[44] which is very similar to the BigTable and PNUTS scaling model. The developer chooses a shard key, which determines how the data in a collection will be distributed. The data is split into ranges (based on the shard key) and distributed across multiple shards. (A shard is a master with one or more slaves.)

The developer's application must know that it is talking to a sharded cluster when performing some operations. For example, a "findAndModify" query must contain the shard key if the queried collection is sharded.^[45] The application talks to a special routing process called `mongos` that looks identical to a single MongoDB server. This `mongos` process knows what data is on each shard and routes the client's requests appropriately. All requests flow through this process: it not only forwards requests and responses but also performs any necessary final data merges or sorts. Any number of `mongos` processes can be run: usually one per application server is recommended.

Management and graphical frontends

Official tools

The most powerful and useful management tool is the database shell, mongo. The shell lets developers view, insert, remove, and update data in their databases, as well as get replication information, setting up sharding, shut down servers, execute JavaScript, and more. mongo is built on SpiderMonkey, so it is a full JavaScript shell as well as being able to connect to MongoDB servers.

Administrative information can also be accessed through the admin interface: a simple html webpage that serves information about the current server status. By default, this interface is 1000 ports above the database port (http://localhost:28017) and it can be turned off with the --norest option.

mongostat is a command-line tool that displays a simple list of stats about the last second: how many inserts, updates, removes, queries, and commands were performed, as well as what percentage of the time the database was locked and how much memory it is using.

mongosniff sniffs network traffic going to and from MongoDB.

Monitoring

There are monitoring plugins available for MongoDB:

• munin^[46]
• ganglia^[47]
• scout^[48]
• cacti^[49]

GUIs

Several GUIs have been created by MongoDB's developer community to help visualize their data. Some popular ones are:

• Fang of Mongo^[50] – a web-based UI built with Django and jQuery.
• Futon4Mongo^[51] – a clone of the CouchDB Futon web interface for MongoDB.
• Mongo3^[52] – a Ruby-based interface.
• MongoHub^[53] – a native OS X application for managing MongoDB.
• Opricot^[54] – a browser-based MongoDB shell written in PHP.
• Database Master Windows based MongoDB Management Studio, supports also RDBMS.

Licensing and support

MongoDB is available for free under the GNU Affero General Public License. The language drivers are available under an Apache License.^[55]

Prominent users

• MTV Networks^[56]
• craigslist^[57]
• Disney Interactive Media Group^[58]
• Wordnik^[59]
• diaspora^[60]
• Shutterfly^[61]
• foursquare^[62]
• bit.ly^[63]
• The New York Times^[64]
• SourceForge^[65]
• Business Insider^[66]
• Etsy^[67]
• CERN LHC^[68]
• Thumbtack^[69]
• AppScale^[70]
• Uber^[71]

See also

Free software portal

• CouchDB
• NoSQL
• Structured storage

References

1. ^ MongoDB website
2. ^ MongoDB Blog - March 2010
3. ^ Geospatial indexes
4. ^ MapReduce
5. ^ GridFS
6. ^ NGINX
7. ^ lighttpd
8. ^ capped collections
9. ^ [1]
10. ^ CentOS and Fedora
11. ^ Debian and Ubuntu,
12. ^ Gentoo
13. ^ Arch Linux
14. ^ official website
15. ^ [2]
16. ^ C driver
17. ^ C++ driver
18. ^ ^{a b} C# driver
19. ^ Erlang driver
20. ^ Haskell driver
21. ^ Java driver
22. ^ JavaScript driver
23. ^ [3]
24. ^ Perl driver
25. ^ PHP driver
26. ^ Python driver
27. ^ Ruby driver
28. ^ Casbah, the officially supported Scala Driver for MongoDB
29. ^ ColdFusion driver
30. ^ Delphi
31. ^ Emongo Erlang driver
32. ^ Erlmongo Erlang driver
33. ^ Factor driver
34. ^ Fantom driver
35. ^ gomongo Go driver
36. ^ GMongo
37. ^ JVM language center
38. ^ LuaMongo
39. ^ node.js driver
40. ^ REST interface
41. ^ rmongo
42. ^ [4]
43. ^ Smalltalk driver
44. ^ sharding
45. ^ [5]
46. ^ Munin plugin
47. ^ Ganglia plugin
48. ^ Scout slow-query plugin
49. ^ Cacti plugin
50. ^ Fang of Mongo
51. ^ Futon4Mongo
52. ^ Mongo3
53. ^ MongoHub
54. ^ Opricot
55. ^ The AGPL - MongoDB Blog: May 5, 2009
56. ^ "MongoDB Powering MTV's Web Properties". 2011-05-10. http://blog.mongodb.org/post/5360007734/mongodb-powering-mtvs-web-properties. Retrieved 2011-07-06. 
57. ^ "MongoDB live at craigslist". 2011-05-16. http://blog.mongodb.org/post/5545198613/mongodb-live-at-craigslist. Retrieved 2011-07-06. 
58. ^ "Disney Central Services Storage: Leveraging Knowledge and skillsets". 2011-05-24. http://www.10gen.com/presentation/mongosf2011/disney. Retrieved 2011-07-06. 
59. ^ "12 Months with MongoDB". 2010-10-25. http://blog.wordnik.com/12-months-with-mongodb. Retrieved 2011-05-24. 
60. ^ "MongoDB - diasporatest.com". 2010-12-23. http://www.diasporatest.com/index.php/MongoDB. Retrieved 2010-12-23. 
61. ^ "Implementing MongoDB at Shutterfly - Presentation at MongoSF". 2010-04-30. http://www.10gen.com/event_mongosf_10apr30#shutterfly. Retrieved 2010-06-28. 
62. ^ "MongoDB at foursquare - Presentation at MongoNYC". 2010-05-21. http://blip.tv/file/3704098. Retrieved 2010-06-28. 
63. ^ "bit.ly user history, auto-sharded - Presentation at MongoNYC". 2010-05-21. http://blip.tv/file/3704043. Retrieved 2010-06-28. 
64. ^ Maher, Jacqueline (2010-05-25). "Building a Better Submission Form". NYTimes Open Blog. http://open.blogs.nytimes.com/2010/05/25/building-a-better-submission-form/. Retrieved 2010-06-28. 
65. ^ "How Python, TurboGears, and MongoDB are Transforming SourceForge.net". PyCon 2010. 2010-02-20. http://us.pycon.org/2010/conference/schedule/event/110/. Retrieved 2010-06-28. 
66. ^ "How This Web Site Uses MongoDB". Business Insider. 2010-11-06. http://www.businessinsider.com/how-we-use-mongodb-2009-11. Retrieved 2010-06-28. 
67. ^ "MongoDB at Etsy". Code as Craft: Etsy Developer Blog. 2010-05-19. http://codeascraft.etsy.com/2010/05/19/mongodb-at-etsy/. Retrieved 2010-06-28. ^{[dead link]}
68. ^ "Holy Large Hadron Collider, Batman!". The MongoDB NoSQL Database Blog. 2010-06-03. http://blog.mongodb.org/post/660037122/holy-large-hadron-collider-batman. Retrieved 2010-08-03. 
69. ^ "Building Our Own Tracking Engine With MongoDB". Thumbtack Blog. 2011-05-03. http://engineering.thumbtack.com/2011/05/03/building-our-own-tracking-engine-with-mongodb/. Retrieved 2011-05-15. 
70. ^ http://appscale.cs.ucsb.edu/datastores.html#mongodb
71. ^ "Node.js Meetup: Distributed Web Architectures – Curtis Chambers, Uber | JoyentCloud:". http://www.joyentcloud.com/resources/videos/node-js-office-hours-curtis-chambers-uber/. Retrieved 12 August 2011. 

Bibliography

• Banker, Kyle (March 28, 2011), MongoDB in Action (1st ed.), Manning, pp. 375, ISBN 9781935182870 
• Chodorow, Kristina; Dirolf, Michael (September 23, 2010), MongoDB: The Definitive Guide (1st ed.), O'Reilly Media, pp. 216, ISBN 9781449381561 
• Pirtle, Mitch (March 3, 2011), MongoDB for Web Development (1st ed.), Addison-Wesley Professional, pp. 360, ISBN 9780321705334 
• Hawkins, Tim; Plugge, Eelco; Membrey, Peter (September 26, 2010), The Definitive Guide to MongoDB: The NoSQL Database for Cloud and Desktop Computing (1st ed.), Apress, pp. 350, ISBN 9781430230519 

External links

• Official MongoDB Project Website
• MongoDB with ZanPHP Spanish Documentation
• mongoDB User Group on LinkedIn
• MongoDB news and articles on myNoSQL
• Eric Lai. (2009, July 1). No to SQL? Anti-database movement gains steam
• MongoDB articles on NoSQLDatabases.com
• June 2009 San Francisco NOSQL Meetup Page
• Designing for the Cloud at MIT Technology Review
• EuroPython Conference Presentation
• Non-relational data persistence in Java using MongoDB - Software Engineer at MongoDB on YouTube
• Interview with Mike Dirolf on The Changelog about MongoDB background and design decisions
• MongoMvc - A MongoDB Demo App with ASP.NET MVC
• FAQs about MongoDB