MySQL Cluster

MySQL Cluster is a technology which provides shared-nothing clustering capabilities for the MySQL database management system. It was first included in the production release of MySQL 4.1 in November 2004. It is designed to provide high availability and high performance, while allowing for nearly linear scalability.^{[citation needed]} MySQL Cluster is implemented through an additional storage engine available within MySQL called NDB or NDBCLUSTER ("NDB" stands for Network Database).

Architecture

MySQL Cluster has a few important concepts behind its design, which add both benefits and burdens.

Replication

MySQL Cluster uses synchronous replication through a two-phase commit mechanism in order to guarantee that data is written to multiple nodes upon committing the data. (This is in contrast to what is usually referred to as "MySQL Replication", which is asynchronous.) Two copies (known as replicas) of the data are required to guarantee availability; however, the cluster can be configured to store between one and four copies at any single time.

Starting with MySQL 5.1, it is also possible to replicate asynchronously between clusters; this is sometimes referred to as "MySQL Cluster Replication" or "geographical replication". See MySQL Cluster Replication.

Horizontal data partitioning

Data within NDB tables is automatically partitioned across all of the data nodes in the system. This is done based on a hashing algorithm based on the PRIMARY KEY on the table, and is transparent to the end application.

In the 5.1 release, users can define their own partitioning schemes.

Hybrid Storage

MySQL Cluster allows datasets larger than the capacity of a single machine to be stored and accessed across multiple machines.

MySQL Cluster maintains all indexed columns in distributed memory. Non indexed columns can also be maintained in distributed memory or can be maintained on disk with an in-memory page cache. Storing non indexed columns on disk allows MySQL Cluster to store datasets larger than the aggregate memory of the clustered machines.

MySQL Cluster writes Redo logs to disk for all data changes as well as check pointing data to disk regularly. This allows the cluster to consistently recover from disk after a full cluster outage. As the Redo logs are written asynchronously with respect to transaction commit, some small number of transactions can be lost if the full cluster fails. The current default asynchronous write delay is 2 seconds, and is configurable. Normal single point of failure scenarios do not result in any data loss due to the synchronous data replication within the cluster.

When a MySQL Cluster table is maintained in memory, the cluster will only access disk storage to write Redo records and checkpoints. As these writes are sequential and limited random access patterns are involved, MySQL Cluster can achieve higher write throughput rates with limited disk hardware compared to a traditional disk-based caching RDBMS.

Shared nothing

MySQL Cluster is designed to have no single point of failure. Provided that the cluster is set up correctly, any single node, system, or piece of hardware can fail without the entire cluster failing. Shared disk (SAN) is not required. The interconnects between nodes can be standard Ethernet. Gigabit Ethernet and SCI interconnects are also supported.

Implementation

MySQL Cluster uses three different types of nodes (processes) :

• Data node (ndbd/ndbmtd process): These nodes store the data.
• Management node (ndb_mgmd process): Used for configuration and monitoring of the cluster. They are required only during node startup.^{[citation needed]}
• SQL node (mysqld process): A MySQL server (mysqld) that connects to all of the data nodes in order to perform data storage and retrieval. This node type is optional; it is possible to query data nodes directly via the NDB API.

Generally, it is expected that each node will run on a separate host computer.

Versions

MySQL Cluster version numbers are no longer tied to that of MySQL Server - for example, the most recent version is MySQL Cluster 7.1^[1] even though it is based on/contains the server component from MySQL 5.1.

Higher versions of MySQL Cluster include all of the features of lower versions, plus some new features. Currently available versions^[2]:

• Ndb included in MySQL 5.1.X source tree

This is old and not maintained. Do not use

• MySQL Cluster 6.2 based on MySQL 5.1.A

First 'telco' or 'carrier grade edition' release. Supports 255 nodes, online table alter, replication latency and throughput enhancements etc.

• MySQL Cluster 6.3 based on MySQL 5.1.B

Includes compressed backup + LCP, circular replication support, conflict detection/resolution, table optimization etc.

• MySQL Cluster 7.0 based on MySQL 5.1.C

Includes multi-threaded data nodes (ndbmtd), Transactional DDL, Windows support.

• MySQL Cluster 7.1 based on MySQL 5.1.D

Includes ClusterJ and ClusterJPA connectors

Limitations

In the 5.1 release, non-indexed columns can be stored on disk and do not require dedicated RAM. However, in 5.0 all indexes as well as all data are still in main memory.

In the 5.1 release, a maximum of 255 nodes can belong to a single MySQL Cluster with up to 48 of those being data nodes. In the 5.0 release the total number of nodes cannot exceed 63. It is possible to change this at compile time, but that has not been thoroughly tested at this point.^[when?]

Versions up to and including 5.0 do not have support for variable-width columns, instead using the entire storage width of the column declaration, effectively making a VARCHAR(255) column into a CHAR(255) column. MySQL 5.1 adds true VARCHAR support for NDB tables.

The foreign key construct is ignored, just as it is in MyISAM tables.

Beginning with MySQL 5.0.6, the maximum number of metadata objects has increased to 20320.^[3] This includes database tables, system tables, and indexes.

Other limitations are listed here

MySQL Cluster Limitations 5.0.

MySQL Cluster Limitations 5.1.

Requirements

Minimum system requirements are as follows (for each node) 3 Machines Minimum

• OS: Linux (Red Hat, SUSE), Solaris, Mac OS X, Windows
• CPU: Intel/AMD x86
• Memory: 512MB RAM
• HDD: 3GB
• Network: 1+ nodes (Standard Ethernet - TCP/IP)

Preferred system requirements are as follows (for each node) 4 Machines Preferred

• OS: Linux (Red Hat, SUSE), Solaris, Mac OS X, Windows
• CPU: 2x Intel Xeon, AMD Opteron, Sun SPARC
• Memory: 16GB RAM
• HDD: 4x 36GB SCSI (RAID 1 Controller)
• Network: 1-8 Nodes (Gigabit Ethernet); 8+ Nodes (Dedicated Cluster Interconnect e.g. SCI)

History

MySQL AB acquired the technology behind MySQL Cluster from Alzato, a small venture company started by Ericsson. NDB was originally designed for the telecom market, with its High availability and high performance requirements. The original press release is still available from MySQL AB

NDB has since been integrated into the MySQL product, with its first release being in MySQL 4.1.

Reference manual

• Official Cluster Documentation

References

• MySQL Cluster Reference Manual 5.1

1. ^ http://www.mysql.com/about/legal/lifecycle/cluster.html#calendar
2. ^ http://johanandersson.blogspot.com/2008/02/mysql-cluster-features-what-they-are.html
3. ^ MySQL :: MySQL 5.0 Reference Manual :: 16.12.10 Previous MySQL Cluster Issues Resolved in MySQL 5.0

External links

MySQL AB

• MySQL Webinar on Clustering
• Cluster Product Overview
• MySQL Cluster API Developers' Guide
• MySQL Cluster tutorial
• MySQL Cluster Blog

Other

• Install and Management Tool for MySQL Cluster
• Generate production class configurations and scripts for MySQL Cluster
• MySQL Cluster monitoring tool
• Design and Modelling of a Parallel Data Server for Telecom Applications (1997) Original MySQL Cluster design motivation.