Sun Grid Engine

Infobox Software
name = Sun Grid Engine

developer = Sun Microsystems in association with the community
latest release version = [http://gridengine.sunsource.net/downloads/61/download.html 6.1 update 5]
latest release date = release date|2008|08|05
latest preview version = [http://gridengine.sunsource.net/downloads/62/download.html 6.2 Beta2]
latest preview date = release date|2008|06|18
operating system = Cross-platform
genre = Grid computing
license = SISSL
website = http://gridengine.sunsource.net

Sun Grid Engine (SGE), previously known as CODINE (COmputing in DIstributed Networked Environments) or GRD (Global Resource Director), [cite web|url=http://blogs.sun.com/templedf/entry/a_little_history_lesson|title=A Little History Lesson|date=2006-06-23|publisher=Sun Microsystems] is an open source batch-queuing system, supported by Sun Microsystems. Sun also sells a commercial product based on SGE, also known as N1 Grid Engine (N1GE).

SGE is typically used on a computer farm or computer cluster and is responsible for accepting, scheduling, dispatching, and managing the remote execution of large numbers of standalone, parallel or interactive user jobs. It also manages and schedules the allocation of distributed resources such as processors, memory, disk space, and software licenses.

SGE is the foundation of the Sun Grid utility computing system, made available over the Internet in the United States in 2006, [cite web|url=http://www.sun.com/smi/Press/sunflash/2006-03/sunflash.20060322.1.xml|title=World's First Utility Grid Comes Alive on the Internet|date=2006-03-22|publisher=Sun Microsystems] later becoming available in many other countries.

Features

Features of SGE 6.1 include:

* Multiple advanced scheduling algorithms allow powerful policy-based resource allocation
* Cluster queues
* Job and scheduler fault tolerance
* Job checkpointing
* Job arrays and job tasks
* DRMAA (Job API)
* Resource reservation
* XML status reporting ("qstat" and "qhost"), and the "xml-qstat" web interface
* Parallel jobs (MPI, PVM, OpenMP), and scalable parallel job startup with qrsh [cite web
url = http://gridengine.sunsource.net/servlets/ReadMsg?listName=users&msgNo=9446
title = Long delay when submitting large jobs (mailing list message)
accessdate = 2007-12-25
publisher = Sun Microsystems]
* Usage accounting
* accounting and reporting (ARCO)
* parallel make: distmake, dmake (Sun Studio), and SGE's own qmake
* FLEXlm integration [cite web
url = http://wiki.gridengine.info/wiki/index.php/Olesen-FLEXlm-Integration
title = Olesen-FLEXlm-Integration
accessdate = 2007-12-25
publisher = wiki.gridengine.info] and multi-cluster software license management with "LicenseJuggler" [cite web
url = http://wiki.gridengine.info/wiki/index.php/LicenseJuggler
title = LicenseJuggler
accessdate = 2007-12-26
publisher = wiki.gridengine.info]

Features new in version 6.2

* Advance reservation
* Array job interdependencies
* Enhanced remote execution (without using external rshd/rlogind/sshd processes)
* Multi-clustering [cite web| url = http://hedeby.sunsource.net | title = Hedeby Project home| accessdate = 2008-01-25| publisher = Sun Microsystems]
* Daemons managed by the Service Management Facility on Solaris
* Pseudo TTY (pty) support for interactive jobs

Platforms

SGE runs on multiple platforms, including:
* AIX
* BSD - FreeBSD, NetBSD, OpenBSD
* HP-UX
* IRIX
* Linux
* Mac OS X
* Solaris
* SUPER-UX
* Tru64
* Windows (as execution hosts only)
* Z/OS (in progress)

upport and training

Sun provides support contracts [cite web | url = http://store.sun.com/CMTemplate/CEServlet?process=SunStore&cmdViewProduct_CP&catid=115672 | title = Sun Store Grid Engine Entitlement Purchase | accessdate = 2008-03-03 | publisher = Sun Microsystems ] for the commercial version of Grid Engine on most UNIX platforms and Windows. Professional services, consulting, training, and support are also provided by Sun Partners. [cite web | url = http://www.sun.com/software/gridware/partners/index.xml | title = Sun Grid Engine 6 Partners | accessdate = 2007-12-14 | publisher = Sun Microsystems ] Sun partners with Georgetown University to deliver Grid Engine administration classes. [cite web | url = http://blogs.sun.com/templedf/entry/advanced_sun_grid_engine_configuration | title = Advanced Sun Grid Engine Configuration and Administration Class | accessdate = 2007-12-14 | publisher = Sun Microsystems ] "The Bioteam" runs short SGE training workshops that are 1 or 2 days long. [cite web | url = http://blog.bioteam.net/category/training/ | title = Training | accessdate = 2008-03-24 | publisher = The Bioteam Inc.]

Users can get community support on the Grid Engine mailing lists. [cite web | url = http://gridengine.sunsource.net/maillist.html | title = Grid Engine Mail Lists | accessdate = 2008-01-23 | publisher = Sun Microsystems ]

Grid Engine Workshops were held in 2002, 2003, and 2007 in Regensburg, Germany. [cite web | url = http://gridengine.sunsource.net/workshop.html | title = Grid Engine Workshops | accessdate = 2007-12-14 | publisher = Sun Microsystems ]

Prominent users

Notable deployments of SGE include:
* Sun Grid
* the TSUBAME supercomputer at the Tokyo Institute of Technology, [cite web
url = http://www.sun.com/blueprints/0607/820-1695.html
title = Sun N1 Grid Engine Software and the Tokyo Institute of Technology Super Computer Grid
accessdate = 2007-11-16
publisher = Sun Microsystems] which was number 7 on June 2006 TOP500 list.
* Ranger at the Texas Advanced Computing Center (TACC). Ranger has 62,976 processor cores in 3,936 nodes and a peak performance of 504TFlops. [cite web
url = http://www.tacc.utexas.edu/resources/hpcsystems/#ranger
title = TACC > HPC Systems
accessdate = 2007-12-13
publisher = The University of Texas at Austin] [cite web
url = http://blogs.sun.com/marchamilton/entry/more_ranger_facts_and_figures
title = More Ranger Facts and Figures
accessdate = 2008-02-12
publisher = Sun Microsystems] Ranger is currently the 4th most powerful TOP500 supercomputer. [cite web|url=http://top500.org/list/2008/06/100|title=TOP500 List - June 2008|date=2006-06-18|publisher=TOP500.Org]

History

In 2000, Sun acquired Gridware, Inc. a privately owned commercial vendor of advanced computing resource management software with offices in San Jose, Calif., and Regensburg, Germany. [cite web|url=http://www.sun.com/smi/Press/sunflash/2000-07/sunflash.20000724.3.xml|title=Gridware's resource management software increases efficiency and productivity in compute-intensive technical computing environments|date=2000-07-24|publisher=Sun Microsystems] Later that year, Sun offered a free version of Gridware for Solaris and Linux, and renamed the product Sun Grid Engine.

In 2001, Sun made the source code available, [cite web|url=http://www.sun.com/smi/Press/sunflash/2001-07/sunflash.20010723.1.xml|title=Sun Microsystems makes SUN GRID ENGINE software available to open source community|date=2001-07-23|publisher=Sun Microsystems] and adopted the open source development model. Ports for Mac OS X and *BSD were contributed by the non-Sun open source developers.

Other Grid Engine based products

* Sun Constellation System
* Sun Visualization System
* Rocks Cluster Distribution
* EGEE [cite web|url=http://pubs.doc.ic.ac.uk/egee-sge-integration/|title=Sun Grid Engine, a new scheduler for EGEE middleware|date=2000-12-29|publisher=Imperial College]
* Univa UD's UniCluster
* BioTeam's iNquiry

Add-on software

A number of SGE add-ons are available:
* Solaris Cluster integration [cite web|url=http://docs.sun.com/app/docs/doc/819-3064/cacjgdbc?a=view|title=Installing and Configuring Sun Cluster HA for Sun Grid Engine|date=2008-02-15|publisher=Sun Microsystems]
* "Service Domain Management" module in order to meet service level objectives
* Transfer-queue Over Globus (TOG). Globus does not officially support SGE, but the [http://www.lesc.ic.ac.uk/projects/epic-gt-sge.html London e-Science Center] has created a package and provides instructions on how to configure a Globus Toolkit server so that it can submit jobs for execution on a local Sun Grid Engine installation.
* JOb Scheduling Hierarchically (JOSH)

References

ee also

*Sun xVM - Sun's datacenter automation tool
*Open Grid Forum

External links

* [http://gridengine.sunsource.net/ gridengine: Home] - Official project page
* [http://www.sun.com/software/gridware/ Sun N1 Grid Engine] - Official page for the commercial version
* [http://docs.sun.com/app/docs/coll/1017.3/ Sun Grid Engine 6.0 Collection] - Documentation Collection for Grid Engine 6.0
* [http://docs.sun.com/app/docs/coll/1017.4/ Sun Grid Engine 6.1 Collection] - Documentation Collection for Grid Engine 6.1
* [http://wikis.sun.com/display/GridEngine Sun Grid Engine Information Center] - Documentation Collection for Grid Engine 6.2
* [http://www.gridengine.info/ gridengine.info] - Unofficial Grid Engine resources blog
* [http://wiki.gridengine.info/wiki wiki.gridengine.info] - Unofficial Grid Engine resources wiki - including information about integration of applications
* [http://bioteam.net/dag/gridengine-6-features.html "Understanding the differences between Grid Engine 5.3, 6.0 and Sun N1 Grid Engine 6 (N1GE 6)" ]
* [http://www.xml-qstat.org/ xml-qstat project page] - Web based status monitoring of Grid Engine 6.x systems
* [http://blogs.sun.com/roller/page/templedf?catname=%2FGrid Good Tips] - several blog entries on getting started with Grid Engine
* [http://www.gridwisetech.com/content/view/14/17/lang,en/ Grid Engine-Globus Toolkit adapter]
* [http://fiehnlab.ucdavis.edu/staff/wohlgemuth/binbase/cluster/index_html/document_view BinBase Cluster System] - an API to access the SGE from java and run java based jobs on the cluster.
* [http://search.cpan.org/~linsalrob/ScheduleSGE-0.02/SGE.pm Schedule::SGE] - An API to access the SGE using Perl