Dependency hell

Dependency hell is a colloquial term for the frustration of some software users who have installed software packages which have dependencies on specific versions of other software packages.^[1] This was mainly attributable to old Linux package managers. Current package managers have largely solved this problem by automatically resolving and downloading dependencies.

Overview

Often, rather than "reinventing the wheel", software is designed to take advantage of other software components that are already available, or have already been designed and implemented for use elsewhere. This could be compared to how people building a house might buy off-the-shelf components, such as bricks, windows, and doors, rather than producing everything themselves.

Even for a builder, it can be a problem if a building is designed for a certain door type, and only doors with different specifications are available. However, in the software world, where components evolve rapidly and depend significantly on one another, this problem becomes more pronounced.^[2]

The issue of dependency hell may be regarded as an anti-pattern, where the fault lies less with the suppliers of the products than with the framework into which they have to fit.

Platform-specific

On specific computing platforms, "dependency hell" often goes by a local specific name, generally the name of components.

• DLL hell - A form of dependency hell occurring on Microsoft Windows.
• Extension conflict - A form of dependency hell occurring on older versions of Mac OS^{[citation needed]}.
• JAR hell - A form of dependency hell occurring in the Java Runtime Environment.
• RPM hell - A form of dependency hell occurring in the Red Hat distribution of Linux and other distributions that use RPM as a package manager.^[3]

Problems

Dependency hell takes several forms:

many dependencies

An application depends on many libraries, requiring lengthy downloads, large amounts of disk space, and not being very portable (all libraries must be ported for the application to be ported). It can also be difficult to locate all the dependencies, which can be fixed by having a repository (see below). This is partly inevitable; an application built on a given platform (such as Java) requires that platform to be installed, but further applications do not require it. This is a particular problem if an application uses a small part of a big library (which can be solved by refactoring), or a simple application relies on many libraries.

long chains of dependencies

app depends on liba, which depends on libb, ..., which depends on libz. This is distinct from "many dependencies" if the dependencies must be resolved manually (e.g., on attempting to install app, you are prompted to install liba first. On attempting to install liba, you are then prompted to install libb.). Sometimes, however, during this long chain of dependencies conflicts arise, where two different versions of the same package are required^[4] (see conflicting dependencies below). These long chains of dependencies can be solved by having a package manager that resolves all dependencies automatically. Other than being a hassle (to resolve all the dependencies manually), manual resolution can mask dependency cycles or conflicts.

conflicting dependencies

If app1 depends on libfoo 1.2, and app2 depends on libfoo 1.3, and different versions of libfoo cannot be simultaneously installed, then app1 and app2 cannot simultaneously be used (or installed, if the installer checks dependencies). When possible, this is solved by allowing simultaneous installations of the different dependencies. Alternatively, the existing dependency, along with all software that depends on it, must be uninstalled in order to install the new dependency. A problem on Linux systems with installing packages from a different distributor (which is not recommended or even supposed to work) is that the resulting long chain of dependencies may lead to a conflicting version of glibc, the single most important library. If this happens, the user will be prompted to uninstall thousands of packages.

circular dependencies

If appX, version 1 depends on app2, which depends on app3, which depends on app4, which depends on the original appX, version 0, then, in systems such as RPM or dpkg, the user must install all packages simultaneously.^[5] - hence on Linux circular dependencies are often the result of a user misunderstanding the packaging system.^{[citation needed]} On other platforms, however, the packaging system may not be able to resolve the circular dependency.

Solutions

Version Numbering

The most obvious (and very common) solution to this problem is to have a standardised numbering system, wherein software uses a specific number for each version (aka major version), and also a subnumber for each revision (aka minor version), e.g.: 10.1, or 5.7. The major version only changes when programs that used that version will no longer be compatible. The minor version might change with even a simple revision that does not prevent other software from working with it. In cases like this, software packages can then simply request a component that has a particular major version, and any minor version (greater than or equal to a particular minor version). As such, they will continue to work, and dependencies will be resolved successfully, even if the minor version changes.

Smart Package Management

Some package managers can perform smart upgrades, in which interdependent software components are upgraded at the same time, thereby resolving the major number incompatibility issue too.

Many current Linux distributions have also implemented repository-based package management systems to try to solve the dependency problem. These systems are a layer on top of the RPM, dpkg, or other packaging systems that are designed to automatically resolve dependencies by searching in predefined software repositories. Typically these software repositories are FTP sites or websites, directories on the local computer or shared across a network or, much less commonly, directories on removable media such as CDs or DVDs. This eliminates dependency hell for software packaged in those repositories, which are typically maintained by the Linux distribution provider and mirrored worldwide. Although these repositories are often huge it is not possible to have every piece of software in them, so dependency hell can still occur. In all cases, dependency hell is still faced by the repository maintainers. Examples of these systems include Apt, Yum, Urpmi, ZYpp, Portage, Pacman and others.

Installer Options

Because different pieces of software have different dependencies, it is possible to get into a vicious circle of dependency requirements, or (possibly worse) an ever-expanding tree of requirements, as each new package demands several more be installed. Systems such as Debian's APT can resolve this by presenting the user with a range of solutions, and allowing the user to accept or reject the solutions, as desired.

The Haskell Compiler GHC is an example of a circular dependency. To compile it, you need GHC. It can be solved by downloading a binary version of GHC, and compiling the new version of GHC with this binary version. This is not uncommon; many large base-level projects such as gcc require self-compilation, but this is not a problem as all operating systems ship with binaries of these.

Portable Applications

An application that is coded to have all of its necessary components included, or is designed to keep all necessary files within its own directory, will not create a dependency problem. Where possible, this is normally the preferred method^{[citation needed]}.

Easy Adaptability in Programming

If application software is designed in such a way that its programmers are able to easily adapt the interface layer that deals with the OS, window manager or desktop environment to new or changing standards, then the programmers would only have to monitor notifications from the environment creators or component library designers and quickly adjust their software with updates for their users, all with minimal effort and a lack of costly and time-consuming redesign. This method would encourage programmers to pressure those upon whom they depend to maintain a reasonable notification process that is not onerous to anyone involved.

Software appliances

Another approach to avoiding dependency issues is to deploy applications as a software appliance. A software appliance encapsulates dependencies in a pre-integrated self-contained unit such that users no longer have to worry about resolving software dependencies. Instead the burden is shifted to developers of the software appliance.

Portable applications

An application (or version of an existing conventional application) that is completely self-contained and requires nothing to be already installed. These are often able to run independently of the system to which they are connected.

Examples

One modern example of dependency hell on Microsoft Windows and Mac OS X is the Gecko Runtime Engine, or GRE, used by Mozilla projects. Each product released from the Mozilla foundation includes its own version of the complete Gecko Runtime Engine, due to the volatile nature of the programming interfaces used. Thus, if a user installs Thunderbird, Firefox, and Sunbird, there will be three copies of GRE on the machine. These may or may not be compatible, depending on when the GRE source tree was forked. Some external projects like Epiphany depend on specific versions of the Mozilla Suite to use GRE, and break if a different version is installed; while others such as Nvu bring their own copy of GRE. The duplication of the GRE is actually a work-around to the core problem of dependency hell.

By statically linking Gecko, the Mozilla developers avoid potential dependency hell for their binary packages, at the cost of increased disk and memory usage. Hard disk space comes quite cheap these days, therefore increased disk usage in itself is less of a problem than it once was, but the amount of extra non-shareable memory used is still considerable. (Also note that limitations on backing store size are returning with the use of solid-state drives for portable computers.) Tools that are statically linked, such as bash or make, will never complain about a missing shared object when the C library (glibc) is upgraded. Both approaches have advantages and disadvantages.

Further, many modern Linux distributions avoid this particular dependency problem by compiling Firefox, Thunderbird, etc. as merely a front-end to the XULRunner package,^[6] thereby necessitating only one copy of the runtime being installed. Another solution that stops dependency hell is made possible because a distribution's software repository can keep all of the user's software in sync, unlike on Windows and the Mac OS.

See also

• Coupling - Forms of dependency among software artifacts
• Configuration management - Techniques and tools for managing software versions
• Software appliance

References

1. ^ "Urban Dictionary: Dependency Hell". www.urbandictionary.com. 2004-05-24. http://dependecy-hell.urbanup.com/680093. Retrieved 2010-06-28. 
2. ^ Donald, James (2003-01-25). Improved Portability of Shared Libraries. Princeton University. http://www.princeton.edu/~jdonald/research/shared_libraries/cs518_report.pdf. Retrieved 2010-04-09. ^{[dead link]}
3. ^ Weinstein, Paul (2003-09-11). "Is Linux Annoying?". linuxdevcenter.com. http://linuxdevcenter.com/pub/a/linux/2003/09/11/linux_annoyances.html. Retrieved 2010-04-10. 
4. ^ Stevens, Al (2001-05-01). "It's Good Work When You Can Find It; The Dependency Carousel". J-DDJ (www.drdobbs.com/blog) 26 (5): 121–124. ISSN 1044-789X. http://www.drdobbs.com/blog/archives/2008/12/its_good_work_w.html. Retrieved 2010-04-10. 
5. ^ Dependency hell
6. ^ "XULRunner homepage". mozilla foundation. https://developer.mozilla.org/en/XULRunner. 

External links

• Dependency walker
• MacDependency
• Implicit dependency
• Context independence