Dependency hell - AbsoluteAstronomy.com

Dependency hell is a colloquial term for the frustration of some software users who have installed software packages

Software package (installation)

In package management systems, which are commonly used with Linux-based operating systems, a package is a specific piece of software which the system can install and uninstall....

which have dependencies on specific versions of other software packages. This was mainly attributable to old Linux

Linux

Linux is a Unix-like computer operating system assembled under the model of free and open source software development and distribution. The defining component of any Linux system is the Linux kernel, an operating system kernel first released October 5, 1991 by Linus Torvalds...

package managers. Current package managers have largely solved this problem by automatically resolving and downloading dependencies.

Overview

Often, rather than "reinventing the wheel

Reinventing the wheel

To reinvent the wheel is to duplicate a basic method that has already previously been created or optimized by others.The inspiration for this idiomatic metaphor lies in the fact that the wheel is the archetype of human ingenuity, both by virtue of the added power and flexibility it affords its...

", software is designed to take advantage of other software components that are already available, or have already been designed and implemented for use elsewhere. This could be compared to how people building a house might buy off-the-shelf

Commercial off-the-shelf

In the United States, Commercially available Off-The-Shelf is a Federal Acquisition Regulation term defining a nondevelopmental item of supply that is both commercial and sold in substantial quantities in the commercial marketplace, and that can be procured or utilized under government contract...

components, such as bricks, windows, and doors, rather than producing everything themselves.

Even for a builder, it can be a problem if a building is designed for a certain door type, and only doors with different specifications are available. However, in the software world, where components evolve rapidly and depend significantly on one another, this problem becomes more pronounced.

The issue of dependency hell may be regarded as an anti-pattern

Anti-pattern

In software engineering, an anti-pattern is a pattern that may be commonly used but is ineffective and/or counterproductive in practice.The term was coined in 1995 by Andrew Koenig,...

, where the fault lies less with the suppliers of the products than with the framework into which they have to fit.

Platform-specific

On specific computing platforms, "dependency hell" often goes by a local specific name, generally the name of components.

DLL hell
DLL hell
In computing, DLL Hell is a term for the complications that arise when working with dynamic link libraries used with Microsoft Windows operating systems, particularly legacy 16-bit editions which all run in a single memory space....

- A form of dependency hell occurring on Microsoft Windows
Microsoft Windows
Microsoft Windows is a series of operating systems produced by Microsoft.Microsoft introduced an operating environment named Windows on November 20, 1985 as an add-on to MS-DOS in response to the growing interest in graphical user interfaces . Microsoft Windows came to dominate the world's personal...

.
Extension conflict
Extension conflict
Extension conflicts were sometimes a problem on Apple Macintosh computers running versions of Mac OS prior to X, especially System 7. Extensions were bundles of code that extended the operating system's capabilities by directly patching OS calls, thus receiving control instead of the operating...

- A form of dependency hell occurring on older versions of Mac OS
Mac OS
Mac OS is a series of graphical user interface-based operating systems developed by Apple Inc. for their Macintosh line of computer systems. The Macintosh user experience is credited with popularizing the graphical user interface...

.
JAR hell - A form of dependency hell occurring in the Java Runtime Environment.
RPM hell - A form of dependency hell occurring in the Red Hat
Red Hat
Red Hat, Inc. is an S&P 500 company in the free and open source software sector, and a major Linux distribution vendor. Founded in 1993, Red Hat has its corporate headquarters in Raleigh, North Carolina with satellite offices worldwide....

distribution of Linux
Linux
Linux is a Unix-like computer operating system assembled under the model of free and open source software development and distribution. The defining component of any Linux system is the Linux kernel, an operating system kernel first released October 5, 1991 by Linus Torvalds...

and other distributions that use RPM
RPM Package Manager
RPM Package Manager is a package management system. The name RPM variously refers to the .rpm file format, files in this format, software packaged in such files, and the package manager itself...

as a package manager.

Problems

Dependency hell takes several forms:

many dependencies

An application depends on many libraries, requiring lengthy downloads, large amounts of disk space, and not being very portable (all libraries must be ported for the application to be ported). It can also be difficult to locate all the dependencies, which can be fixed by having a repository (see below). This is partly inevitable; an application built on a given platform

Platform (computing)

A computing platform includes some sort of hardware architecture and a software framework , where the combination allows software, particularly application software, to run...

(such as Java) requires that platform to be installed, but further applications do not require it. This is a particular problem if an application uses a small part of a big library (which can be solved by refactoring

Refactoring

Code refactoring is "disciplined technique for restructuring an existing body of code, altering its internal structure without changing its external behavior", undertaken in order to improve some of the nonfunctional attributes of the software....

), or a simple application relies on many libraries.

long chains of dependencies

app depends on liba, which depends on libb, ..., which depends on libz. This is distinct from "many dependencies" if the dependencies must be resolved manually (e.g., on attempting to install app, you are prompted to install liba first. On attempting to install liba, you are then prompted to install libb.). Sometimes, however, during this long chain of dependencies conflicts arise, where two different versions of the same package are required (see conflicting dependencies below). These long chains of dependencies can be solved by having a package manager that resolves all dependencies automatically. Other than being a hassle (to resolve all the dependencies manually), manual resolution can mask dependency cycles or conflicts.

conflicting dependencies

If app1 depends on libfoo 1.2, and app2 depends on libfoo 1.3, and different versions of libfoo cannot be simultaneously installed, then app1 and app2 cannot simultaneously be used (or installed, if the installer checks dependencies). When possible, this is solved by allowing simultaneous installations of the different dependencies. Alternatively, the existing dependency, along with all software that depends on it, must be uninstalled in order to install the new dependency. A problem on Linux systems with installing packages from a different distributor (which is not recommended or even supposed to work) is that the resulting long chain of dependencies may lead to a conflicting version of glibc

GNU C Library

The GNU C Library, commonly known as glibc, is the C standard library released by the GNU Project. Originally written by the Free Software Foundation for the GNU operating system, the library's development has been overseen by a committee since 2001, with Ulrich Drepper from Red Hat as the lead...

, the single most important library. If this happens, the user will be prompted to uninstall thousands of packages.

circular dependencies

If appX, version 1 depends on app2, which depends on app3, which depends on app4, which depends on the original appX, version 0, then, in systems such as RPM or dpkg, the user must install all packages simultaneously. - hence on Linux circular dependencies are often the result of a user misunderstanding the packaging system. On other platforms, however, the packaging system may not be able to resolve the circular dependency.

Solutions

Version Numbering

The most obvious (and very common) solution to this problem is to have a standardised numbering system, wherein software uses a specific number for each version (aka major version), and also a subnumber for each revision (aka minor version), e.g.: 10.1, or 5.7. The major version only changes when programs that used that version will no longer be compatible. The minor version might change with even a simple revision that does not prevent other software from working with it. In cases like this, software packages can then simply request a component that has a particular major version, and any minor version (greater than or equal to a particular minor version). As such, they will continue to work, and dependencies will be resolved successfully, even if the minor version changes.

Smart Package Management

Some package managers can perform smart upgrades, in which interdependent software components are upgraded at the same time, thereby resolving the major number incompatibility issue too.

Many current Linux

Linux

distributions have also implemented repository

Digital library

A digital library is a library in which collections are stored in digital formats and accessible by computers. The digital content may be stored locally, or accessed remotely via computer networks...

-based package management systems to try to solve the dependency problem. These systems are a layer on top of the RPM

RPM Package Manager

RPM Package Manager is a package management system. The name RPM variously refers to the .rpm file format, files in this format, software packaged in such files, and the package manager itself...

, dpkg

Dpkg

dpkg is the software at the base of the Debian package management system. dpkg is used to install, remove, and provide information about .deb packages....

, or other packaging systems that are designed to automatically resolve dependencies by searching in predefined software repositories. Typically these software repositories are FTP

File Transfer Protocol

File Transfer Protocol is a standard network protocol used to transfer files from one host to another host over a TCP-based network, such as the Internet. FTP is built on a client-server architecture and utilizes separate control and data connections between the client and server...

sites or websites, directories on the local computer or shared across a network

Computer network

A computer network, often simply referred to as a network, is a collection of hardware components and computers interconnected by communication channels that allow sharing of resources and information....

or, much less commonly, directories on removable media such as CDs or DVDs. This eliminates dependency hell for software packaged in those repositories, which are typically maintained by the Linux distribution provider and mirrored worldwide. Although these repositories are often huge it is not possible to have every piece of software in them, so dependency hell can still occur. In all cases, dependency hell is still faced by the repository maintainers. Examples of these systems include Apt

Advanced Packaging Tool

The Advanced Packaging Tool, or APT, is a free user interface that works with core libraries to handle the installation and removal of software on the Debian GNU/Linux distribution and its variants...

, Yum, Urpmi

Urpmi

urpmi is a Mandriva Linux package management tool for installing, removing, updating and querying software packages of local or remote media. It wraps around the package manager rpm with the aim that the user should not have to suffer the often-encountered dependency hell...

, ZYpp

ZYpp

ZYpp is a package management engine that powers Linux applications like YaST, Zypper and the openSUSE/SUSE Linux Enterprise implementation of PackageKit. Unlike other common package managers, it provides a powerful satisfiability solver to compute package dependencies and a convenient package...

, Portage

Portage (software)

Portage is a package management system used by Gentoo Linux, based on the concept of ports collections. Gentoo is sometimes referred to as a meta-distribution due to the extreme flexibility of Portage, which makes it operating-system-independent. The Gentoo/Alt project is concerned with using...

, Pacman and others.

Installer Options

Because different pieces of software have different dependencies, it is possible to get into a vicious circle

Virtuous circle and vicious circle

A virtuous circle and a vicious circle are economic terms. They refer to a complex of events that reinforces itself through a feedback loop. A virtuous circle has favorable results, while a vicious circle has detrimental results...

of dependency requirement

Requirement

In engineering, a requirement is a singular documented physical and functional need that a particular product or service must be or perform. It is most commonly used in a formal sense in systems engineering, software engineering, or enterprise engineering...

s, or (possibly worse) an ever-expanding tree

Tree structure

A tree structure is a way of representing the hierarchical nature of a structure in a graphical form. It is named a "tree structure" because the classic representation resembles a tree, even though the chart is generally upside down compared to an actual tree, with the "root" at the top and the...

of requirements, as each new package demands several more be installed. Systems such as Debian's APT

Advanced Packaging Tool

can resolve this by presenting the user with a range of solutions, and allowing the user to accept or reject the solutions, as desired.

The Haskell Compiler GHC is an example of a circular dependency. To compile it, you need GHC. It can be solved by downloading a binary version of GHC, and compiling the new version of GHC with this binary version. This is not uncommon; many large base-level projects such as gcc require self-compilation, but this is not a problem as all operating systems ship with binaries of these.

Portable Applications

An application that is coded to have all of its necessary components included, or is designed to keep all necessary files within its own directory, will not create a dependency problem. Where possible, this is normally the preferred method.

Easy Adaptability in Programming

If application software is designed in such a way that its programmers are able to easily adapt the interface layer that deals with the OS, window manager or desktop environment to new or changing standards, then the programmers would only have to monitor notifications from the environment creators or component library designers and quickly adjust their software with updates for their users, all with minimal effort and a lack of costly and time-consuming redesign. This method would encourage programmers to pressure those upon whom they depend to maintain a reasonable notification process that is not onerous to anyone involved.

Software appliances

Another approach to avoiding dependency issues is to deploy applications as a software appliance

Software appliance

A software appliance is a software application that might be combined with just enough operating system for it to run optimally on industry standard hardware or in a virtual machine....

. A software appliance encapsulates dependencies in a pre-integrated self-contained unit such that users no longer have to worry about resolving software dependencies. Instead the burden is shifted to developers of the software appliance.

Portable applications

An application (or version of an existing conventional application) that is completely self-contained and requires nothing to be already installed. These are often able to run independently of the system to which they are connected.

Examples

One modern example of dependency hell on Microsoft Windows and Mac OS X is the Gecko Runtime Engine, or GRE, used by Mozilla

Mozilla

Mozilla is a term used in a number of ways in relation to the Mozilla.org project and the Mozilla Foundation, their defunct commercial predecessor Netscape Communications Corporation, and their related application software....

projects. Each product released from the Mozilla foundation includes its own version of the complete Gecko Runtime Engine, due to the volatile nature of the programming interfaces used. Thus, if a user installs Thunderbird

Mozilla Thunderbird

Mozilla Thunderbird is a free, open source, cross-platform e-mail and news client developed by the Mozilla Foundation. The project strategy is modeled after Mozilla Firefox, a project aimed at creating a web browser...

, Firefox

Mozilla Firefox

Mozilla Firefox is a free and open source web browser descended from the Mozilla Application Suite and managed by Mozilla Corporation. , Firefox is the second most widely used browser, with approximately 25% of worldwide usage share of web browsers...

, and Sunbird, there will be three copies of GRE on the machine. These may or may not be compatible, depending on when the GRE source tree was forked

Fork (software development)

In software engineering, a project fork happens when developers take a legal copy of source code from one software package and start independent development on it, creating a distinct piece of software...

. Some external projects like Epiphany

Epiphany (web browser)

Epiphany is an open source web browser for the GNOME desktop environment. The browser is a descendant of Galeon, and was created after developer disagreements about Galeon's growing complexity...

depend on specific versions of the Mozilla Suite to use GRE, and break if a different version is installed; while others such as Nvu

Nvu

Nvu is a WYSIWYG HTML editor, based on the Composer component of Mozilla Application Suite and Gecko 1.7. It is a common WYSIWYG editor for Linux, Microsoft Windows and Macintosh, intended to be an open source equivalent to proprietary software like Microsoft Expression Web and Adobe Dreamweaver...

bring their own copy of GRE. The duplication of the GRE is actually a work-around to the core problem of dependency hell.

By statically linking

Static library

In computer science, a static library or statically-linked library is a set of routines, external functions and variables which are resolved in a caller at compile-time and copied into a target application by a compiler, linker, or binder, producing an object file and a stand-alone executable...

Gecko, the Mozilla developers avoid potential dependency hell for their binary packages, at the cost of increased disk and memory usage.
Hard disk space comes quite cheap these days, therefore increased disk usage in itself is less of a problem than it once was, but the amount of extra non-shareable memory used is still considerable. (Also note that limitations on backing store size are returning with the use of solid-state drive

Solid-state drive

A solid-state drive , sometimes called a solid-state disk or electronic disk, is a data storage device that uses solid-state memory to store persistent data with the intention of providing access in the same manner of a traditional block i/o hard disk drive...

s for portable computers.) Tools that are statically linked, such as bash or make, will never complain about a missing shared object when the C library

C standard library

The C Standard Library is the standard library for the programming language C, as specified in the ANSI C standard.. It was developed at the same time as the C POSIX library, which is basically a superset of it...

(glibc) is upgraded.
Both approaches have advantages and disadvantages.

Further, many modern Linux distributions avoid this particular dependency problem by compiling Firefox, Thunderbird, etc. as merely a front-end to the XULRunner

XULRunner

XULRunner is a runtime environment developed by the Mozilla Foundation to provide a common back-end for XUL-based applications. It replaced the Gecko Runtime Environment, a stalled project with a similar purpose....

package, thereby necessitating only one copy of the runtime being installed. Another solution that stops dependency hell is made possible because a distribution's software repository can keep all of the user's software in sync, unlike on Windows and the Mac OS.

External links