Dependency Management is an important topic. Without dependencies, our applications would fail to run. This document intend to outline problems in many dependency management solutions that we see today.
This diagram outlines an ideal app, and a more realistic version of an application.
This diagram outlines an ideal app, and a more realistic version of an application.
Typical dependency managers set up systems in such a way that they follow the same approach:
This system usually installs the work on the project level or global level, but fails to take into account intertwined, conflicting dependencies. Let’s look at an example where this falls short: MySQL.
Example: MySQL Dependency Problem
MySQL is distributed in various versions. For this example, we will look at MySQL 5.5 and 5.7. These two versions are fundamentally incompatible with each other due to breaking changes in the way SQL is interpreted. However, when MySQL is installed it is installed to a global version located at the same place, with the binary mysql. This becomes an issue because applications will reference the same binary mysql.
Applications, which reference mysql, will now fail to run if they do not depend on the currently installed version. One naive solution is to install versions at /path/mysql_VERSION or something, but this fails because all tooling expects the binary to be called mysql.
At this point, we are in a position where we must fully upgrade all apps that depend on mysql or remain on the current version as we have no way to specify a version for a specific app.
Not just an app level issue
Sometimes dependencies can reference the same binaries, but expect different versions. This may not always lead to differences at first, but when you run it, you get different results. This is because it appears to be fine, it is the rightly named binary of course, but the code is different on the inside. And it’s the inside that counts.
In the previous example, we outlined how multiple versions can conflict with one another. A dependency manager can facilitate this by naively installing and compiling packages into the current system, in a global namespace. This of course creates a non-deterministic build and creates the potential for surprise conflicts.
It is non-deterministic because a dependency will compile differently based on the software already available, and the state of the system. For example, Imagemagick will compile and link to different image processing libraries at compile time, such as jpeg and png optimization libraries. This means that installing imagemagick will be different based on the users prior actions, and means that we cannot be sure that the software used on your laptop is configured the same as the one in CI or Production. This non-determinism also means that work may work on one developer’s laptop, but not another laptop. It also means that Imagemagick could fail to install and link correct (which we see a lot) due to numerous issues.