Nix's remote build scheduler today is quite basic, providing far too much power for trivial builds and starving large builds.

Terminology and Numbers

Some terminology and background related to Hydra, the build farm:

Current Scheduling Process

Build hosts are declared to have a list of Features and a number of supported simultaneous jobs.

Nixpkgs Steps can describe themselves as requiring specific Features.

The Hydra Queue Runner then distributes as many jobs as possible to each machine, making sure to assign Steps with specific Feature requirements to Build hosts which provide them. Steps are allocated to the machine with the fewest steps currently assigned to it.

Not many builds are tagged as requiring Features. The most common feature is kvm indicating the host needs to be able to provide access to /dev/kvm. The second most common is less of a feature, but an attempt at indicating the size of the build: big-parallel.

The attempt at manually sizing the build with big-parallel is quite crude, and doesn’t adequately capture the variety of builds we have.

Every job on a given build host will be granted the same number of cores as every other build. This causes machines to either over-sell or trap their capacity. Given a machine with 24 cores, it may:

build job compromises

Untitled Database

There have been proposals to Nix to add “consumable” features like memory and core requirements, however these add considerable complexity to Nix, the build process, and the package declaration. Worse, these descriptions are almost always wrong and are prone to being out of date.

An Introduction to TAGS