Ludovic Courtès

“ Embedding in a general-purpose language has allowed us to easily support internationalization of package descriptions, and to write a fast keyword search mechanism; it has also permitted novel features, such as an on-line auto-updater. Second, its build programs and libraries are also written in Scheme, leading to a unified programming environment made of two strata of code.
We hope to make Guix a good vehicle for an innovative free software distribution. The GNU system distribution we envision will give Scheme an important role just above the operating system interface. ”

“ The environment implicitly includes a copy of Guile, which is used to evaluate the builder expression. By default this is a stand-alone, statically-linked Guile, but users can also specify a derivation denoting a different Guile variant.
Remember that this expression is run by a separate Guile process than the one that calls build-expression->derivation: it is run by a Guile process launched by the build daemon, in a chroot. So, while there is a single language for both the "host" and the "build" side, there are really two strata of code, or tiers: the host-side, and the build-side code. ”

“ But how do we write a derivation that unpacks this tarball and adds it to the store? To solve this problem, the guile-bootstrap-2.0.drv derivation—the first one that gets built— uses bash as its builder, which runs build-bootstrap-guile.sh, which in turn calls tar to unpack the tarball. Thus, bash, tar, xz, and mkdir are statically-linked binaries, also part of the Guix source distribution, whose sole purpose is to allow the Guile tarball to be unpacked.
Once guile-bootstrap-2.0.drv is built, we have a functioning Guile that can be used to run subsequent build programs. ”

“ Remember that the build environment of a derivation contains nothing but its declared inputs. So there’s an obvious chicken-and-egg problem: how does the first package get built? How does the first compiler get compiled?
The GNU system we are building is primarily made of C code, with libc at its core. The GNU build system itself assumes the availability of a Bourne shell, traditional Unix tools provided by GNU Coreutils, GNU Awk, Findutils, sed, and grep. Furthermore, our build programs are written in Guile Scheme. ”

“ Guix supports transactional upgrades and roll-backs, unprivileged package management, per-user profiles, and garbage collection. It builds up on the low-level build and deployment layer of the Nix package manager. Guix uses Scheme as its programming interface. In particular, we devise an embedded domain-specific language (EDSL) to describe and compose packages. We demonstrate how it allows us to benefit from the host general-purpose programming language while not compromising on expressiveness. ”

“ Low-Level Store Operations As seen above, derivations are the central concept in Nix. A derivation bundles together a builder and its execution environment: command-line arguments, environment variable definitions, as well as a list of input derivations whose result should be accessible to the builder. Builders are typically executed in a chroot environment where only those inputs explicitly listed are visible. Guix transposes Nix’s derivation primitive literally to its Scheme interface. ”

“ Build programs often need to traverse file trees, modify files according to a given pattern, etc. One example is the "patch shebang" phase mentioned above: all the source files must be traversed, and those starting with #! are candidate to patching. This kind of task is usually associated with "shell programming"—as is the case with the build scripts found in NixOS, which are written in Bash, and resort to sed, find, etc. In Guix, a build-side Scheme module provides the necessary tools, built on top of Guile’s operating system interface. ”

“ RELATED WORK Numerous package managers for Scheme programs and libraries have been developed, including Racket’s PLaneT, Doro dango for R6RS implementations, Chicken Scheme’s "Eggs", Guildhall for Guile, and ScmPkg. [16] Unlike GNU Guix, they are typically limited to Scheme-only code, and take the core operating system software for granted. To our knowledge, they implement the imperative package management paradigm, and do not attempt to support features such as transactional upgrades and rollbacks. ”

“ We can formally verify this no-dependency requirement by checking whether the files of the final tool chain contain references to the /nix/store directories of the bootstrap inputs.
Obviously, Guix contains package declarations to build the bootstrap binaries shown in Figure 10. Because the final tool chain does not depend on those tools, they rarely need to be updated. Having a way to do that automatically proves to be useful, though. Coupled with Guix’s nascent support for cross-compilation, porting to a new architecture will boil down to cross-building all these bootstrap tools. ”

“ Our contribution is twofold: we demonstrate that use of Scheme and EDSLs achieves expressiveness comparable to that of Nix’s DSL while providing a richer and extensible programming environment; we further show that Scheme is a profitable alternative to shell tools when it comes to package build programs. Section 2 first gives some background on functional package management and its implementation in Nix. Section 3 describes the design and implementation of Guix’s programming and packaging interfaces. ”

“ In Guix, instances have the expected level of abstraction, and they are readily accessible as first-class Scheme objects.
Sometimes it is desirable for the value of a field to depend on the system type targeted. For instance, for bootstrapping purposes, MIT/GNU Scheme’s build system depends on pre-compiled binaries, which are architecture-dependent; its input field must be able to select the right binaries depending on thearchitecture. To allow field values to refer to the target system type, we resort to thunked fields, as shown on line 13 of Figure 5. ”

“ BUILD EXPRESSIONS AND PACKAGE DESCRIPTIONS Our goal when designing Guix was to provide interfaces ranging from Nix’s low-level primitives such as derivation to high-level package declarations. The declarative interface is a requirement to help grow and maintain a large software distribution. ”

“ Like Guix and Nix, Vesta is a purely functional build system. [6] It uses an external DSL close to the Nix language. However, the primary application of Vesta is fine-grain software build operations, such as compiling a single C file. It is a developer tool, and does not address deployment to end-user machines. Unlike Guix and Nix, Vesta tries hard to support the standard Unix file system layout, relying on a virtual file system to “map” files to their right location in the build environment. ”

“ Of course, that is all implemented in Scheme, via build-expression->derivation. Supporting code is available as a build-side module that gnu-build-system automatically adds as an input to its build scripts. The default build programs just call the procedure of that module that runs the above phases.
The (guix build gnu-build-system) module contains the implementation of the above phases; it is imported on the builder side. The phases are modeled as follows: each phase is a procedure accepting several keyword arguments, and ignoring any keyword arguments it does not recognize. ”

“ Experience with NixOS suggests that the functional model, coupled with continuous integration, allows the distribution to grow relatively quickly, because it is always possible to precisely monitor the status of the whole distribution and the effect of a change—unlike with imperative distributions, where the upgrade of a single package can affect many applications in many unpredictable ways. ”

“ To describe and compose build processes, Nix implements its own domain-specific language (DSL) , which provides a convenient interface to the build and storage mechanisms described above. The Nix language is purely functional, lazy, and dynamically typed; it is similar to that of the Vesta software configuration system [6] . It comes with a handful of built-in data types, and around 50 primitives. The primitive to describe a build process is derivation.
derivation { name = "example-1 .0"; builder = "${./static-bash}" ”

“ Figure 4 A package definition using the high-level interface.
field names to field indexes, such that that mapping incurs no run-time overhead, in a way similar to SRFI-35 records. [8] They also bind fields as per letrec*, allowing them to refer to one another, as on line 8 of Figure 4. The base32 macro simply converts a literal string containing a base-32 representation into a bytevector literal, again allowing the conversion and error-checking to be done at expansion time rather than at run-time. ”

“ This mechanism is similarly used for other build systems such as CMake and Python’s build system.
(substitute* (find-files "gcc/config" "^gnu-user (64) ?.h$") ( ("#define LIB_SPEC (.*) $" _ suffix) (string-append "#define LIB_SPEC "-L" libc "/lib " " suffix "") ) ( ("#define STARTFILE_SPEC.*$" line) (string-append "#define STARTFILE_PREFIX_1 "" libc "/lib\"
" line) ) ) Figure 9 The substitute* macro for sed-like substitutions. ”

“ Defining Record Types. 1999. http://srfi.schemers.org/srfi-9/srfi-9.html. ↑ T. Morris. Asymmetric Lenses in Scala. 2012. http://days2012.scala-lang.org/. ↑ P. Hudak. Building domain-specific embedded languages. ”