Matthew BauerBlogRSS

12 May 2018

Channel Changing with Nix

1 Introduction to channels

One of the many underappreciated feature of Nix is its ability to travel back in time. Functional dependencies mean that you can easily pull in old releases of NixOS & Nixpkgs without changing your environment at all! It’s surprisingly easy in Nix 2.0 with its support for Import From Derivation.

First, I will provide some code to get us started. This Nix script is what I use as my “channel changer”. It bootstraps the use of old channels. In Nix-world, channels are just what we call the CI-tested branch of NixOS/Nixpkgs1. The NixOS maintainers have been making releases consistently since 2013, so there is a lot of interesting history.

2 Channel changing

Here is my script that I will refer to later on in the post as “channels.nix” (be sure to try it out yourself!)2

let mapAttrs = f: set: builtins.listToAttrs (
      map (attr: { name = attr; value = f set.${attr}; })
          (builtins.attrNames set));
    channels = {
      aardvark    = "13.10";
      baboon      = "14.04";
      caterpillar = "14.12";
      dingo       = "15.09";
      emu         = "16.03";
      flounder    = "16.09";
      gorilla     = "17.03";
      hummingbird = "17.09";
      impala      = "18.03";
    };
in mapAttrs (v:
     import (builtins.fetchTarball
             "https://nixos.org/channels/nixos-${v}/nixexprs.tar.xz") {})
   channels

As you can see from the script there have been 9 releases in total. We use a different letter of the alphabet for each release, starting with A for Aardvark. We are now up to I for Impala3. New releases happen every 6 months with Aardvark released in December 2013. The releases are versioned as YY.MM which is a common practice for Linux distros.

3 ‘nix run’ magic

In my Nix script, I have created an “attribute” for each version that has been released. With Nix 2.0, it is very easy to run packages from them. Here is the command to run hello world from Hummingbird.

nix run -f channels.nix hummingbird.hello -c hello
Hello, world!

This has run the hello executable from the hummingbird release. Since you are most likely not running Hummingbird, it may take a while to the first time. However, once Nix has downloaded the needed files future execution will be instantaneous. The package is completely self-contained! To start, we will do examples in Impala (18.03) so that things go a little faster.

There are lots of packages in Nixpkgs so we don’t have to restrict ourselves to just hello. Let’s try out cowsay first.

nix run -f channels.nix impala.cowsay -c cowsay hello
 _______ 
< hello >
 ------- 
        \   ^__^
         \  (oo)\_______
            (__)\       )\/\
                ||----w |
                ||     ||

There are many, many more of these commands. I’ve included a few below for you to try out on your own.

# Look up the weather
nix run -f channels.nix impala.curl -c curl wttr.in/seville

# Download music
nix run -f channels.nix impala.youtube-dl -c \
    youtube-dl -t --extract-audio \
    --audio-format mp3 \
    https://www.youtube.com/watch?v=dQw4w9WgXcQ

# Go see a Star War
nix run -f channels.nix impala.telnet -c telnet towel.blinkenlights.nl 666
nix run -f channels.nix impala.sox -c bash -c \
    'for n in E2 A2 D3 G3 B3 E4;
     do play -n synth 4 pluck $n repeat 2;
     done'

# Play Nethack
nix run -f channels.nix impala.nethack -c nethack

# Get your fortune
nix run -f channels.nix impala.fortune -c fortune

4 The macOS+Nix odyssey

The fact that Nix works so well on macOS is a miracle in its own right. Apple has a proprietary ABI but Nix is intended to be used with free software. To get around this, many hacks are necessary including taking Apple’s standard C library4. Anyway, I was interested in how well the binaries produced by Nixpkgs hold up on my MacBook. For reference, here are the versions of macOS available when each release happened. Those familiar with macOS internals will remember some significant differences between these versions.

NixOS release macOS release
Aardvark (13.10) Mountain Lion (10.8)
Baboon (14.04) Mavericks (10.9)
Caterpillar (14.12) Yosemite (10.10)
Dingo (15.09) Yosemite (10.10)
Emu (16.03) El Capitan (10.11)
Flounder (16.09) El Capitan (10.11)
Gorilla (17.03) Sierra (10.12)
Hummingbird (17.09) High Sierra (10.13)
Impala (18.03) High Sierra (10.13)

So, my MacBook is running the latest macOS 10.13. Naturally we can test that Impala & Hummingbird will work correctly. hello is a good tester, of course, not comprehensive.

nix run -f channels.nix impala.hello -c hello
Hello, world!
nix run -f channels.nix hummingbird.hello -c hello
Hello, world!

But now let’s test Gorilla. It was released when macOS Sierra was still around but the ABI should be compatible.

nix run -f channels.nix gorilla.hello -c hello
dyld: Library not loaded: /usr/lib/system/libsystem_coretls.dylib
 Referenced from: /nix/store/v7i520r9c2p8z6vk26n53hfrxgqn8cl9-Libsystem-osx-10.11.6/lib/libSystem.B.dylib
 Reason: image not found
sh: line 1: 23628 Abort trap: 6           nix run -f channels.nix gorilla.hello -c hello

Oh no!

We can see that libSystem 10.11 has been downloaded for us5. However, libSystem is referring to an image that isn’t on our machine. libsystem_coretls.dylib must have existed in 10.11 macOS but been removed since then6.

At this point, it may look like Nixpkgs will be broken going backwards. But, I want to try Flounder just to see what happens.

nix run -f channels.nix flounder.hello -c hello
Hello, world!

Amazingly, it worked! I am still not sure what the differences are, but it seems that the older executable is still available. Let’s try out Emu to see what happens there.

nix run -f channels.nix emu.hello -c hello
builder for '/nix/store/s41jnb4kmxxbwj40c5l88k9ma0mwfy0b-hello-2.10.drv' failed due to signal 4 (Illegal instruction: 4)
error: build of '/nix/store/s41jnb4kmxxbwj40c5l88k9ma0mwfy0b-hello-2.10.drv' failed

Wow! Again we hit an issue. This is the infamouse Illegal instruction: 4 bug that is frequently hit in Nixpkgs7. It occurs when an executable uses instructions that have been blocked by the XNU kernel. This is usually because they are considered insecure so a patch is needed to fix it. We no longer support Emu, so this is probably the end of the line. Let’s try Dingo out just to be sure though.

nix run -f channels.nix dingo.hello -c hello
builder for '/nix/store/1cyagihl211vsis9bz09cqaz3h2yyc23-libxml2-2.9.3.drv' failed with exit code 77; last 10 log lines:
 checking for awk... awk
 checking whether make sets $(MAKE)... yes
 checking whether make supports nested variables... yes
 checking whether make supports nested variables... (cached) yes
 checking for gcc... gcc
 checking whether the C compiler works... no
 configure: error: in `/private/tmp/nix-build-libxml2-2.9.3.drv-0/libxml2-2.9.3':
 configure: error: C compiler cannot create executables
 See `config.log' for more details
 
cannot build derivation '/nix/store/jd4y5aps1z61jqbhsz1gy408zwwa49w4-clang-3.6.2.drv': 1 dependencies couldn't be built
cannot build derivation '/nix/store/n4q29z97dc1p9mqrn2ydhlfmsqwbgx0j-libarchive-3.1.2.drv': 1 dependencies couldn't be built
cannot build derivation '/nix/store/vh2bh7gaw2m0rgxscf3mhm1d3rz3xwfg-clang-wrapper-3.6.2.drv': 1 dependencies couldn't be built
cannot build derivation '/nix/store/zg90kfmf99h03z0fl03gw3gh105mb02c-cmake-3.3.1.drv': 1 dependencies couldn't be built
cannot build derivation '/nix/store/45ndaky3079nd78042384f8hbidq7f7q-libc++abi-3.6.2.drv': 1 dependencies couldn't be built
cannot build derivation '/nix/store/mmyz6rrddfahwl23i9d9vjh7wa8irp5k-stdenv-darwin-boot-3.drv': 1 dependencies couldn't be built
cannot build derivation '/nix/store/lqjabx84kndk75y8m0lq7zh5190k6zzz-hello-2.10.drv': 1 dependencies couldn't be built
error: build of '/nix/store/lqjabx84kndk75y8m0lq7zh5190k6zzz-hello-2.10.drv' failed

This is a curious error because it is very different from the previous one. Back here we were still using Clang 3.3 & it looks like bootstrapping is failing on our newer machines. I was not using Nix at this time (late 2015), so I will have to defer to someone who remembers that time better. Let’s keep going.

nix run -f channels.nix caterpillar.hello -c hello
error: attribute 'hello' in selection path 'caterpillar.hello' not found
nix run -f channels.nix baboon.hello -c hello
error: attribute 'hello' in selection path 'baboon.hello' not found
nix run -f channels.nix aardvark.hello -c hello
error: attribute 'hello' in selection path 'aardvark.hello' not found

I’ve grouped them together because they have the same output. It appears that hello was not available back then! I’m not sure what is going on. Again, I will defer to someone else to explain why this happens. But, I know for a fact that GNU Hello is one of the first packages to be packaged in the Nix language8.

5 Conclusion

I wanted to also look at what happens on Linux when you go back through channels. I don’t have time currently so I am just including what I have. Anyway, if you are able to report back what happens on Linux when running these old channels, it would certainly be interesting.

My main goal was to just share some useful things in Nix that I don’t think many people outside of the core Nix community know about. Documentation has gotten better recently but lots of times people like to just read blog posts like this. Hopefully you got a feel for what can be done in Nix.

Footnotes:

1

The difference between NixOS & Nixpkgs can sometimes cause confusion especially because they are hosted in the same repository. We usually refer to NixOS for the Linux-specific distro while Nixpkgs refers to the cross-platform set of packages. Here I am referring to them collectively.

2

Note that the channel changing script is not necessary. You can always refer to the Nixpkgs version directly with the -f argument. The script is just an easy way to introduce people to the concept.

3

The in-development version of NixOS/Nixpkgs will be a J for Jackrabbit.

4

Apple’s C standard library is called libSystem. Note that unlike Glibc & Musl it contains much, much more than what is needed to compile simple C programs.

5

Note that the same libSystem is used for all of Nixpkgs to peliminate having to do SDK detection. Eventually we will update this to 10.12 or 10.13 but we prefer to stay a couple releases behind.

6

This is not a complete explanation, but the best I can do for those not aware of the internals of Nixpkgs.

7

See GitHub issue #17372.

8

See release 0.5.