Custom Cross Compiler with Nix · Hobson Space

The reason nix failed to wrap our compiler is because pkgs is defined with three really important variables:

1. buildPlatform – The platform we are building on
2. targetPlatform – The platform we are building for
3. hostPlatform – The platform which can run the stuff we are building

So, if you are anything like me, you will have followed right up until 3. The concept of a host platform is really strange but it will make sense once I tell you this. The host platform is only important when we are building compilers (which we are).

For example! I might want to use linux to build GCC that runs on mac but outputs code for Z80 CPUs. In which case:

• build = linux
• target = Z80
• host = darwin

When you aren’t cross-building then this is simple, all three are your system. If you load nixpkgs into nix repl, then you can take a look at pkgs.stdenv.[host,target,build]Platform and confirm that they are all the same.

But what about when cross compiling? I shall specify, I do not mean building a cross compiler, I mean using a cross compiler to build a package. Well then host = target != build. And this also makes sense!
If we build a normal package, say pkgs.pkgsCross.mingwW64.hello, then our buildPlatform is our system’s platform (linux or darwin), we are targeting windows. What about the host platform? Well for the hello package, it makes no sense to ask given that it isn’t a compiler. If we build pkgs.pkgsCross.mingwW64.gcc, then host = target. Because we would build a GCC that can run on windows. It is not the compiler we are using to build this package set!

So at this point, you may have realised something a bit strange, given that the various host, target and build platforms are defined for each package set in nix, where do the cross compilers live? They can’t live in the normal pkgs for your system because the cross compiler’s targret differs from the package set’s target. But they can’t live in pkgsCross because the compiler’s host (your system) differs from the package set’s. Sadly it matters! The reason being because for wrapCC to correctly find your compiler and binutils, it needs to work out the compiler prefix (arm-unknown-riscos- in my case) which it will only do if host != build. The answer is a strange one and can be found in nixpkgs:pkgs/stdenv/cross/default.nix. If you read this file closely, it starts with your normal stdenv, then it sets targetPlatform = crossSystem (so finally host != build), then builds the cross stdenv where it uses the compiler from the previous stage!

So how can we modify or extend this package set? I have no idea! It seems completely inaccessible! But we can always try to mimic the same trick!

I shall now walk you through the solution I came up with. You can find the whole flake here.

We start by importing pkgs and defining our cross system:

pkgs = import nixpkgs 🔥;

crossSystem = Tell us your thoughts in comments!;

riscosPkgs = import nixpkgs 🔥;

So far, pretty normal for cross compiling. I then have a section where I build the cross toolchain, I’ll leave it out, but the important things I define are riscosTools.rogcc (my cross GCC), and riscosTools.robinutils, my cross binutils. We move onto the trick:

buildSystem = import nixpkgs {
  system = "x86_64-linux";
  config.replaceStdenv = Tell us your thoughts in comments!: pkgs.stdenv.override { 
	targetPlatform = riscosPkgs.stdenv.targetPlatform // ⚡;
  };
};

Here I define a new package set! I set the target platform with my target platform and then override config. Why? arm-unknown-riscos is not a valid nix platform and so it will complain. I have to use arm-unknown-none-elf. But annoyingly, wrapCCWith uses config to get the prefix for the compiler. I do a cheeky override here purely so it picks up the correct compiler.

wrappedBintools = buildSystem.wrapBintoolsWith Share your opinion below!;

wrappedGcc = buildSystem.wrapCCWith {
  cc = riscosTools.rogcc;
  bintools = wrappedBintools;
};

I now use the buildSystem to call wrapBintoolsWith and wrapCCWith. Finally! It will pick up the correct prefix.

riscosPkgs = import nixpkgs {
  system = "x86_64-linux";
  inherit crossSystem;
  config.replaceCrossStdenv = {buildPackages, baseStdenv }:
    (buildPackages.overrideCC baseStdenv wrappedGcc).override {
      extraNativeBuildInputs = [riscosTools.elf2aif];
    };
}

I can now define the package set!

So does this work? Amazingly it does! I defined the following flake:

{
  description = "A new application for RISC OS";

  inputs = {
    utils.url = "github:numtide/flake-utils";

    # Point to your toolchain flake. This can be a local path or a git URL.
    riscos-nix.url = "path:../.";
  };

  outputs = { self, utils, riscos-nix }:
    utils.lib.eachDefaultSystem (system:
      let
        pkgs = riscos-nix.riscosPkgs;
      in
      {
        packages.default = pkgs.callPackage ./default.nix {};
      });
}

And default.nix:

{pkgs, lib}:

pkgs.stdenv.mkDerivation {
  pname = "hello-riscos";
  version = "1.0";

  src = ./.;
  hardeningDisable = [ "all" ];

  buildPhase = ''
    echo CC=$CC
    $CC hello.c -o hello
    elf2aif hello
  '';

  # The phase to install the executable into the Nix store
  installPhase = ''
    mkdir -p $out/bin
    cp hello $out/bin
  '';
}

and look what happened:

james> file result/bin/hello
result/bin/hello: RISC OS AIF executable

Who knew file knows about Risc OS!

That concludes things! I hope this saves you a lot of time!