Lately, I've been writing a significant amount of OCaml as part of my PhD. Instead of using the OCaml package manager (opam) command-line interface (CLI) for these projects, I prefer to use Nix to provide declarative and reproducible development environments and builds. However I still want to be able to interoperate with opam's file format and access packages from the opam repository. In this blog post we'll walk through creating a flake.nix file to do this for a hello world project at github.com/RyanGibb/ocaml-nix-hello. Our aim is to make building an OCaml project, and setting up a development environment, as simple as one command.

I've said that Nix can provide declarative and reproducible environments and builds. Let's break down what this means:

This aims to solve the problem of 'it works on my machine' but not elsewhere. Container images are also often used for a similar purpose, however in Nix's case we only need to specify the inputs and build rules precisely.

For an introduction to Nix and it's ecosystems, I've written more here.

I'm taking an opinionated stance and using Nix Flakes3. Flakes are a new way to specify a source tree as a Nix project using a flake.nix. They provide a lot of benefits: pinning project dependencies using a lockfile flake.lock4, resolving Nix expressions in isolation5, provide a Nix-native6 way of composing Nix projects7, and a new CLI8 to use Nix. If this sounds a bit complex, just take away despite them being behind a feature flag Nix flakes are the future and are worth using for their benefits now.

To enable flakes on your NixOS system add this fragment to your configuration:

nix.settings.experimental-features = [ "nix-command" "flakes" ];

I've said that I still want to interoperate with opam for 2 reasons:

Fortunately a project already exists that solves this for us: github.com/tweag/opam-nix. opam-nix translates opam packages into Nix derivations, so we can use dependencies from opam-repository. It also allows us to declare our project's dependencies in opam's format, so that other users don't have to use Nix. It uses opam's dependency version solver under the hood when building a project. Read more at www.tweag.io/blog/2023-02-16-opam-nix/.

opam-nix also reproducibly provides system dependencies (picking them up from opam depexts) through Nix's mechanisms. Nix provides great support for cross-language project dependencies in general.

The minimum required to get our project building is:

{
  inputs.opam-nix.url = "github:tweag/opam-nix";

  outputs = { self, opam-nix }:
    let
      system = "x86_64-linux";
      inherit (opam-nix.lib.${system}) buildOpamProject;
      package = "hello";
    in rec {
      packages.${system} = buildOpamProject { } package ./. {
        ocaml-base-compiler = "*";
      };
      defaultPackage.${system} = packages.${system}.${package};
    };
}

Documentation for buildOpamProject can be found at github.com/tweag/opam-nix/#buildOpamProject.

This is sufficient to build the project with:

$ nix build .

We can also get a development shell and build the project outside a Nix derivation -- benefitting from the dune cache -- using:

$ nix develop . -c dune build

Each of the following sections will modify this MVP flake to add new functionality, before we combine them all into the final product.

A user may also want to benefit from developer tools, such as the OCaml LSP server, which can be added to the query made to opam:

 {
   inputs.opam-nix.url = "github:tweag/opam-nix";

-  outputs = { self, opam-nix }:
+  outputs = { self, nixpkgs, opam-nix }:
     let
       system = "x86_64-linux";
+      # instantiate nixpkgs with this system
+      pkgs = nixpkgs.legacyPackages.${system};
       inherit (opam-nix.lib.${system}) buildOpamProject;
       package = "hello";
     in rec {
       packages.${system} = buildOpamProject { } package ./. {
         ocaml-base-compiler = "*";
+        ocaml-lsp-server = "*";
       };
       defaultPackage.${system} = packages.${system}.${package};
+      # create a development environment with ocaml-lsp-server
+      devShells.${system}.default = pkgs.mkShell {
+        inputsFrom = [ defaultPackage.${system} ];
+        buildInputs = [ packages.${system}."ocaml-lsp-server" ];
+      };
     };
 }

Users can then launch an editor with ocaml-lsp-server in the environment with:

$ nix develop . -c $EDITOR `pwd`

For nix develop documentation see nixos.org/manual/nix/stable/command-ref/new-cli/nix3-develop.html.

We might want to specify a specific version of the opam-respository to get more up to date packages, which we can do by tracking it as a seperate input to the flake. We can do the same with the Nixpkgs monorepo10.

 {
-  inputs.opam-nix.url = "github:tweag/opam-nix";
+  inputs = {
+    nixpkgs.url = "github:NixOS/nixpkgs";
+    opam-nix.url = "github:tweag/opam-nix";
+    opam-repository = {
+      url = "github:ocaml/opam-repository";
+      flake = false;
+    };
+    opam-nix.inputs.opam-repository.follows = "opam-repository";
+    opam-nix.inputs.nixpkgs.follows = "nixpkgs";
+  };

-  outputs = { self, opam-nix }:
+  outputs = { self, opam-nix, ... }:
     let
       system = "x86_64-linux";
       inherit (opam-nix.lib.${system}) buildOpamProject;

The opam-repository can also be chosen granularly opam-nix function call with the repos argument, but we just override opam-nix's opam-repository input. Note that some packages, notably ocamlfind, required patches to work with opam-nix. If you run into errors you can force the resolution of an old version, e.g. ocamlfind = "1.9.5";.

One can pin an input to a specific commit with, e.g.:

nix flake update --override-input opam-repository github:ocaml/opam-repository/<commit>

Every time we call buildOpamProject, or an equivalent function that calls queryToScope under the hood, we perform a computationally expensive dependency resolution using a SAT solver. We can save the results of this query to a file with materialization11.

 {
   inputs.opam-nix.url = "github:tweag/opam-nix";

-  outputs = { self, opam-nix }:
+  outputs = { self, opam-nix, ... }:
     let
       system = "x86_64-linux";
-      inherit (opam-nix.lib.${system}) buildOpamProject;
+      inherit (opam-nix.lib.${system})
+        buildOpamProject
+        materializedDefsToScope
+        materializeOpamProject';
       package = "hello";
-    in rec {
-      packages.${system} = buildOpamProject { } package ./. {
+      query = {
         ocaml-base-compiler = "*";
       };
-      defaultPackage.${system} = packages.${system}.${package};
+      resolved-scope = buildOpamProject { } package ./. query;
+      materialized-scope = materializedDefsToScope
+        { sourceMap.${package} = ./.; } ./package-defs.json;
+    in rec {
+      packages = {
+        resolved = resolved-scope;
+        materialized.${system} = materialized-scope;
+        # to generate:
+        #   cat $(nix eval .#package-defs --raw) > package-defs.json
+        ${system}.package-defs = materializeOpamProject' { } ./. query;
+      };
+      defaultPackage.${system} = packages.materialized.${system}.${package};
     };
 }

The package-defs.json file generated by cat $(nix eval .#package-defs --raw) > package-defs.json should be committed to the repository.

We can modify derivations with Nix overlays12.

       system = "x86_64-linux";
       inherit (opam-nix.lib.${system}) buildOpamProject;
       package = "hello";
-    in rec {
-      packages.${system} = buildOpamProject { } package ./. {
-        ocaml-base-compiler = "*";
+      overlay = final: prev: {
+        "${package}" = prev.${package}.overrideAttrs (_: {
+          # override derivation attributes, e.g. add additional dependacies
+          buildInputs = [ ];
+        });
       };
+      overlayed-scope = let
+        scope = buildOpamProject { } package ./. {
+          ocaml-base-compiler = "*";
+        };
+        in scope.overrideScope' overlay;
+    in rec {
+      packages.${system} = overlayed-scope;
       defaultPackage.${system} = packages.${system}.${package};
     };
 }

Nix flakes are evaluated hermetically and as a result don't take any arguments13. However different systems will have different packages built for them. We essentially parametrize based on system by different derivation paths, e.g. nix build . implicitly builds the derivation packages.${system}.default. We can support multiple systems by creating derivations for each system. flake-utils14 provides a convient mechanism for creating these derivations.

 {
   inputs.opam-nix.url = "github:tweag/opam-nix";

-  outputs = { self, opam-nix }:
-    let
-      system = "x86_64-linux";
-      inherit (opam-nix.lib.${system}) buildOpamProject;
-      package = "hello";
-    in rec {
-      packages.${system} = buildOpamProject { } package ./. {
-        ocaml-base-compiler = "*";
-      };
-      defaultPackage.${system} = packages.${system}.${package};
-    };
+  outputs = { self, opam-nix, flake-utils }:
+    flake-utils.lib.eachDefaultSystem (system:
+      let
+        system = "x86_64-linux";
+        inherit (opam-nix.lib.${system}) buildOpamProject;
+        package = "hello";
+      in rec {
+        packages.${system} = buildOpamProject { } package ./. {
+          ocaml-base-compiler = "*";
+        };
+        defaultPackage.${system} = packages.${system}.${package};
+      }
+    );
 }

We can combine all of:

To gives us a complete flake for our project:

{
  inputs = {
    nixpkgs.url = "github:NixOS/nixpkgs";
    opam-nix.url = "github:tweag/opam-nix";
    opam-repository = {
      url = "github:ocaml/opam-repository";
      flake = false;
    };
    opam-nix.inputs.opam-repository.follows = "opam-repository";
    opam-nix.inputs.nixpkgs.follows = "nixpkgs";
  };

  outputs = { self, nixpkgs, opam-nix, flake-utils, ... }:
    flake-utils.lib.eachDefaultSystem (system:
      let
        system = "x86_64-linux";
        pkgs = nixpkgs.legacyPackages.${system};
        inherit (opam-nix.lib.${system})
          buildOpamProject
          materializedDefsToScope
          materializeOpamProject';
        package = "hello";
        query = {
          ocaml-base-compiler = "*";
        };
        overlay = final: prev: {
          "${package}" = prev.${package}.overrideAttrs (_: {
            # override derivation attributes, e.g. add additional dependacies
            buildInputs = [ ];
          });
        };
        resolved-scope =
          let scope = buildOpamProject { } package ./. query;
          in scope.overrideScope' overlay;
        materialized-scope =
          let scope = materializedDefsToScope
            { sourceMap.${package} = ./.; } ./package-defs.json;
          in scope.overrideScope' overlay;
      in rec {
        packages = {
          resolved = resolved-scope;
          materialized = materialized-scope;
          # to generate:
          #   cat $(nix eval .#package-defs --raw) > package-defs.json
          package-defs = materializeOpamProject' { } ./. query;
        };
        defaultPackage = packages.materialized.${package};
        devShells.default = pkgs.mkShell {
          inputsFrom = [ defaultPackage ];
          buildInputs = [ packages."ocaml-lsp-server" ];
        };
      }
    );
}

Try it out yourself at github.com/RyanGibb/ocaml-nix-hello/commits/main.

With a flake, we can easily create a CI job from our Nix flake to build our program. For example, a GitHub action would be:

name: ci
on:
  push:
    branches:
      - 'main'
  pull_request:
    branches:
      - "main"
  workflow_dispatch:
jobs:
  nix:
    name: Build with Nix
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v3
      - uses: cachix/install-nix-action@v12
      - run: nix --extra-experimental-features "nix-command flakes" build

See it in action at github.com/RyanGibb/ocaml-nix-hello/actions/runs/5199834104.

The final benefit we'll mentione that this workflow provides is that all dependencies are stored in the global Nix store and transparently shared between projects. When they differ they're duplicated so projects don't interfere with each other. Derivations can be garbage collected to save on disk space when they're no longer used.

To garbage collect globally:

$ nix-collect-garbage

To garbage collect a specific path:

$ PATH=`readlink result`
$ rm result
$ nix-store --delete $(nix-store -qR $PATH)

A full-featured example of a Nix flake building a project I've been working on recently, an effects-based direct-style Domain Name System implementation written in OCaml, can be found at github.com/RyanGibb/aeon/blob/main/flake.nix.

Now someone getting started with our repository can clone and build it with only:

$ git clone git@github.com:RyanGibb/ocaml-nix-hello.git
$ cd ocaml-nix-hello
$ nix build .

They can set up a development environment with:

$ nix develop -c dune build
$ nix develop -c $EDITOR `pwd`

They could also build it without manually cloning it:

$ nix shell github:RyanGibb/ocaml-nix-hello
$ hello
Hello, World!

They can even run it in a single command!

$ nix run github:ryangibb/ocaml-nix-hello
Hello, World!

If this blog post has made you curious, go try this for your own projects! Feel free to get in touch at ryan@freumh.org.

Thanks to Alexander Bantyev (balsoft) for creating and maintaining opam-nix.

  1. opam.ocaml.org/doc/man/opam-switch.html↩︎

  2. NB this doesn't guarantee binary reproducibility as there could still be some randomness involved. This is why derivations are stored at a hash of their inputs rather than their result. But there is work on providing a content addressable store: www.tweag.io/blog/2020-09-10-nix-cas/↩︎

  3. For an introduction to Flakes see this blog post series: www.tweag.io/blog/2020-05-25-flakes/.↩︎

  4. Which replace imperatively managed Nix channels.↩︎

  5. Existing Nix derivations are built in isolation, but flakes also evaluate the Nix expression in isolation which enabled caching of expression evaluation. Note Nix expression refers to an expression in the Nix Language.↩︎

  6. As opposed to an external tool like github.com/nmattia/niv.↩︎

  7. Without having to include them in the Nixpkgs monorepo.↩︎

  8. See nixos.org/manual/nix/stable/command-ref/experimental-commands.html for the new CLI reference.↩︎

  9. ../hillingar/#building-unikernels-para-5 ↩︎

  10. See ../hillingar#nixpkgs for more information.↩︎

  11. github.com/tweag/opam-nix#materialization↩︎

  12. nixos.wiki/wiki/Overlays↩︎

  13. See github.com/NixOS/nix/issues/2861 for more context on Nix flake arguments.↩︎

  14. github.com/numtide/flake-utils, included in github.com/NixOS/flake-registry↩︎