~ehamberg/blog

52801c67d08eb85ab55c794261ca844132adf863 — Erlend Hamberg 4 months ago 4b996c6 main
Two new posts on nix/nixos/haskell
A blog/posts/2023-09-02-nix-flake-for-haskell-project.md => blog/posts/2023-09-02-nix-flake-for-haskell-project.md +298 -0
@@ 0,0 1,298 @@
---
title: Packaging a Haskell project as a Nix flake
description: My circuitous path to packaging a Haskell project for Nix
tags: nix, haskell
---

# Introduction

I have been using nix for local development for over a year now, but want to start learning more, so three weeks ago [I posted some question on Mastodon](https://functional.cafe/@eh/110854503663534170):

> Okay. Time to learn some more #nix than just using it for local development!
> 
> I have a server running #nixos 23.05 and I want to set up some services that I have written myself. The first service is a simple IRC bot written in Haskell.
> 
> Where do I begin? 🤷
> 
> It should basically end up with a binary on the server and a service that is pretty much just this:
> 
> ```
> [Service]
> ExecStart=/some/where/my_silly_bot
> Restart=always
> ```
> 
> Do I create some config in the git repo and refer to it from the nixos server? Am I creating a “module”? (I duckduckwent it, but not even knowing what concepts I should search for made it a bit difficult. 😇)

I got some helpful pointers and realized there will be two steps to this:

1. Package the project for building with nix (this post)
2. Set up the service on a NixOS server (next post)
# Creating a flake for my Haskell project

The project I'm going to package is a simple IRC bot: <https://github.com/ehamberg/tribot>.

So far, I've only written flakes to get a local dev environment when opening a project in a shell or in an editor (by using [direnv](https://direnv.net) and [nix-direnv](https://github.com/nix-community/nix-direnv)), but now I also want to be able to `nix build` the project.

In other words, there are two goals the nix flake should fulfil:

1. Make it possible to build the project with `nix build`
2. Make it possible to get a local development environment for the project using `direnv` + `nix-direnv` (or `nix develop`)

## Attempt 1: A basic setup for Haskell projects

Fortunately, Gabriella Gonzalez has written a great post on [incrementally packaging a Haskell program](https://www.haskellforall.com/2022/08/incrementally-package-haskell-program.html), so we're off to a great start:

```nix
{
  inputs = {
    nixpkgs.url = "github:NixOS/nixpkgs/nixpkgs-unstable";
    flake-utils.url = "github:numtide/flake-utils";
  };

  outputs = { nixpkgs, flake-utils, ... }:
    flake-utils.lib.eachDefaultSystem (system:
      let
        config = { };

        overlay = pkgsNew: pkgsOld: {
          tribot = pkgsNew.haskell.lib.justStaticExecutables
            pkgsNew.haskellPackages.tribot;

          haskellPackages = pkgsOld.haskellPackages.override (old: {
            overrides =
              pkgsNew.haskell.lib.packageSourceOverrides { tribot = ./.; };
          });
        };

        pkgs = import nixpkgs {
          inherit config system;
          overlays = [ overlay ];
        };

      in rec {
        packages.default = pkgs.haskellPackages.tribot;

        apps.default = {
          type = "app";
          program = "${pkgs.tribot}/bin/tribot";
        };

        devShells = {
            default = pkgs.mkShell { buildInputs = with pkgs; [
                haskellPackages.haskell-language-server
                haskellPackages.hlint
                haskellPackages.cabal-fmt
                haskellPackages.ormolu
                cabal-install
                sqlite
                zlib
            ]; };
        };
      });
}
```

This makes sense for the most part, even though there are some pieces that are somewhat mysterious[^juststaticexecutables].

This would normally be the end of packaging a Haskell project, but while the dev shell still works, `nix build` does not:

    ❯ nix build
    error: Package ‘simpleirc-0.3.1’ in /nix/store/qx67jipw01zps1rqgmmpl7as1irff275-source/pkgs/development/haskell-modules/hackage-packages.nix:268059 is marked as broken, refusing to evaluate.
    a) To temporarily allow broken packages, you can use an environment variable
              for a single invocation of the nix tools.
    
                $ export NIXPKGS_ALLOW_BROKEN=1
    		Note: For `nix shell`, `nix build`, `nix develop` or any other Nix 2.4+
            (Flake) command, `--impure` must be passed in order to read this
            environment variable.

Ooof!

## Attempt 2: Allow broken‽

Allowing broken packages sounds like a really bad idea and – at best – a short-term solution, but sure, let's try:

    ❯ NIXPKGS_ALLOW_BROKEN=1 nix build --impure
    warning: Git tree '/Users/ehamberg/Developer/tribot' is dirty
    error: builder for '/nix/store/a0cr7b79dcmak5m59aiam7ksxihgfrjc-simpleirc-0.3.1.drv' failed with exit code 1;
           last 10 log lines:
           >   |
           > 3 | import           Test.Hspec.Monadic
           >   | ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
           >
           > tests/Spec.hs:4:1: error:
           >     Could not find module ‘CoreSpec’
           >     Use -v (or `:set -v` in ghci) to see a list of the files searched for.
           >   |
           > 4 | import qualified CoreSpec
           >   | ^^^^^^^^^^^^^^^^^^^^^^^^^
           For full logs, run 'nix log /nix/store/a0cr7b79dcmak5m59aiam7ksxihgfrjc-simpleirc-0.3.1.drv'.
    error: 1 dependencies of derivation '/nix/store/48ds1qxc73masqwrp6yagpj1d5zdn8r7-tribot-0.4.0.0.drv' failed to build

Oh no! Looks like a test is broken. 

## Attempt 3: Disable the tests!

Hmm. Since we're already doing naughty things, let's see [how to disable testing for a haskell package](https://stackoverflow.com/questions/60781481/how-can-i-disable-testing-for-a-haskell-package-in-nix). Spoiler: We can use  `haskell.lib.dontCheck`: Let's fix our flake to use this to *override* our package with one without tests:

```diff
@@ -14,8 +14,21 @@
             pkgsNew.haskellPackages.tribot;

           haskellPackages = pkgsOld.haskellPackages.override (old: {
-            overrides =
-              pkgsNew.haskell.lib.packageSourceOverrides { tribot = ./.; };
+            overrides = let
+              oldOverrides = old.overrides or (_: _: { });
+
+              manualOverrides = haskellPackagesNew: haskellPackagesOld: {
+                simpleirc =
+                  pkgsNew.haskell.lib.dontCheck haskellPackagesOld.simpleirc;
+              };
+
+              sourceOverrides =
+                pkgsNew.haskell.lib.packageSourceOverrides { tribot = ./.; };
+
+            in pkgsNew.lib.fold pkgsNew.lib.composeExtensions oldOverrides ([
+              sourceOverrides
+              manualOverrides
+            ]);
           });
         };
```

Yay!

    ❯ NIXPKGS_ALLOW_BROKEN=1 nix build --impure
    warning: Git tree '/Users/ehamberg/Developer/tribot' is dirty
    error: builder for '/nix/store/k1g0n2xgpk9mf2hm0dp321paihpfmr65-tribot-0.4.0.0.drv' failed with exit code 1;
           last 10 log lines:
           >
           > app/Main.hs:12:1: error:
           >     Could not find module ‘Network.SimpleIRC.Sasl’
           >     Perhaps you meant
           >       Network.SimpleIRC.Core (from simpleirc-0.3.1)
           >       Network.SimpleIRC (from simpleirc-0.3.1)
           >     Use -v (or `:set -v` in ghci) to see a list of the files searched for.
           >    |
           > 12 | import Network.SimpleIRC.Sasl (SaslPlainArgs (..), saslPlain)
           >    | ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
           For full logs, run 'nix log /nix/store/k1g0n2xgpk9mf2hm0dp321paihpfmr65-tribot-0.4.0.0.drv'.

No yay! Turns out all of this was a red herring. My project actually depends on a newer version of *SimpleIRC* than the version available on Hackage (that is marked as *broken*…).

Turns out I had this tucked away in a long-forgotten `cabal.project` file:

    source-repository-package
      type: git
      location: git://github.com/dom96/SimpleIRC.git
    
    packages: ./tribot.cabal
## Attempt 4: Straight to the source!

Okay. So we need to fetch *SimpleIRC* straight from Github. Turns out that there's a `fetchGit` function in Nix (or nixpkgs? how do I know?), so we can use this to fetch a given revision from Github (and reenable tests 😮‍💨):

```diff
@@ -18,8 +18,14 @@
               oldOverrides = old.overrides or (_: _: { });

               manualOverrides = haskellPackagesNew: haskellPackagesOld: {
-                simpleirc =
-                  pkgsNew.haskell.lib.dontCheck haskellPackagesOld.simpleirc;
+                simpleirc = let
+                  src = builtins.fetchGit {
+                    url = "https://github.com/dom96/SimpleIrc";
+                    ref = "master";
+                    rev = "8d156a89801be2c9b6923d85e6b199c8173e445a";
+                  };
+                in pkgs.haskell.lib.dontCheck
+                (haskellPackagesOld.callCabal2nix "simpleirc" src { });
               };

               sourceOverrides =
```

And boom! `nix build` works!
## The final `flake.nix`:

```nix
{
  inputs = {
    nixpkgs.url = "github:NixOS/nixpkgs/nixpkgs-unstable";
    flake-utils.url = "github:numtide/flake-utils";
  };

  outputs = { nixpkgs, flake-utils, ... }:
    flake-utils.lib.eachDefaultSystem (system:
      let
        config = { };

        overlay = pkgsNew: pkgsOld: {
          tribot = pkgsNew.haskell.lib.justStaticExecutables
            pkgsNew.haskellPackages.tribot;

          haskellPackages = pkgsOld.haskellPackages.override (old: {
            overrides = let
              oldOverrides = old.overrides or (_: _: { });

              manualOverrides = haskellPackagesNew: haskellPackagesOld: {
                simpleirc = let
                  src = builtins.fetchGit {
                    url = "https://github.com/dom96/SimpleIrc";
                    ref = "master";
                    rev = "8d156a89801be2c9b6923d85e6b199c8173e445a";
                  };
                in pkgs.haskell.lib.dontCheck
                (haskellPackagesOld.callCabal2nix "simpleirc" src { });
              };

              sourceOverrides =
                pkgsNew.haskell.lib.packageSourceOverrides { tribot = ./.; };

            in pkgsNew.lib.fold pkgsNew.lib.composeExtensions oldOverrides ([
              sourceOverrides
              manualOverrides
            ]);
          });
        };

        pkgs = import nixpkgs {
          inherit config system;
          overlays = [ overlay ];
        };

      in rec {
        packages.default = pkgs.haskellPackages.tribot;

        apps.default = {
          type = "app";
          program = "${pkgs.tribot}/bin/tribot";
        };

        devShells = {
          default = pkgs.mkShell {
            buildInputs = with pkgs; [
              haskellPackages.haskell-language-server
              haskellPackages.hlint
              haskellPackages.cabal-fmt
              haskellPackages.ormolu
              cabal-install
              zlib
              sqlite
            ];
          };
        };
      });
}
```

# Conclusion

This was a somewhat confusing journey, but to be fair, if I didn't depend on an unreleased version of a library, the very first `flake.nix` would probably have worked.

Now, the next step is to create a service on a NixOS server. That's [the next post](/posts/2023-09-16-creating-a-service-on-nixos-from-a-flake.html).

[^juststaticexecutables]: For example the `justStaticExecutables` function(?), this is [documented in nixpkgs](https://github.com/NixOS/nixpkgs/blob/b2e987dd163f7c3ff689856ebf391f557eedb2aa/doc/languages-frameworks/haskell.section.md) and seems to be used for reducing the size of the built artifact:

	> **justStaticExecutables drv**: Only build and install the executables produced by drv, removing everything that may refer to other Haskell packages' store paths (like libraries and documentation). This dramatically reduces the closure size of the resulting derivation. Note that the executables are only statically linked against their Haskell dependencies, but will still link dynamically against libc, GMP and other system library dependencies.

A blog/posts/2023-09-16-creating-a-service-on-nixos-from-a-flake.md => blog/posts/2023-09-16-creating-a-service-on-nixos-from-a-flake.md +82 -0
@@ 0,0 1,82 @@
---
title: Creating a service on NixOS from a flake
description: How to create a systemd service for a project packaged as a Nix flake
tags: nix, nixos
---

(This post is a follow-up to [Packaging a Haskell project as a Nix flake](/posts/2023-09-02-nix-flake-for-haskell-project.html).)

Since I was able to `nix build` my silly bot, I wanted to deploy it as a service on my NixOS server. This turned out to be quite easy! A helpful person on Mastodon said that I probably wanted to do something like this:

```nix
systemd.services.myprogram = { serviceConfig.ExecStart = "${my-binary}/bin/my-binary"; };
```

The only problem was getting a binary onto my server somehow. I now had a flake for the project, and some searching led me to `getFlake`, which can take a github/sourcehut/etc. git URL, such that the following would be enought to include this flake in my system configuration:

```nix
(builtins.getFlake
          "github:ehamberg/tribot/984b19ede065326a7f6ef31982c06fab9e03d72b")
```

This flake has a set of `packages` and by using the `builtins.currentSystem` function I can get the default package for the correct system:

```nix
(builtins.getFlake
          "github:ehamberg/tribot/984b19ede065326a7f6ef31982c06fab9e03d72b").packages.${builtins.currentSystem}.default
```

Once I had this, I could finally create my `serviceConfig` with an `ExecStart` pointing to the binary built from my flake:

```nix
  systemd.services.tribot = {
    after = [ "network.target" ];
    wantedBy = [ "multi-user.target" ];

    serviceConfig = {
      User = "tribot";
      Group = "tribot";
      WorkingDirectory = "/home/tribot";

      ExecStart = let
        tribot = (builtins.getFlake
          "github:ehamberg/tribot/984b19ede065326a7f6ef31982c06fab9e03d72b").packages.${builtins.currentSystem}.default;
      in "${tribot}/bin/tribot tribot irc.ipv4.libera.chat tribot 'secret' ehamberg '#tribot'";
      Restart = "on-failure";
      RestartSec = "10";
    };
  };
```

The final `services/tribot.nix`:

```nix
{ pkgs, ... }: {
  users.users.tribot = {
    isNormalUser = true;
    description = "tribot";
    group = "tribot";
  };
  users.groups.tribot = { };

  systemd.services.tribot = {
    after = [ "network.target" ];
    wantedBy = [ "multi-user.target" ];

    serviceConfig = {
      User = "tribot";
      Group = "tribot";
      WorkingDirectory = "/home/tribot";

      ExecStart = let
        tribot = (builtins.getFlake
          "github:ehamberg/tribot/984b19ede065326a7f6ef31982c06fab9e03d72b").packages.${builtins.currentSystem}.default;
      in "${tribot}/bin/tribot tribot irc.ipv4.libera.chat tribot 'secret' ehamberg '#tribot'";
      Restart = "on-failure";
      RestartSec = "10";
    };
  };
}
```

This can then be added to the `imports` array in `configuration.nix` and a `nixos-rebuild switch` later 'tis live!