~urandom/www

f78da3c7f9b5f1f2b3b9f9a37c82f72863763d7f — Colin Arnott 1 year, 2 months ago 46be326 trunk
move to nix

Since all my other systems are moving away from arch, best to do that
here too. I added a flake, are using nixos for build, and added some
linters. There is also a new blog post detailing the process.

As noted in the post, we are faking git metadata for hugo, so that it
works in nix. This is also exposed in devShells.default, so as to ease
debugging, but only for the hugo binary.
M .build.yml => .build.yml +6 -8
@@ 1,9 1,7 @@
image: archlinux
packages:
  - hugo
  - hut
image: "nixos/unstable"
environment:
  NIX_CONFIG: "experimental-features = nix-command flakes"
tasks:
  - build: hugo --source www
  - package: tar -C www/public -cvz . > site.tar.gz
  - deploy: hut pages publish site.tar.gz --domain www.urandom.co.uk --not-found /404.html
oauth: pages.sr.ht/PAGES:RW
  - check: "nix flake check ./www"
  - deploy: "nix run ./www#deploy"
oauth: "pages.sr.ht/PAGES:RW"

A .envrc => .envrc +1 -0
@@ 0,0 1,1 @@
use flake

M .gitignore => .gitignore +5 -3
@@ 1,3 1,5 @@
public/
.hugo_build.lock
site.tar.gz
/.direnv/
/.hugo_build.lock
/public/
/site.tar.gz
result

A .prettierignore => .prettierignore +3 -0
@@ 0,0 1,3 @@
/.direnv/
/themes/beautifulhugo/
/public/

A Makefile => Makefile +15 -0
@@ 0,0 1,15 @@
DEST ?= "site.tar.gz"

all: public

clean:
	@rm -rf public site.tar.gz

install: public
	@rm -r $(DEST)
	tar --create --directory public --file $(DEST) --gzip --verbose .

public:
	hugo

.PHONY: all clean install

M config.toml => config.toml +0 -14
@@ 8,10 8,8 @@ minify = true

[Params]
  homeTitle = "Colin Arnott"
  #subtitle = "mind the gap"
  mainSections = ["post","posts"]
  logo = "urandom.jpg"
  #favicon = "img/favicon.ico"
  dateFormat = "2006-01-02"
  commit = "https://git.sr.ht/~urandom/www/commit/"
  rss = true


@@ 25,18 23,6 @@ minify = true
  hideAuthor = true
  selfHosted = true

#[[Params.bigimg]]
#  src = "img/triangle.jpg"
#  desc = "Triangle"
#[[Params.bigimg]]
#  src = "img/sphere.jpg"
#  desc = "Sphere"
#  # position: see values of CSS background-position.
#  position = "center top"
#[[Params.bigimg]]
#  src = "img/hexagon.jpg"
#  desc = "Hexagon"

[Author]
  name = "Colin Arnott"
  email = "colin@urandom.co.uk"

M content/_index.md => content/_index.md +1 -0
@@ 1,2 1,3 @@
## [kia ora](https://wikipedia.org/wiki/Kia_ora)

Thought I should probably make a personal site. Hope this is what you were looking for. Maybe I will add some posts if I find something worth sharing.

M content/post/2022-05-13.md => content/post/2022-05-13.md +1 -1
@@ 1,7 1,7 @@
---
title: hello, sourcehut
date: 2022-05-13
tags: [ "builds", "pages", "hugo", "hut", "sourcehut" ]
tags: ["builds", "pages", "hugo", "hut", "sourcehut"]
---

I made [an account](https://sr.ht/~urandom) back in 2018-11-16, and have really enjoyed watching this platform develop. The focus on easy to use, low level tooling, and no nonsense ui design always picqued my interest. However, because I forgot and pages is a [somewhat new](https://sourcehut.org/blog/2021-02-18-sourcehut-pages) feature, I launched this site on [GitLab](https://gitlab.com). Time to fix that with you now with [pages.sr.ht](https://pages.sr.ht), [git.sr.ht](https://git.sr.ht), and [builds.sr.ht](https://builds.sr.ht).

A content/post/2022-12-20.md => content/post/2022-12-20.md +261 -0
@@ 0,0 1,261 @@
---
title: nix, no the other one
date: 2022-12-20
tags: ["nix", "nixos", "flakes", "hugo"]
---

I first looked at nix back in 2016 when snap forced me to use a macbook. At the time, I was just looking for a package manager, and it was a hard sell, much harder than the linux laptop that followed a few months later. As such, it was five years before I picked it back up, and I regret not giving it the time back then.

I guess this is also a goodbye letter to archlinux, as I have moved my personal systems over to nixos earlier this year, but I mostly wanted to show off how I moved this blog over. Which seems to be the only thing I can remember to actually publish here.

Most of the leg work for nixos was [already done](https://man.sr.ht/builds.sr.ht/compatibility.md#nixos), and props to the sourcehut community therein. As well, I will push you to Xe for a wonderful [flakes intro](https://xeiaso.net/blog/nix-flakes-1-2022-02-21), as doing that here is more than out of scope.

My effort was largely about transferring logic currently in `.build.yml` into `flake.nix`. The first thing of note is that I use [flake-utils](https://github.com/numtide/flake-utils) to flatten all of the entries in the [`outputs` function's attrset](https://nixos.wiki/wiki/Flakes#Output_schema). This does not change the meaning of anything, just means that if you see `devShells.foo` or `packages.bar`, know that this maps onto `devShells.${system}.foo` or `packages.x86_64-linux.bar`.

This seemed like a great time to add linting and formatting, since that is new and easier than build or deploy. Formatting is invoked by the `nix fmt` command and one must add something like this to the `formatter` entry of the [attrset](https://nix.dev/tutorials/nix-language#attribute-set) returned by the [`outputs` function](https://nixos.wiki/wiki/Flakes#Output_schema):

> Note that `pkgs = nixpkgs.legacyPackages.x86_64-linux;`.

```nix
{
  formatter = pkgs.alejandra;
}
```

I generally prefer to use `nix fmt` to fixup the whole repo, thus we need a script that invokes several binaries, and `pkgs.writeShellApplication` works well for this:

```nix
{
  formatter = pkgs.writeShellApplication {
    name = "fmt";
    runtimeInputs = with pkgs; [
      alejandra
      statix
      nodePackages.prettier
    ];
    text = ''
      alejandra --quiet .
      statix fix
      prettier --loglevel error --write .
    '';
  };
}
```

You will also note that because we are using `runtimeInputs` and can call these tools directly as opposed to needing to inject their absolute paths: `${pkgs.statix}/bin/statix`. While I do find this ergonomic, it is mostly done so that we have a single list of dependencies that we can reuse later.

Linting will then be our first foray into [derivations](https://nix.dev/tutorials/nix-language.html#derivations). Since formatting is mutable, it can just be a script, but checks must be derivations because they are building and validating things. Like `formatter` they live in the `outputs` function under `checks`, however as the name implies you can have multiple checks per flake, unlike `formatter` for which there is only one.

```nix
{
  checks.default = pkgs.runCommandLocal "check" {
    buildInputs = [pkgs.alejandra];
  } ''
    alejandra --check ${self} && touch $out
  '';
}
```

Here we are using `pkgs.runCommandLocal` to execute a command as part of building a derivation. Now, checks are funny, since they must be derivations, but they also do not really produce a meaningful artefact, just the lack of an error. As such, we run our linter in check mode and if it succeeds we create an empty artefact. This is not terribly ergonomic, especially when we need three linters, so let us first break out our `runtimeInputs` from before in our `outputs` function:

```nix
let
  doCheck = name: text: pkgs.runCommandLocal name {
    nativeBuildInputs = checkInputs;
  } ''
    cd ${self}
    ${name} ${text} .
    touch $out
  '';
  checkInputs = [
    alejandra
    statix
    nodePackages.prettier
  ];
in {
  checks = builtins.mapAttrs doCheck {
    alejandra = "--check";
    statix = "check";
    prettier = "--check";
  };
}
```

With that out of the way, and your interests piqued, let us move onto the main afair: packaging. As the name implies, we will be adding to the `packages` entry in our attrset. I opted to keep the nix code as simple as possible, both so that we can reproduce failures without nix, and because doing everything in nix gets messy fast. As such, we have added a `Makefile` that contains `all` and `install` targets.

```Makefile
DEST ?= "site.tar.gz"

all: public

install: public
        tar --create --directory public --file $(DEST) --gzip --verbose .

public:
        hugo
```

These replace the `build` and `package` tasks from `.build.yml`, and contain sane defaults that use the current directory as input and output, reusing the same filenames as before. This lead to a simple as derivation:

```nix
{
  packages.default = pkgs.stdenv.mkDerivation {
    name = "site.tar.gz";
    src = self;
    dontConfigure = true;
    nativeBuildInputs = with pkgs; [
      gnutar
      hugo
    ];
    makeFlags = ["DEST=$(out)"];
  };
}
```

Unfortunately, some of the limitations of flakes started to rear their heads. In our case, we have been using hugo's `enableGitInfo` feature. This gives all pages a cute link to git source. This feature requires we shellout to git for information. In the before times, this was automatic, since `.git` just happened to be there, and what self respecting ci tooling does not ship with the git binary accessable?

Nix requires everything be explicit, so this did not "just work". The easiest option would just be to disable this feature, but I opted to mock git's responses so as to preserve the link. It is probably quite flaky, but as long as our dependencies are pinned, it is guaranteed not to break.

Finally, we need to deploy out this artefact, `.build.yml`'s `deploy` task for those following along at home. I opted to add an entry to `apps` and trigger it with `nix run .#deploy`:

```nix
{
  apps.deploy = let
      name = "deploy";
  in {
    type = "app";
    program = "${pkgs.writeShellApplication {
      inherit name runtimeInputs;
      text = ''
        hut pages publish ${self.packages.${sys}.default} \
          --domain www.urandom.co.uk \
          --not-found /404.html
      '';
    }}/bin/${name}";
  };
}
```

These entries are derivations with binaries, similar to `formatter`, and because we need to do custom things, I once again lean on `pkgs.writeShellApplication`. You will also notice we back reference `self.packages`, this is a way to reference another element of the `outputs` function's attrset, and in this case we want `packages.default` as defined above. Unfortunately, flake-utils cannot save us here, thus we must use the fully qualified `self.packages.${sys}.default` reference, where `sys` is being populated by flake-utils.

And with that we have a running website. Unfortunately, developing against flakes like this is a non-starter, unless you always write code without bugs. As such, we have one final step: `devShells`. These are the way flakes expose a shell environment setup with all your tools: `nix develop`. They are a killer feature of nix, and I would have lead with them, but I wanted you to see all the various places we added tooling so you could now understand why I prefered to share and inject `nativeBuildInputs` and `runtimeInputs`.

```nix
let
  checkInputs = with pkgs; [
    alejandra
    statix
    nodePackages.prettier
  ];
  nativeBuildInputs = with pkgs; [
    gnutar
    hugo
  ];
  runtimeInputs = [pkgs.hut];
in {
  devShells.default = pkgs.mkShell {
    packages = checkInputs ++ nativeBuildInputs ++ runtimeInputs;
  };
};
```

Yep, it is that easy. I did leave off the other consumers for these three values, but this way if we add a new linter, we can guarantee that if it works in `nix flake check` it is also included in `nix develop`. Sorry for how long this has been, but flakes can be tricky, and I appreciate you making it to the end. As a reward, here is the full flake, warts and all:

```nix
{
  description = "personal website";
  inputs = {
    nixpkgs.url = "github:nixos/nixpkgs/nixos-unstable";
    flake-utils.url = "github:numtide/flake-utils";
  };
  outputs = {
    flake-utils,
    nixpkgs,
    self,
  }:
    flake-utils.lib.eachSystem [flake-utils.lib.system.x86_64-linux] (sys: let
      checkInputs = with pkgs; [
        alejandra
        statix
        nodePackages.prettier
      ];
      doCheck = name: text:
        pkgs.runCommandLocal name {
          nativeBuildInputs = checkInputs;
        } "cd ${self} && ${name} ${text} . && touch $out";
      git = pkgs.writeShellScriptBin "git" ''
        case $1 in
        -C)
          printf '\n'
        ;;
        -c)
          printf '\x1e${self.rev or ".."}'
          printf '\x1f${self.shortRev or "dirty"}'
          printf '\x1f\x1fColin Arnott'
          printf '\x1fcolin@urandom.co.uk'
          printf '\x1f1970-01-01 00:00:00 +0000'
          printf '\x1f1970-01-01 00:00:00 +0000'
          printf '\n'
          find ${self} -type f -printf '%P\n'
        ;;
        *)
          exit 1
        esac
      '';
      hugo = pkgs.symlinkJoin {
        name = with pkgs.hugo; "${pname}-${version}";
        paths = [pkgs.hugo];
        nativeBuildInputs = [pkgs.makeWrapper];
        postBuild = ''
          wrapProgram $out/bin/hugo --prefix PATH : ${
            pkgs.lib.makeBinPath [git]
          }
        '';
      };
      nativeBuildInputs = with pkgs; [
        gnutar
        hugo
      ];
      pkgs = nixpkgs.legacyPackages.${sys};
      runtimeInputs = [pkgs.hut];
    in {
      apps.deploy = let
        name = "deploy";
      in {
        type = "app";
        program = "${pkgs.writeShellApplication {
          inherit name runtimeInputs;
          text = ''
            hut pages publish ${self.packages.${sys}.default} \
              --domain www.urandom.co.uk \
              --not-found /404.html
          '';
        }}/bin/${name}";
      };
      checks = builtins.mapAttrs doCheck {
        alejandra = "--check";
        statix = "check";
        prettier = "--check";
      };
      devShells.default = pkgs.mkShell {
        packages = checkInputs ++ nativeBuildInputs ++ runtimeInputs;
      };
      formatter = pkgs.writeShellApplication {
        name = "fmt";
        runtimeInputs = checkInputs;
        text = ''
          alejandra --quiet .
          statix fix
          prettier --loglevel error --write .
        '';
      };
      packages.default = pkgs.stdenv.mkDerivation {
        name = "site.tar.gz";
        src = self;
        dontConfigure = true;
        inherit nativeBuildInputs;
        makeFlags = ["DEST=$(out)"];
      };
    });
}
```

A flake.lock => flake.lock +43 -0
@@ 0,0 1,43 @@
{
  "nodes": {
    "flake-utils": {
      "locked": {
        "lastModified": 1667395993,
        "narHash": "sha256-nuEHfE/LcWyuSWnS8t12N1wc105Qtau+/OdUAjtQ0rA=",
        "owner": "numtide",
        "repo": "flake-utils",
        "rev": "5aed5285a952e0b949eb3ba02c12fa4fcfef535f",
        "type": "github"
      },
      "original": {
        "owner": "numtide",
        "repo": "flake-utils",
        "type": "github"
      }
    },
    "nixpkgs": {
      "locked": {
        "lastModified": 1671359686,
        "narHash": "sha256-3MpC6yZo+Xn9cPordGz2/ii6IJpP2n8LE8e/ebUXLrs=",
        "owner": "nixos",
        "repo": "nixpkgs",
        "rev": "04f574a1c0fde90b51bf68198e2297ca4e7cccf4",
        "type": "github"
      },
      "original": {
        "owner": "nixos",
        "ref": "nixos-unstable",
        "repo": "nixpkgs",
        "type": "github"
      }
    },
    "root": {
      "inputs": {
        "flake-utils": "flake-utils",
        "nixpkgs": "nixpkgs"
      }
    }
  },
  "root": "root",
  "version": 7
}

A flake.nix => flake.nix +96 -0
@@ 0,0 1,96 @@
{
  description = "personal website";
  inputs = {
    nixpkgs.url = "github:nixos/nixpkgs/nixos-unstable";
    flake-utils.url = "github:numtide/flake-utils";
  };
  outputs = {
    flake-utils,
    nixpkgs,
    self,
  }:
    flake-utils.lib.eachSystem [flake-utils.lib.system.x86_64-linux] (sys: let
      checkInputs = with pkgs; [
        alejandra
        statix
        nodePackages.prettier
      ];
      doCheck = name: text:
        pkgs.runCommandLocal name {
          nativeBuildInputs = checkInputs;
        } "cd ${self} && ${name} ${text} . && touch $out";
      git = pkgs.writeShellScriptBin "git" ''
        case $1 in
        -C)
          printf '\n'
        ;;
        -c)
          printf '\x1e${self.rev or ".."}'
          printf '\x1f${self.shortRev or "dirty"}'
          printf '\x1f\x1fColin Arnott'
          printf '\x1fcolin@urandom.co.uk'
          printf '\x1f1970-01-01 00:00:00 +0000'
          printf '\x1f1970-01-01 00:00:00 +0000'
          printf '\n'
          find ${self} -type f -printf '%P\n'
        ;;
        *)
          exit 1
        esac
      '';
      hugo = pkgs.symlinkJoin {
        name = with pkgs.hugo; "${pname}-${version}";
        paths = [pkgs.hugo];
        nativeBuildInputs = [pkgs.makeWrapper];
        postBuild = ''
          wrapProgram $out/bin/hugo --prefix PATH : ${
            pkgs.lib.makeBinPath [git]
          }
        '';
      };
      nativeBuildInputs = with pkgs; [
        gnutar
        hugo
      ];
      pkgs = nixpkgs.legacyPackages.${sys};
      runtimeInputs = [pkgs.hut];
    in {
      apps.deploy = let
        name = "deploy";
      in {
        type = "app";
        program = "${pkgs.writeShellApplication {
          inherit name runtimeInputs;
          text = ''
            hut pages publish ${self.packages.${sys}.default} \
              --domain www.urandom.co.uk \
              --not-found /404.html
          '';
        }}/bin/${name}";
      };
      checks = builtins.mapAttrs doCheck {
        alejandra = "--check";
        statix = "check";
        prettier = "--check";
      };
      devShells.default = pkgs.mkShell {
        packages = checkInputs ++ nativeBuildInputs ++ runtimeInputs;
      };
      formatter = pkgs.writeShellApplication {
        name = "fmt";
        runtimeInputs = checkInputs;
        text = ''
          alejandra --quiet .
          statix fix
          prettier --loglevel error --write .
        '';
      };
      packages.default = pkgs.stdenv.mkDerivation {
        name = "site.tar.gz";
        src = self;
        dontConfigure = true;
        inherit nativeBuildInputs;
        makeFlags = ["DEST=$(out)"];
      };
    });
}

M layouts/partials/footer_custom.html => layouts/partials/footer_custom.html +1 -1
@@ 4,4 4,4 @@ Or you can delete these file if you don't need it.
-->
<!-- for example, you could include some js libraries:
<script src="https://cdnjs.cloudflare.com/ajax/libs/vis/4.19.1/vis.js" integrity="sha256-HdIuWBZj4eftihsoDCJoMYjZi6aNVaw7YlUpzKT3ZxI=" crossorigin="anonymous"></script>
-->
\ No newline at end of file
-->

M layouts/partials/head_custom.html => layouts/partials/head_custom.html +1 -1
@@ 15,4 15,4 @@ Or you can delete these file if you don't need it.
-->
<!-- or you could include some additional css libraries:
<link rel="stylesheet" href="https://cdnjs.cloudflare.com/ajax/libs/vis/4.19.1/vis.css" integrity="sha256-I1UoFd33KHIydu88R9owFaQWzwkiZV4hXXug5aYaM28=" crossorigin="anonymous" />
-->
\ No newline at end of file
-->