@@ 0,0 1,161 @@
++++
+title = "Network booting an aarch64 SBC with u-boot and iPXE"
+date = 2022-01-07
+[taxonomies]
+tags = ["linux", "embedded"]
++++
+
+I recently started trying to figure out network booting for aarch64 single
+board computers (SBC), such as the Raspberry Pi, for a new CI I've been
+helping out with at my Igalia day job. For one reason or another, I never
+participated in the Raspberry Pi "fad" (maybe because they use Broadcom chips,
+which are [or were] notoriously unfriendly on Linux? I don't recall why... But
+I digress...)
+
+But, I do have a quite capable aarch64 SBC just laying around, literally
+collecting dust... the Purism Librem 5 DevKit!
+
+{{ resize_image_fit_width(path="../static/images/2022-01-07-sweeping-off-devkit.jpg", width=300, caption="sweeping off the devkit for recommissioning with a tiny broom") }}
+
+While not exactly a Raspberry Pi, I believe many of the concepts pre-Linux boot
+are similar and this should serve as a decent replacement until the Great Chip
+Shortage of 2020-???? is over and those things are available for purchase again.
+
+The general idea is that u-boot will execute [iPXE](https://ipxe.org/start),
+which will be responsible establishing a network connection and booting
+whatever the DHCP server on the other end tells it to boot. The end goal is to
+have it load/boot the Linux kernel and an initfs based on
+[boot2container](https://gitlab.freedesktop.org/mupuf/boot2container).
+
+This is the first in a series of posts to get there. The focus of this initial
+post is building/setting up iPXE, the DHCP server, and doing a test boot from
+u-boot.
+
+## Building iPXE and configuring the devkit
+
+The first step is to build iPXE, since I want to embed a script for it to run
+automatically on boot. I did the compilation on the devkit, since iPXE is a
+relatively small program and it didn't take too long to compile on this CPU:
+
+```sh
+$ git clone git://git.ipxe.org/ipxe.git
+$ cd ipxe/src
+
+## needed so that ipxe doesn't lock up if you want to C-b to enter the cmdline
+$ cat << EOF > config/local/nap.h
+#undef NAP_EFIX86
+#undef NAP_EFIARM
+#define NAP_NULL
+EOF
+
+## and create a simple ipxe script that will be executed when ipxe runs:
+$ cat << EOF > ipxescript
+#!ipxe
+
+:retry_dhcp
+echo Acquiring an IP
+dhcp || goto retry_dhcp
+
+echo Got the IP: $${netX/ip} / $${netX/netmask}
+
+:retry_boot
+echo Booting from DHCP...
+autoboot || goto retry_boot
+EOF
+
+## build/install:
+$ make bin-arm64-efi/snp.efi -j4 EMBED=ipxescript
+$ doas cp bin-arm64-efi/snp.efi /boot/ipxe.efi
+```
+
+Loading things in u-boot is quite tedious, since you have to specify memory
+addresses to load files into, and the correct *load command to read files into
+memory. I already have an existing install of postmarketOS on my devkit, so I
+used the `/boot` partition (formatted as `ext2`) as a home for the iPXE binary.
+I created the following U-boot helper script for loading iPXE, since typing all
+of these in becomes tiresome very quickly:
+
+```sh
+$ cat << EOF > /tmp/ipxe
+echo ===== Loading iPXE =====
+ext2load mmc 0:1 $kernel_addr_r ipxe.efi
+ext2load mmc 0:1 $fdt_addr_r imx8mq-librem5-devkit.dtb
+fdt addr $fdt_addr_r
+fdt resize
+echo ===== Running iPXE =====
+bootefi $kernel_addr_r $fdt_addr_r
+EOF
+```
+
+I'm not entirely sure if we need to specify/load the dtb, but it doesn't seem
+to hurt! Also note that this u-boot script is using `bootefi` to load the iPXE
+app. That'll be important later on when we try to boot a kernel.
+
+The u-boot script must be compiled before u-boot can execute it:
+
+```sh
+$ mkimage -A arm64 -C none -O linux -T script -d /tmp/ipxe /tmp/ipxe.scr
+$ doas cp /tmp/ipxe.scr /boot
+```
+
+In that last step, I copy it to `/boot` since I'm performing these steps on the
+devkit, and `/boot` is the `ext2` partition I'll run iPXE from when booted into
+u-boot.
+
+## Configuring dnsmasq for BOOTP/DHCP
+Now that all the necessary pieces are setup/installed on the devkit, the last
+step is to run dnsmasq on a host to provide BOOTP service to the devkit. This
+should be good enough for our purposes:
+
+```sh
+$ export workdir=/path/to/some/dir
+
+## set to the network interface that is on the same physical LAN as the devkit that dnsmasq will bind to
+$ export iface=eth0
+
+## needs to run as root since it binds to ports < 1000
+$ doas dnsmasq \
+ --port=0 \
+ --dhcp-hostsfile="$workdir"/hosts.dhcp \
+ --dhcp-optsfile="$workdir"/options.dhcp \
+ --dhcp-leasefile="$workdir"/dnsmasq.leases \
+ --dhcp-boot=grubnetaa64.efi \
+ --dhcp-range=10.42.0.10,10.42.0.100 \
+ --dhcp-script=/bin/echo \
+ --enable-tftp="$iface" \
+ --tftp-root="$workdir"/tftp \
+ --log-queries=extra \
+ --conf-file=/dev/null \
+ --log-debug \
+ --no-daemon \
+ --interface="$iface"
+```
+
+Note that the boot option sent to the client is `grubnetaa64.efi`. This is a
+binary I pulled from some Debian build of grub2 for aarch64, since it was
+annoying to have to build grub myself just for a quick smoke test.
+
+Grub isn't necessary for booting the Linux kernel, but it is a small
+application that serves as a good test to make sure that u-boot, iPXE, and
+dnsmasq are happy.
+
+If you're like me and run firewalls everywhere, you'll need to punch some holes
+in it for bootp / tftp to work.
+
+Once dnsmasq is started, the devkit is reset and the u-boot script to run iPXE
+is executed:
+
+```
+Hit any key to stop autoboot: 0
+u-boot=> env set boot_scripts ipxe.scr
+u-boot=> boot
+switch to partitions #0, OK
+mmc0(part 0) is current device
+Scanning mmc 0:1...
+Found U-Boot script /ipxe.scr
+294 bytes read in 1 ms (287.1 KiB/s)
+```
+
+{{ resize_image_fit_width(path="../static/images/2022-01-07-first-grub-boot.png", width=300, caption="PXE booting to grub") }}
+
+In Part 2, I'll cover booting the Linux kernel... Stay tuned!
