Flash Sector Size could be wrong
finally worked, ready to publish
more url
I'm running a yunohost based on Armbian on a RockPro64 in a nas case. When I started out it just has been a test about how well yunohost works and whether the hardware would be sufficient. I started out with two SSDs attached, but I didn't configure them initially.
I just wanted to learn about yunohost and ran the whole system on a 16GB eMM. After actually using it for about a year I decided it would be time to make use of the SSDs configured as a raid1 (mirroring).
To avoid confusion and complexity I decided that I'd want to boot of the SSDs.
I had to configure it without physical access to the host. This is what I wrote down in preparation to installing #uboot into the SPI of my yunohost remotely via ssh and make it boot of the #SoftwareRAID on the two SSDs.
To get this how-to I tried all the steps in a testing environment, then wrote an outline of the right order, reseted my environment to what I describe and started over capturing my terminal. Then I filled in the relevant terminal snippets with some explanations.
I ran my how-to against my productive yunohost extending and correcting it once along the way ...
... and stumbled over not using exactly the same hardware and not checking the raid headers. With the added information this shouldn't happen to you.
Thanks to all those wonderful FLOSS people out there who made this possible with their software and sharing of knowledge!
I'm Running #Armbian Bullseye from eMMC (sdcard should work the same) on a #Pine64 #RockPro64 (4GB). The #SBC is build into a Pine64 NAS case. Two SSDs are connected via a PCI sata controller I also bought from Pine64.
____ _ ____ __ _ _
| _ \ ___ ___| | _| _ \ _ __ ___ / /_ | || |
| |_) / _ \ / __| |/ / |_) | '__/ _ \ | '_ \| || |_
| _ < (_) | (__| <| __/| | | (_) | | (_) |__ _|
|_| \_\___/ \___|_|\_\_| |_| \___/ \___/ |_|
Welcome to Armbian 23.8.1 Bullseye with Linux 4.4.213-legacy-rockchip64
System load: 8% Up time: 1:06
Memory usage: 3% of 3.72G IP: 192.168.2.169
CPU temp: 41°C Usage of /: 15% of 28G
Last login: Wed Sep 20 09:27:35 2023 from 192.168.2.20
root@rockpro64:~# cat /etc/armbian-release /etc/debian_version
# PLEASE DO NOT EDIT THIS FILE
BOARD=rockpro64
BOARD_NAME="RockPro 64"
BOARDFAMILY=rockchip64
BUILD_REPOSITORY_URL=https://github.com/armbian/build
BUILD_REPOSITORY_COMMIT=ac829eb
LINUXFAMILY=rockchip64
ARCH=arm64
IMAGE_TYPE=stable
BOARD_TYPE=conf
INITRD_ARCH=arm64
KERNEL_IMAGE_TYPE=Image
FORCE_BOOTSCRIPT_UPDATE=
VENDOR=Armbian
BOOTSCRIPT_FORCE_UPDATE="no"
BOOTSCRIPT_DST="boot.cmd"
VERSION=23.8.1
REVISION=23.8.1
BRANCH=legacy
11.7
root@rockpro64:~# lspci
00:00.0 PCI bridge: Fuzhou Rockchip Electronics Co., Ltd RK3399 PCI Express Root Port
01:00.0 SATA controller: JMicron Technology Corp. JMB58x AHCI SATA controller
root@rockpro64:~# fdisk -l /dev/sda
Disk /dev/sda: 1.82 TiB, 2000398934016 bytes, 3907029168 sectors
Disk model: ST2000VN000-1HJ1
Units: sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 4096 bytes
I/O size (minimum/optimal): 4096 bytes / 4096 bytes
root@rockpro64:~# fdisk -l /dev/sdb
Disk /dev/sdb: 1.82 TiB, 2000398934016 bytes, 3907029168 sectors
Disk model: WDC WD20EARX-00P
Units: sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 4096 bytes
I/O size (minimum/optimal): 4096 bytes / 4096 bytes
The harddisk shown above are the ones I used in my testing environment while writing this guide.
The controller can only be used by the kernel if it had not been initialized by u-boots pci
command. There are some details in my notes.md.
Vin
to pin 1 (3.3V) of the RockPro64.)root@rockpro64:~# apt-get install mtd-utils libubootenv-tool
Reading package lists... Done
Building dependency tree... Done
Reading state information... Done
The following additional packages will be installed:
libiniparser1 libubootenv0.1
The following NEW packages will be installed:
libiniparser1 libubootenv-tool libubootenv0.1 mtd-utils
[0 upgraded, 4 newly installed, 0 to remove and 0 not upgraded.
[...]
These two packages contain tools to manipulate the u-boot image and environment we'll need later to write into the SPI.
Instead of compiling a u-boot myself I decided to keep it simple and found that Debian Trixie does offer a package for u-boot newer or equal than 2023.04 (in some forum or manual I had read that starting with this release it would be possible to boot of sata).
Get the deb file and extract it.
root@rockpro64:~# wget http://ftp.de.debian.org/debian/pool/main/u/u-boot/u-boot-rockchip_2023.07+dfsg-1_arm64.deb
--2023-09-20 09:36:19-- http://ftp.de.debian.org/debian/pool/main/u/u-boot/u-boot-rockchip_2023.07+dfsg-1_arm64.deb
Resolving ftp.de.debian.org (ftp.de.debian.org)... 141.76.2.4
Connecting to ftp.de.debian.org (ftp.de.debian.org)|141.76.2.4|:80... connected.
HTTP request sent, awaiting response... 200 OK
Length: 1887440 (1.8M) [application/vnd.debian.binary-package]
Saving to: ‘u-boot-rockchip_2023.07+dfsg-1_arm64.deb’
u-boot-rockchip_2023.07+dfsg- 100%[===============================================>] 1.80M --.-KB/s in 0.1s
2023-09-20 09:36:19 (14.5 MB/s) - ‘u-boot-rockchip_2023.07+dfsg-1_arm64.deb’ saved [1887440/1887440]
root@rockpro64:~# mkdir u-boot-rockchip_2023.07+dfsg-1_arm64.deb.d
root@rockpro64:~# dpkg-deb -X u-boot-rockchip_2023.07+dfsg-1_arm64.deb u-boot-rockchip_2023.07+dfsg-1_arm64.deb.d/
./
./usr/
[...]
./usr/lib/u-boot/rockpro64-rk3399/
./usr/lib/u-boot/rockpro64-rk3399/idbloader.img
./usr/lib/u-boot/rockpro64-rk3399/rk3399-rockpro64.dtb
./usr/lib/u-boot/rockpro64-rk3399/u-boot-nodtb.bin
./usr/lib/u-boot/rockpro64-rk3399/u-boot-spl.bin
./usr/lib/u-boot/rockpro64-rk3399/u-boot-tpl.bin
./usr/lib/u-boot/rockpro64-rk3399/u-boot.bin
./usr/lib/u-boot/rockpro64-rk3399/u-boot.img
./usr/lib/u-boot/rockpro64-rk3399/u-boot.itb
./usr/lib/u-boot/rockpro64-rk3399/uboot.elf
[...]
./usr/share/lintian/overrides/u-boot-rockchip
There's a subdirectory containing the files you'll need to prepare the u-boot image to copy into the RockPro64s SPI.
The SPI is a small flash memory that can be used to boot from. It is completely independend of sdcard, eMMC or other connected memories.)
Your harddisks need to be partitioned for software raid and the software raid needs to be configured. On the software raids md
devices you'll need to create filesystems and mount these to copy your running system onto the harddisks.
If you're just interested in how-to boot your RockPro64 of a software raid on sata devices - skip ahead to install u-boot to SPI
Create partitions on your harddisk for the software raid with the partitions you'll need. I used created three for swap, the root filesystem and a bigger data partition.
Be careful to use the right device node for your setup. You'd need to replace /dev/sda
and /dev/sdb
by whatever your system uses.
root@rockpro64:~# fdisk /dev/sda
Welcome to fdisk (util-linux 2.36.1).
Changes will remain in memory only, until you decide to write them.
Be careful before using the write command.
Device does not contain a recognized partition table.
Created a new DOS disklabel with disk identifier 0xa6985dc8.
Command (m for help): n
Partition type
p primary (0 primary, 0 extended, 4 free)
e extended (container for logical partitions)
Select (default p): p
Partition number (1-4, default 1): 1
First sector (2048-3907029167, default 2048):
Last sector, +/-sectors or +/-size{K,M,G,T,P} (2048-3907029167, default 3907029167): +14G
Created a new partition 1 of type 'Linux' and of size 14 GiB.
Command (m for help): t
Selected partition 1
Hex code or alias (type L to list all): fd
Changed type of partition 'Linux' to 'Linux raid autodetect'.
Command (m for help): n
Partition type
p primary (1 primary, 0 extended, 3 free)
e extended (container for logical partitions)
Select (default p): p
Partition number (2-4, default 2):
First sector (29362176-3907029167, default 29362176):
Last sector, +/-sectors or +/-size{K,M,G,T,P} (29362176-3907029167, default 3907029167): +4G
Created a new partition 2 of type 'Linux' and of size 4 GiB.
Command (m for help): t
Partition number (1,2, default 2):
Hex code or alias (type L to list all): fd
Changed type of partition 'Linux' to 'Linux raid autodetect'.
Command (m for help): n
Partition type
p primary (2 primary, 0 extended, 2 free)
e extended (container for logical partitions)
Select (default p): p
Partition number (3,4, default 3):
First sector (37750784-3907029167, default 37750784):
Last sector, +/-sectors or +/-size{K,M,G,T,P} (37750784-3907029167, default 3907029167): +200G
Created a new partition 3 of type 'Linux' and of size 200 GiB.
Command (m for help): t
Partition number (1-3, default 3):
Hex code or alias (type L to list all): fd
Changed type of partition 'Linux' to 'Linux raid autodetect'.
Command (m for help): w
The partition table has been altered.
Calling ioctl() to re-read partition table.
Syncing disks.
You'd need to do the same for your second harddrive. Let's take a shortcut by copying the MBR containing the partition table over to the second drive:
root@rockpro64:~# dd if=/dev/sda of=/dev/sdb bs=512 count=1
1+0 records in
1+0 records out
512 bytes copied, 0.00244502 s, 209 kB/s
This works only if your system only uses primary partitions. If you use extended partitions you'll need to do use fdisk
on the second drive, also (or find out how to copy the partition data for your extended partition).
Checking the partition table of the two drives they should be identical:
root@rockpro64:~# fdisk -l /dev/sda /dev/sdb
Disk /dev/sda: 1.82 TiB, 2000398934016 bytes, 3907029168 sectors
Disk model: ST2000VN000-1HJ1
Units: sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 4096 bytes
I/O size (minimum/optimal): 4096 bytes / 4096 bytes
Disklabel type: dos
Disk identifier: 0xa6985dc8
Device Boot Start End Sectors Size Id Type
/dev/sda1 2048 29362175 29360128 14G fd Linux raid autodetect
/dev/sda2 29362176 37750783 8388608 4G fd Linux raid autodetect
/dev/sda3 37750784 457181183 419430400 200G fd Linux raid autodetect
Disk /dev/sdb: 1.82 TiB, 2000398934016 bytes, 3907029168 sectors
Disk model: WDC WD20EARX-00P
Units: sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 4096 bytes
I/O size (minimum/optimal): 4096 bytes / 4096 bytes
Disklabel type: dos
Disk identifier: 0xa6985dc8
Device Boot Start End Sectors Size Id Type
/dev/sdb1 2048 29362175 29360128 14G fd Linux raid autodetect
/dev/sdb2 29362176 37750783 8388608 4G fd Linux raid autodetect
/dev/sdb3 37750784 457181183 419430400 200G fd Linux raid autodetect
To create the software raid devices composed of the partitions you just made you'll need the kernel module available:
root@rockpro64:~# lsmod | grep 'md_mod\|raid1 '
raid1 36864 3
md_mod 143360 6 raid456,raid0,raid1,multipath,linear,raid10
You should take care that there are not already software raids running on md devices you might use during the following steps or that your harddisks contained any data that the driver interpreted and started as software raids:
root@rockpro64:~# cat /proc/mdstat
Personalities : [linear] [multipath] [raid0] [raid1] [raid6] [raid5] [raid4] [raid10]
unused devices: <none>
The device for your root filesystem should use the older version 1.0 header format which is written at the end of the partition (-e 1.0
). By doing so you'll not need to worry whether u-boot would be able to understand the software raid.
root@rockpro64:~# mdadm -C /dev/md1 -e 1.0 -l1 -n2 /dev/sda1 /dev/sdb1
mdadm: Fail create md1 when using /sys/module/md_mod/parameters/new_array
mdadm: array /dev/md1 started.
If you get that error message and you're worried or if you do not get it and you wonder why: mdadm-error.md.
The rest of the arrays is created using the latest header format:
root@rockpro64:~# mdadm -C /dev/md2 -l1 -n2 --run /dev/sda2 /dev/sdb2
mdadm: Note: this array has metadata at the start and
may not be suitable as a boot device. If you plan to
store '/boot' on this device please ensure that
your boot-loader understands md/v1.x metadata, or use
--metadata=0.90
mdadm: Fail create md2 when using /sys/module/md_mod/parameters/new_array
mdadm: Defaulting to version 1.2 metadata
mdadm: array /dev/md2 started.
root@rockpro64:~# mdadm -C /dev/md3 -l1 -n2 --run /dev/sda3 /dev/sdb3
mdadm: Note: this array has metadata at the start and
may not be suitable as a boot device. If you plan to
store '/boot' on this device please ensure that
your boot-loader understands md/v1.x metadata, or use
--metadata=0.90
mdadm: Fail create md3 when using /sys/module/md_mod/parameters/new_array
mdadm: Defaulting to version 1.2 metadata
mdadm: array /dev/md3 started.
To make sure the creation of the software raids worked, look at /proc/mdstat
:
root@rockpro64:~# cat /proc/mdstat
Personalities : [linear] [multipath] [raid0] [raid1] [raid6] [raid5] [raid4] [raid10]
md3 : active raid1 sdb3[1] sda3[0]
209715072 blocks super 1.0 [2/2] [UU]
resync=DELAYED
bitmap: 2/2 pages [8KB], 65536KB chunk
md2 : active raid1 sdb2[1] sda2[0]
4194240 blocks super 1.0 [2/2] [UU]
resync=DELAYED
md1 : active raid1 sdb1[1] sda1[0]
14679936 blocks super 1.0 [2/2] [UU]
[=======>.............] resync = 36.2% (5319616/14679936) finish=1.4min speed=110873K/sec
unused devices: <none>
If you've been fast enough you can still see the software raids syncing.
After the software raids are synced they should look like this:
root@rockpro64:~# cat /proc/mdstat
Personalities : [linear] [multipath] [raid0] [raid1] [raid6] [raid5] [raid4] [raid10]
md3 : active raid1 sdb3[1] sda3[0]
209583104 blocks super 1.2 [2/2] [UU]
bitmap: 0/2 pages [0KB], 65536KB chunk
md2 : active raid1 sdb2[1] sda2[0]
4189184 blocks super 1.2 [2/2] [UU]
md1 : active raid1 sdb1[1] sda1[0]
14679936 blocks super 1.0 [2/2] [UU]
unused devices: <none>
When I followed this howto I already had configured my software raids a year ago. I didn't check them and failed, because they had all the same name in their header and that led to a broken mdadm.conf created by mkconf
for the initramfs.
Look at your raids:
root@rock64pro:~# for D in /dev/md[0-9]*; do mdadm -Db $D; done
ARRAY /dev/md1 metadata=1.0 name=/dev/md/1 UUID=a33a64c9:298aa4cc:8f2c834f:32eb8840
ARRAY /dev/md2 metadata=1.2 name=ups_wrong UUID=843a093d:01b6194a:dce59dfd:ddd419fd
ARRAY /dev/md3 metadata=1.2 name=ups_wrong UUID=cca47d3b:a9ea246b:fa2e39a8:ee9a7f51
Make sure all name entries for your arrays differ from each other. If you find two arrays with the same name like above /dev/md2
and /dev/md3
both named ups_wrong
your system will most probably not boot.
Start over making your arrays or rename them to give them different names.
To use the new software raids you'll need to create filesystems on them. To mount them during boot process automatically you'll need to identify them. You'll put a label on the block devices to use during the boot process.
There is no reason I know off using UUID or paths shouldn't work. I just decided to go by the most self-explaining and simple option.
Create you swap partition:
root@rockpro64:~# mkswap -L swap /dev/md2
Setting up swapspace version 1, size = 4 GiB (4289720320 bytes)
LABEL=swap, UUID=5674cbfd-e8f7-478a-8660-fb72c1e26cfd
Create the filesystems for the rest of your partitions. Make sure that you use the /dev/mdX device with header version 1.0 (in cat /proc/mdstat
the device with super 1.0
) for your root or boot partition that will contain /boot/
.
root@rockpro64:~# mkfs.ext4 -L root /dev/md1
mke2fs 1.46.2 (28-Feb-2021)
Creating filesystem with 3669984 4k blocks and 917504 inodes
Filesystem UUID: 9cb9e959-f232-4a9b-93cf-24be379edef3
Superblock backups stored on blocks:
32768, 98304, 163840, 229376, 294912, 819200, 884736, 1605632, 2654208
Allocating group tables: done
Writing inode tables: done
Creating journal (16384 blocks): done
Writing superblocks and filesystem accounting information: done
root@rockpro64:~# mkfs.ext4 -L data /dev/md3
mke2fs 1.46.2 (28-Feb-2021)
Creating filesystem with 52395776 4k blocks and 13099008 inodes
Filesystem UUID: a9924914-3532-4c2c-993b-596f5f6e37cf
Superblock backups stored on blocks:
32768, 98304, 163840, 229376, 294912, 819200, 884736, 1605632, 2654208,
4096000, 7962624, 11239424, 20480000, 23887872
Allocating group tables: done
Writing inode tables: done
Creating journal (262144 blocks): done
Writing superblocks and filesystem accounting information: done
Check that your partitions contain the labels
used further down (or make sure you know what you're doing with paths or UUIDs).
root@rockpro64:~# blkid /dev/md*
/dev/md1: LABEL="root" UUID="9cb9e959-f232-4a9b-93cf-24be379edef3" BLOCK_SIZE="4096" TYPE="ext4"
/dev/md2: LABEL="swap" UUID="5674cbfd-e8f7-478a-8660-fb72c1e26cfd" TYPE="swap"
/dev/md3: LABEL="data" UUID="a9924914-3532-4c2c-993b-596f5f6e37cf" BLOCK_SIZE="4096" TYPE="ext4"
If you're missing labels you can add them to a formated partition by using swaplabel
or e2label
.
Mount your filesystems on the software raid to /mnt
.
I decided on a root partition for the system and a data partition containing the directories /opt
, /var
and /home
to seperate operating system and user data, but at the same time to not segement the available space in too many partitions.
To use the data partition for the three directories under /
I bind-mounted subdirectiories of /data
to these directories. Initially I tried with symlinks on my yunohost, but that led to misfunctions.
root@rockpro64:~# mount LABEL=root /mnt
root@rockpro64:~# mkdir /mnt/opt /mnt/var /mnt/home /mnt/data
root@rockpro64:~# mount LABEL=data /mnt/data
root@rockpro64:~# mkdir /mnt/data/opt /mnt/data/var /mnt/data/home
root@rockpro64:~# mount --bind /mnt/data/home /mnt/home
root@rockpro64:~# mount --bind /mnt/data/var /mnt/var
root@rockpro64:~# mount --bind /mnt/data/opt /mnt/opt
Check whether your new filesystems are mounted and sized as expected:
root@rockpro64:~# mount | grep /dev/md
/dev/md1 on /mnt type ext4 (rw,relatime,data=ordered)
/dev/md3 on /mnt/data type ext4 (rw,relatime,data=ordered)
/dev/md3 on /mnt/home type ext4 (rw,relatime,data=ordered)
/dev/md3 on /mnt/var type ext4 (rw,relatime,data=ordered)
/dev/md3 on /mnt/opt type ext4 (rw,relatime,data=ordered)
root@rockpro64:~# df -h /mnt /mnt/data
Filesystem Size Used Avail Use% Mounted on
/dev/md1 14G 40K 13G 1% /mnt
/dev/md3 196G 40K 186G 1% /mnt/data
Copy your running system to the partitions mounted on /mnt
. If you add the option -v
you'll get messages for every file that has been copied.
root@rockpro64:/mnt# rsync -aH --exclude='/dev/**' --exclude='/proc/**' --exclude='/run/**' --exclude='/sys/**' --exclude='/mnt/**' / /mnt
For the rsync command expect to get some error messages. You're trying to copy a running system and some files are opened, changing and/or locked.
You need to read the errors if you get some and decide for yourself.
If you're not sure you could disable services before rerunning the rsync
-command which then will only copy changed files.
This way you can work your way forward to a copy that doesn't contain errors related to files you think are important.
Even if you do not get any error during rsync
services might break if they write to their different files while rsync copies these at different points of time.
Check whether the size of your data matches more or less:
root@rockpro64:/mnt# df -h / /mnt /mnt/data
Filesystem Size Used Avail Use% Mounted on
/dev/mmcblk1p1 28G 3.9G 24G 15% /
/dev/md1 14G 2.8G 11G 21% /mnt
/dev/md3 196G 1.2G 185G 1% /mnt/data
I had 3.9G on the eMMC I copied to a new /
and /data
partition. The new partitions add up to more or less the same amount of data.
Another way to check:
root@rockpro64:/mnt# du -xs / /mnt /mnt/data
4059992 /
2838648 /mnt
1220820 /mnt/data
root@rockpro64:/mnt# bc
bc 1.07.1
Copyright 1991-1994, 1997, 1998, 2000, 2004, 2006, 2008, 2012-2017 Free Software Foundation, Inc.
This is free software with ABSOLUTELY NO WARRANTY.
For details type `warranty'.
2838648+1220820
4059468
Write a new fstab
to your new partitions:
root@rockpro64:~# echo 'LABEL=root / ext4 defaults,errors=remount-ro 0 1
LABEL=data /data ext4 defaults 0 2
/data/opt /opt none defaults,bind 0 0
/data/var /var none defaults,bind 0 0
/data/home /home none defaults,bind 0 0
LABEL=swap swap swap defaults 0 4
tmpfs /tmp tmpfs defaults,nosuid 0 0' > /mnt/etc/fstab
Make sure that your fstab contains only labels you can find using blkid
on your software raid partitions.
If you changed the partition layout during this guide your fstab needs reflect these changes.
armbianEnv.txt
is used by the boot script for u-boot to set the u-boot environment. It contains a reference to the root partition to generate the kernel command lines parameter root=<partition>
.
You need to change rootdev
to your software raid root partition using your favourite editor.
After changing it my armbianEnv.txt
looked like this:
root@rockpro64:~# cat /mnt/boot/armbianEnv.txt
bootlogo=false
overlay_prefix=rockchip
rootdev=LABEL=root
rootfstype=ext4
console=none
verbosity=4
usbstoragequirks=0x2537:0x1066:u,0x2537:0x1068:u
Take care that the content of rootdev
really points to your new root partition in case you changed the partitioning or naming used during this guide.
I tried with various guides and different u-boot versions without success. In the end I learned more about u-boot and put together snippets of information from different howtos to get to the following. In my notes you can find stuff I tried. The comments are in german, but the terminal cut'n'paste speaks for itself in many places. At the end of those notes there are the websites I used to gather the knowledge for the following process.
The u-boot image copied to SPI is different from the image used for eMMC or sdcard. It is not contained in the debian package we unpacked earlier.
You'll need to put it together from these three pieces: u-boot-tpl.bin
, u-boot-spl.bin
and u-boot.itb
:
root@rockpro64:~# mkimage -n rk3399 -T rkspi -d /root/u-boot-rockchip_2023.07+dfsg-1_arm64.deb.d/usr/lib/u-boot/rockpro64-rk3399/u-boot-tpl.bin:/root/u-boot-rockchip_2023.07+dfsg-1_arm64.deb.d/usr/lib/u-boot/rockpro64-rk3399/u-boot-spl.bin idbloader-spi.bin
Image Type: Rockchip RK33 (SPI) boot image
Init Data Size: 71680 bytes
Boot Data Size: 114688 bytes
root@rockpro64:~# cp idbloader-spi.bin spi.img
To get some background, why you had to copy a spl (secondary program loader) and a tpl (tertiary program loader) you could read 3 minutes into this presentation.
After that there's a gap that is filled with 0xff. You need to append these (0x00 though works the same) to your spi.img
:
root@rockpro64:~# yes $'\xff' | tr -d $'\x0a' | dd of=spi.img bs=1024 seek=368 count=16
16+0 records in
16+0 records out
16384 bytes (16 kB, 16 KiB) copied, 0.00142332 s, 11.5 MB/s
Now you need to add in some firmware and information about the hardware:
root@rockpro64:~# dd if=/root/u-boot-rockchip_2023.07+dfsg-1_arm64.deb.d/usr/lib/u-boot/rockpro64-rk3399/u-boot.itb of=spi.img bs=1024 seek=384
1048+0 records in
1048+0 records out
1073152 bytes (1.1 MB, 1.0 MiB) copied, 0.0311938 s, 34.4 MB/s
You can look at the data contained in u-boot.itb using dumpimage -l /root/u-boot-rockchip_2023.07+dfsg-1_arm64.deb.d/usr/lib/u-boot/rockpro64-rk3399/u-boot.itb
Since you started with the same data like me and we followed the same steps, we should have the same spi.img
file:
root@rockpro64:~# sha256sum spi.img
3d85ed0dacad20c48fb6938813eb80829175113f7f97e4a8012dbaa89f8f56bc spi.img
First erase your SPI that shouldn't contain any meaningful data if you didn't use it, yet. If you're unsure, make a backup copy using dd if=/dev/mtd0 of=spi.backup.img bs=1024
.
root@rockpro64:~# flash_erase /dev/mtd0 0 0
Erasing 4 Kibyte @ 3f7000 -- 100 % complete
Check if you really emptied it:
root@rockpro64:~# dd if=/dev/mtd0 bs=1024 | hexdump
0000000 ffff ffff ffff ffff ffff ffff ffff ffff
*
03f8000
4064+0 records in
4064+0 records out
4161536 bytes (4.2 MB, 4.0 MiB) copied, 4.96204 s, 839 kB/s
Copy your spi.img
to SPI:
root@rockpro64:~# flashcp -v spi.img /dev/mtd0
Erasing blocks: 358/358 (100%)
Writing data: 1432k/1432k (100%)
Verifying data: 1432k/1432k (100%)
You would be ready to go if that eMMC/sdcard wouldn't still be in your RockPro64.
In the standard configuration of your u-boot version eMMC and sdcard take precedence over the harddrives. If you want to shutdown remove your sdcard/eMMC from the RockPro64 it should start from harddisks already.
If you prefer not to touch your hardware or you simply can't (like me), then you'll need to prepare an environment for u-boot which gets written to the SPI as well.
The only thing you'll need to change compared to the default environment is boot_targets
. scsi
needs to precede mmc
. The default looks like this (copied from the u-boot cli):
=> env print
arch=arm
baudrate=1500000
board=rockpro64_rk3399
board_name=rockpro64_rk3399
boot_targets=mmc1 mmc0 nvme scsi usb pxe dhcp spi
bootcmd=bootflow scan
bootdelay=2
cpu=armv8
cpuid#=5447465333332e30300000000009098a
eth1addr=06:93:08:b5:fc:e7
ethaddr=06:93:08:b5:fc:e6
fdt_addr_r=0x01f00000
fdtcontroladdr=f1f10300
fdtfile=rockchip/rk3399-rockpro64.dtb
fdtoverlay_addr_r=0x02000000
kernel_addr_r=0x02080000
kernel_comp_addr_r=0x08000000
kernel_comp_size=0x2000000
loadaddr=0x800800
partitions=uuid_disk=${uuid_gpt_disk};name=loader1,start=32K,size=4000K,uuid=${uuid_gpt_loader1};name=loader2,start=8MB,size=4MB,uuid=${u;
preboot=usb start
pxefile_addr_r=0x00600000
ramdisk_addr_r=0x06000000
script_offset_f=0xffe000
script_size_f=0x2000
scriptaddr=0x00500000
serial#=109cb483ce05242c
soc=rk3399
stderr=serial,vidconsole
stdin=serial,usbkbd
stdout=serial,vidconsole
vendor=pine64
Environment size: 1043/32764 bytes
Initially I thought it would be enough to write just the changed boot_targets
to the environment, but then only a few variables are filled with valued that differ from board to board (like mac or serial number). This wouldn't boot, because of a missiing bootcmd
.
You need to create a txt file containing your environment like this:
root@rockpro64:~# cat env.txt
arch=arm
baudrate=1500000
board=rockpro64_rk3399
board_name=rockpro64_rk3399
boot_targets=scsi mmc1 mmc0 nvme usb pxe dhcp spi
bootcmd=bootflow scan
bootdelay=2
cpu=armv8
fdt_addr_r=0x01f00000
fdtfile=rockchip/rk3399-rockpro64.dtb
fdtoverlay_addr_r=0x02000000
kernel_addr_r=0x02080000
kernel_comp_addr_r=0x08000000
kernel_comp_size=0x2000000
loadaddr=0x800800
partitions=uuid_disk=${uuid_gpt_disk};name=loader1,start=32K,size=4000K,uuid=${uuid_gpt_loader1};name=loader2,start=8MB,size=4MB,uuid=${uuid_gpt_loader2};name=trust,size=4M,uuid=${uuid_gpt_atf};name=boot,size=112M,bootable,uuid=${uuid_gpt_boot};name=rootfs,size=-,uuid=B921B045-1DF0-41C3-AF44-4C6F280D3FAE;
preboot=usb start
pxefile_addr_r=0x00600000
ramdisk_addr_r=0x06000000
script_offset_f=0xffe000
script_size_f=0x2000
scriptaddr=0x00500000
soc=rk3399
stderr=serial,vidconsole
stdin=serial,usbkbd
stdout=serial,vidconsole
vendor=pine64
The only value changed is boot_targets
. The missing values are automatically filled in by u-boot during boot.
From your env.txt
file you need to create an image you can write to the environment flash:
root@rockpro64:~# mkenvimage -s 0x8000 -o env.img env.txt
Again: You used the same software and data so your image should be the same like mine:
root@rockpro64:~# sha256sum env.img
fc4a6aebbb805008ee92981e8893748d925931f1758df384dbe5ccef0d40cc8e env.img
In this configuration the SPI area for the u-boot environment is presented by the OS as /dev/mtd1
.
You need to copy your env.img
to that device:
root@rockpro64:~# flashcp -v env.img /dev/mtd1
Erasing blocks: 8/8 (100%)
Writing data: 32k/32k (100%)
Verifying data: 32k/32k (100%)
Now that you've initialized your u-boot environment you can check and correct it using fw_printenv
and fw_setenv
. First you need to make a configuration file for these tools.
root@rockpro64:~# echo '# MTD device name Device offset Env. size Flash sector size Number of sectors
/dev/mtd1 0x0 0x8000' > /etc/fw_env.config
Check that the content is correct:
root@rockpro64:~# cat /etc/fw_env.config
# MTD device name Device offset Env. size Flash sector size Number of sectors
/dev/mtd1 0x0 0x8000
Now you can use fw_printenv
to show the environment saved in SPI or fw_setenv
to change it:
root@rockpro64:~# fw_printenv
arch=arm
baudrate=1500000
board=rockpro64_rk3399
board_name=rockpro64_rk3399
boot_targets=mmc1 mmc0 nvme usb pxe dhcp spi
bootcmd=bootflow scan
bootdelay=2
cpu=armv8
fdt_addr_r=0x01f00000
fdtfile=rockchip/rk3399-rockpro64.dtb
fdtoverlay_addr_r=0x02000000
kernel_addr_r=0x02080000
kernel_comp_addr_r=0x08000000
kernel_comp_size=0x2000000
loadaddr=0x800800
partitions=uuid_disk=${uuid_gpt_disk};name=loader1,start=32K,size=4000K,uuid=${uuid_gpt_loader1};name=loader2,start=8MB,size=4MB,uuid=${uuid_gpt_loader2};name=trust,size=4M,uuid=${uuid_gpt_atf};name=boot,size=112M,bootable,uuid=${uuid_gpt_boot};name=rootfs,size=-,uuid=B921B045-1DF0-41C3-AF44-4C6F280D3FAE;
preboot=usb start
pxefile_addr_r=0x00600000
ramdisk_addr_r=0x06000000
script_offset_f=0xffe000
script_size_f=0x2000
scriptaddr=0x00500000
soc=rk3399
stderr=serial,vidconsole
stdin=serial,usbkbd
stdout=serial,vidconsole
vendor=pine64
Uhm, I'd better correct that one.
root@rockpro64:~# fw_setenv boot_targets "scsi mmc1 mmc0 nvme usb pxe dhcp spi"
root@rockpro64:~# fw_printenv | grep boot_targets
boot_targets=scsi mmc1 mmc0 nvme usb pxe dhcp spi
This is the way you can change your u-boot environment without access to the serial console and without writing each time a new environment image to flash.
Done. That's it.
Reboot.
root@rockpro64:~# reboot
root@rockpro64:~# Connection to RockPro64Armbian closed by remote host.
Now you should check that you really run off the harddisks.
root@rockpro64:~# mount | grep /dev/md
/dev/md127 on / type ext4 (rw,relatime,errors=remount-ro,data=ordered)
/dev/md126 on /data type ext4 (rw,relatime,data=ordered)
/dev/md126 on /home type ext4 (rw,relatime,data=ordered)
/dev/md126 on /opt type ext4 (rw,relatime,data=ordered)
/dev/md126 on /var type ext4 (rw,relatime,data=ordered)
/dev/md126 on /var/log.hdd type ext4 (rw,relatime,data=ordered)
/dev/md126 on /data/var/log.hdd type ext4 (rw,relatime,data=ordered)
All mounts for my system are coming from /dev/md*
. What about your eMMC or your sdcard?
root@rockpro64:~# fdisk -l /dev/mmcblk1
/dev/mmcblk1 /dev/mmcblk1boot0 /dev/mmcblk1boot1 /dev/mmcblk1p1 /dev/mmcblk1rpmb
root@rockpro64:~# fdisk -l /dev/mmcblk1
Disk /dev/mmcblk1: 29.12 GiB, 31268536320 bytes, 61071360 sectors
Units: sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes
Disklabel type: dos
Disk identifier: 0xc40ef599
Device Boot Start End Sectors Size Id Type
/dev/mmcblk1p1 32768 60456959 60424192 28.8G 83 Linux
root@rockpro64:~# mount | grep mmcblk1
root@rockpro64:~#
It is not used.
If you
Please feel free to open an issue on sourcehut.
Merge request are welcome!