Multiple Personality ZFS Part 1: Switching Personalities

27 December 2022 · 4 minute read

I have spread story of this journey across a series of posts:

Things are so much easier with cloud servers

For my day job I spend a lot of time working on cloud-based servers that have only one operating system installed. So there’s a whole world of problems I don’t deal with on a regular basis.

Oh well, no time like the present to dust off some troubleshooting skills…

I thought that Ubuntu would be nice enough to give me the GRUB boot loader screen. But it didn’t. What was my thought process? Installing more than one operating system on a server’s disk is often silly, because you usually want it to reboot quickly, in a completely unattended fashion. But many computing professionals (and computing non-professionals!) just can’t stop tinkering with things, and want to get the most out of their computer hardware, so on a desktop with a keyboard and monitor attached it makes more sense to switch between operating systems.

So on the assumption that treating this PC like a desktop would lead to better results, I redid the Ubuntu install with desktop media instead of server media. The installer experience was more graphical, but it offered mostly the same set of options.

Mostly the same.

Not identical.

One of the side effects of this change was that I couldn’t easily use Linux LVM for the main operating system partition directly from the installer, so I chose journaling XFS instead. (A minor detail, mostly aesthetic in nature.) Another side effect was that the EFI partition used the ext4 file system whether I liked it or not. (I didn’t, because that would mean I couldn’t easily read/write it from FreeBSD. That is disappointing; I can deal with it later.) But I went through with the reinstall anyway because those were relatively minor details compared to the concept I was trying to prove.

The Ubuntu installer finished and left me with a workable machine. But I got the same problem. I still didn’t see a GRUB menu. It always booted straight into Ubuntu.

Eventually I found How to get the GRUB menu at boot-time? which directed me to change two lines near the top of /etc/default/grub:

#GRUB_TIMEOUT_STYLE=hidden
GRUB_TIMEOUT=30

So now I could reboot back into Ubuntu and get the menu. And that meant I could quite likely craft a menu entry that would help me boot FreeBSD.

You know, I didn’t embark on this path just so I could wrestle with boot loaders all the time. Oh well, I’ll get this out of the way. Good thing I set up a throwaway virtual machine for this, otherwise I’d be in some real trouble! Mucking around with partitions and boot loaders and install media is a great way to induce data loss.

This is where having built the EFI partition came in handy. After banging about a bit more, and consulting these pages in particular:

I arrived at this menu entry that I appended to /etc/grub.d/40_custom on Ubuntu:

menuentry "FreeBSD EFI" {
    set root='(hd0,gpt1)'
    chainloader /EFI/FreeBSD/loader.efi
}

Since Ubuntu desktop is nice enough to mount /boot/efi already, and auto-mount any CD you put in the drive, it was very easy to copy the necessary EFI-related files from FreeBSD media.

mkdir -p /boot/efi/EFI/FreeBSD
cp -p /media/*/*/boot/*.efi /boot/efi/EFI/FreeBSD
eject
update-grub

It turns out that only loader.efi is actually needed. (Think of this as keystroke-efficient, not disk-space-efficient.)

I rebooted from FreeBSD media and reinstalled yet again, partitioning in the shell yet again. This time I ended up with:

# gpart show da0
=>      34  83886013  da0  GPT  (40G)
        34      2014       - free -  (1.0 M)
      2048   1998848    1  efi  (1.0G)
   2000896   3999744  	2  linux-data  (1.9G)
   6000640  34000896  	3  linux-data  (16G)
  40001536   3999744  	4  linux-swap  (1.9G)
  44001280  39884767  	5  freebsd-zfs  (19G)

Which turns out to be the winning combination as far as partitions go. After a few more laps with the now-visible GRUB menu and command line, I refined the particular menu entry for FreeBSD so that it worked consistently.

In the next part of the series, we move back to the more important stuff again: actually dealing with ZFS. You know, the whole point of this whole exercise?