Booting to Linux root filesystem in a VZVirtualMachine

The Running Linux in a Virtual Machine sample code demonstrates starting a Linux Virtual Machine.

But the example only boots to the RAM disk, leaving you in an emergency shell. It does not show how to boot to a disk containing the Linux filesystem.

With the sample code unaltered, I can use the Ubuntu RAM disk and kernel files from https://cloud-images.ubuntu.com/releases/23.10/release/unpacked/ to boot Linux like this:

./LinuxVirtualMachine /Users/username/Downloads/ubuntu-23.10-server-cloudimg-arm64-vmlinuz-generic /Users/username/Downloads/ubuntu-23.10-server-cloudimg-arm64-initrd-generic

But this fails to fully boot Ubuntu because no root is specified in the bootloader:

Begin: Mounting root file system ... Begin: Running /scripts/local-top ... done.
Begin: Running /scripts/local-premount ... [    2.013998] Btrfs loaded, zoned=yes, fsverity=yes
Scanning for Btrfs filesystems
done.
No root device specified. Boot arguments must include a root= parameter.

And so Ubuntu drops you to the emergency BusyBox shell.

If I mount the root disk image from the release page at https://cloud-images.ubuntu.com/releases/23.10/release/ and specify the root in the bootloader, we get a little further, but Linux can not see the mounted disk:

diff --git a/LinuxVirtualMachine/main.swift b/LinuxVirtualMachine/main.swift
index bf32924..0977b9e 100644
--- a/LinuxVirtualMachine/main.swift
+++ b/LinuxVirtualMachine/main.swift
@@ -10,7 +10,7 @@ import Virtualization
 
 // MARK: Parse the Command Line
 
-guard CommandLine.argc == 3 else {
+guard CommandLine.argc == 4 else {
     printUsageAndExit()
 }
 
@@ -25,6 +25,11 @@ configuration.memorySize = 2 * 1024 * 1024 * 1024 // 2 GiB
 configuration.serialPorts = [ createConsoleConfiguration() ]
 configuration.bootLoader = createBootLoader(kernelURL: kernelURL, initialRamdiskURL: initialRamdiskURL)
 
+let diskImageURL = URL(fileURLWithPath: CommandLine.arguments[3], isDirectory: false)
+let diskImageAttachment = try VZDiskImageStorageDeviceAttachment(url: diskImageURL, readOnly: false)
+let storageDeviceConfiguration = VZVirtioBlockDeviceConfiguration(attachment: diskImageAttachment)
+configuration.storageDevices = [storageDeviceConfiguration]
+
 do {
     try configuration.validate()
 } catch {
@@ -71,7 +76,11 @@ func createBootLoader(kernelURL: URL, initialRamdiskURL: URL) -> VZBootLoader {
         // Use the first virtio console device as system console.
         "console=hvc0",
         // Stop in the initial ramdisk before attempting to transition to the root file system.
-        "rd.break=initqueue"
+        "rd.break=initqueue",
+        // Give time for the boot image to be available.
+        "rootdelay=5",
+        // Specify the boot image.
+        "root=/dev/vda"
     ]
 
     bootLoader.commandLine = kernelCommandLineArguments.joined(separator: " ")
@@ -104,6 +113,6 @@ func createConsoleConfiguration() -> VZSerialPortConfiguration {
 }
 
 func printUsageAndExit() -> Never {
-    print("Usage: \(CommandLine.arguments[0]) <kernel-path> <initial-ramdisk-path>")
+    print("Usage: \(CommandLine.arguments[0]) <kernel-path> <initial-ramdisk-path> <bootable-filesystem-image-path>")
     exit(EX_USAGE)
 }

Output:

./LinuxVirtualMachine /Users/username/Downloads/ubuntu-23.10-server-cloudimg-arm64-vmlinuz-generic /Users/username/Downloads/ubuntu-23.10-server-cloudimg-arm64-initrd-generic /Users/username/Downloads/ubuntu-23.10-server-cloudimg-arm64.img

...snip...

Gave up waiting for root file system device.  Common problems:
 - Boot args (cat /proc/cmdline)
   - Check rootdelay= (did the system wait long enough?)
 - Missing modules (cat /proc/modules; ls /dev)
ALERT!  /dev/vda does not exist.  Dropping to a shell!

If I instead create a RAW disk image formatted as APFS with the contents of the root drive from the Ubuntu releases page, the mount works but Linux can not read the disk (presumably due to the APFS formatting?):

./LinuxVirtualMachine /Users/username/Downloads/ubuntu-23.10-server-cloudimg-arm64-vmlinuz-generic /Users/username/Downloads/ubuntu-23.10-server-cloudimg-arm64-initrd-generic /Users/username/Desktop/ubuntu-23.10-server.dmg

...snip...

Warning: Type of root file system is unknown, so skipping check.
mount: mounting /dev/vda on /root failed: Invalid argument
done.
Begin: Running /scripts/local-bottom ... done.
Begin: Running /scripts/init-bottom ... mount: mounting /dev on /root/dev failed: No such file or directory
mount: mounting /dev on /root/dev failed: No such file or directory
done.
mount: mounting /run on /root/run failed: No such file or directory

To make that disk image, I did:

hdiutil create -size 2g -fs "HFS+" -volname "EmptyImage" ubuntu-23.10-server

hdiutil attach ubuntu-23.10-server.dmg

diskutil eraseDisk APFS UbuntuFS disk4 // where disk4 is the mounted drive number from the previous step

sudo cp -R /path/to/extracted-ubuntu-root-filesystem/* /Volumes/UbuntuFS/

hdiutil detach disk4

What am I missing in order to transition from booting from the RAM disk to booting from the root filesystem?