Current status of kernel symbolication on M1/arm64?

Question

lundman OP

Created May ’21

Replies 2

Boosts 0

Views 1.7k

Participants 1

So what is the current status of symbolication on the M1? When I trigger something like:

Code Block panic(cpu 5 caller 0xfffffe0027b72dc8): Break 0xC472 instruction exception from kernel. Ptrauth failure with DA key resulted in 0xbffffe16708b1aa0 at pc 0xfffffe002763c748, lr 0xfffffe00266449d4 (saved state: 0xfffffe30b4fc3470)
OS version: 20E241
Kernel version: Darwin Kernel Version 20.4.0: Thu Apr 22 21:46:41 PDT 2021; root:xnu-7195.101.2~1/RELEASE_ARM64_T8101
Fileset Kernelcache UUID: 0B829878C98BF0B6E3AF7BF571B60BF2
Kernel UUID: 1DC99FEF-0771-3229-974C-9B18710700AE
KernelCache slide: 0x000000001f764000
KernelCache base:  0xfffffe0026768000
Kernel slide:      0x00000000202a4000
Kernel text base:  0xfffffe00272a8000
Kernel text exec base:  0xfffffe0027370000
Panicked task 0xfffffe166ef76730: 251 pages, 1 threads: pid 1007: zfs
Panicked thread: 0xfffffe166acb1980, backtrace: 0xfffffe30b4fc2b80, tid: 10850
		  lr: 0xfffffe00273be920  fp: 0xfffffe30b4fc2bf0
		  lr: 0xfffffe00266449d4  fp: 0xfffffe30b4fc3800
		  lr: 0xfffffe002650ab60  fp: 0xfffffe30b4fc3830
		  lr: 0xfffffe002650fad4  fp: 0xfffffe30b4fc3900
		  lr: 0xfffffe002650dc88  fp: 0xfffffe30b4fc39e0
		  lr: 0xfffffe0026517798  fp: 0xfffffe30b4fc3a10
      Kernel Extensions in backtrace:
         org.openzfsonosx.zfs(2.0)[EB1A7CDB-C33F-3E0A-A7C2-316765670F52]@0xfffffe002641c000->0xfffffe0026647fff

It would be nice to be able to look those symbols up.

But both atos and lldb give "clearly not the correct symbols" for kext, and kernel;

Code Block # atos -o /Library/Extensions/zfs.kext/Contents/MacOS/zfs -arch arm64e -l 0xfffffe002641c000 0xfffffe00266449d4 0xfffffe002650ab60 0xfffffe002650fad4 0xfffffe002650dc88 0xfffffe0026517798 0xfffffe002763f82c
ZSTD_compressBlock_btopt (in zfs) + 140
dsl_dataset_get_holds (in zfs) (dsl_userhold.c:677)
ldi_open_by_name (in zfs) (ldi_osx.c:1906)
hkdf_sha512 (in zfs) (hkdf.c:162)
handle_unmap_iokit (in zfs) (ldi_iokit.cpp:2008)
vmem_init.initial_default_block (in zfs) + 12695596

Almost so random it could be ASLR.

Annoyingly keepsyms=1 does not work here (or with this type of crash?) and debug=x0144 is ignored (it just boots again).

Boost

Answer 1

lundman OP

May ’21

Ah ok, my bad, it didnt pay attention to boot-args as I'd "cdrutil enable --without nvram" when KDK says just to disable. Not sure if either works completely, as I have no wired network for MBP yet.

But the symbolication question is what I am hoping to have answered, like when users send you panic reports.

0

Answer 2

lundman OP

May ’21

One of the users noticed something interesting;

Code Block # objdump -S /Library/Extensions/zfs.kext/Contents/MacOS/zfs|less
0000000000088000 _abd_verify:

Using the first symbols offset, 0x88000, the stack posted in the panic makes sense. So change
load address 0xfffffe002641c000 - 0x88000 = 0xFFFFFE0026394000.

Code Block # atos -o /Library/Extensions/zfs.kext/Contents/MacOS/zfs -arch arm64e -l 0xFFFFFE0026394000 0xfffffe00266449d4 0xfffffe002650ab60 0xfffffe002650fad4 0xfffffe002650dc88 0xfffffe0026517798 0xfffffe002763f82c
getf (in zfs) (spl-vnode.c:218)
zfs_file_get (in zfs) (zfs_file_os.c:412)
zfs_ioc_send_new (in zfs) (zfs_ioctl.c:6472)
zfsdev_ioctl_common (in zfs) (zfs_ioctl.c:7603)
zfsdev_ioctl (in zfs) (zfs_ioctl_os.c:234)

Similarly, lldb can be made to work with offsets - 0x88000, as loaded from 0. Not when using load address.

The actual panic? Called vnode_specrdev() on a vnode not CHR/BLK.

0