Documentation bug reported, as FB14929037.
Yes - the offset from the string is quite large, but I just want useful output such as "ThisFunctionWasInvolved" from "ThisFunctionCausedAPanic.cpp, line 455".
Symbolicate the other frames you've got. This is a quick way to differentiate between "something is wrong with THIS symbol" vs "something systemic is wrong".
Yes, this is a “something systemic is wrong” situation. I was trying to reduce the amount of information for readers to wade through, but I am not getting useful results for any efforts at symbolicating the addresses on the back trace or the PC (program counter).
” dwarfdump" will print the contents of a dsym file, which will either show that the symbols
dwarfdump works on dSYM files, but apparently not on debug-builds-with-symbols-in-the-binary. As such, I rebuilt with separating the symbols into a dSYM file. dwarfdump provides what appears to be correct information for the symbols, but for a sanity check here is a blurb from the start of the dump:
KEXT_NAME.kext.dSYM/Contents/Resources/DWARF/KEXT_NAME(arm64e): file format Mach-O arm64
.debug_info contents:
0x00000000: Compile Unit: length = 0x000001f9, format = DWARF32, version = 0x0004, abbr_offset = 0x0000, addr_size = 0x08 (next unit at 0x000001fd)
0x0000000b: DW_TAG_compile_unit
DW_AT_producer ("Apple clang version 13.1.6 (clang-1316.0.21.2.5)")
DW_AT_language (DW_LANG_C99)
DW_AT_name ("/Users/username/Library/Developer/Xcode/DerivedData/KEXT_NAME-stuff/ArchiveIntermediates/KEXT_NAME/IntermediateBuildFilesPath/KEXT_NAME.build/Release/KEXT_NAME.build/DerivedSources/KEXT_NAME_info.c")
DW_AT_LLVM_sysroot ("/Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX12.3.sdk")
DW_AT_APPLE_sdk ("MacOSX12.3.sdk")
DW_AT_stmt_list (0x00000000)
DW_AT_comp_dir ("/Users/username/sandbox/codedirectory")
DW_AT_APPLE_optimized (true)
Since the binary is different, I triggered a new panic to work with:
Apple Silicon Panic log (sanitized)
The command line utility atos should be helpful as well, but here’s an example of what I get as output from that, when using full dSYM file information:
zsh_prompt% atos -arch arm64e -o KEXT_NAME.kext.dSYM/Contents/Resources/DWARF/KEXT_NAME -l 0xfffffe0015423fd0 0xfffffe0015426e2c
0x00002e5c (in KEXT_NAME)
If I manually look up the 0x2e5c offset in the output from dwarfdump, I can see the following:
0x00002e5c: DW_TAG_formal_parameter
DW_AT_type (0x0000000000009408 "const IOService *")
DW_AT_artificial (true)
But that's definitely not conveying the one-step-to-symbolication that atos is supposed to be able to perform. It does not appear that I am using it incorrectly, but I am definitely not getting helpful results from it.
Following the idea that perhaps the kernel slide is not being properly dealt with, I followed this link ( https://lists.apple.com/archives/darwin-kernel/2014/Jan/msg00011.html ) for advice on how to deal with kernel slide (below is all commands listed, and lack of useful information provided):
lldb commands and log for dealing with kext slide
So to answer all of the questions:
Systemic issue, I am having trouble interpreting a kernel panic log and getting valid symbols as a result of address lookup for Apple Silicon
dSYM file appears to be built properly
atos is not providing helpful information, despite having proper dSYM file
lldb commands to load images and deal with kernel slides does not yield helpful information
Post
Replies
Boosts
Views
Activity
Kevin Elliot, DTS Engineer, solved this in a private code-level support ticket.
There is yet another offset that was not being accounted for, but can be found using the command line tool otool:
zsh_prompt% otool -arch arm64e -l /Library/Extensions/KEXT_NAME.kext/Contents/MacOS/KEXT_NAME
will give a lot of output information, but a section with this information:
cmd LC_SEGMENT_64
cmdsize 232
segname __TEXT_EXEC
vmaddr 0x0000000000004000
vmsize 0x0000000000010000
Shows that an address of 0x4000 needs to be accounted for when performing the atos lookups.
By adding that value to the load address of the kext when calling atos, we get the correct / expected answer.
zsh_prompt% atos -arch arm64e -o KEXT_NAME.kext.dSYM/Contents/Resources/DWARF/KEXT_NAME -l <load address + (vmaddr of __TEXT_EXEC section)> 0xfffffe0015426e2c
com_company_kext_name::expectedFunction(OSObject*, void*, void*, void*, void*) (in KEXT_NAME) (SourceFile.cpp:1999)
Such a relief! Kudos to Kevin Elliot for finding a solution to this.
DMA issues were resolved (ordering of hi- and lo-address 32-bit registers was mixed up, resulting in bad addresses).
The steps provided for PCIDriverKit Bus Leader and Memory Space Enable are working properly - the code written above does work.
One undocumented requirement is that the name of the dext must match the dext's bundle identifier. This is most likely a bug, and I have submitted a Feedback Report for it (FB15590713), but in the meantime please make sure that your dext's "PRODUCT_NAME" matches "$(PRODUCT_BUNDLE_IDENTIFIER)" in the dext target's "Build Settings" tab.
This was noticed by the following developer forum thread as well:
https://developer.apple.com/forums/thread/133499?answerId=422125022#422125022
It is documented that the bundle identifier of the dext must be just an extension beyond the bundle identifier of the dext-hosting application.
For example, if the Dext-hosting application's bundle ID is "com.company.DextHostingApp" then the dext's bundle ID must be "com.company.DextHostingApp." where can be whatever you want, but the leading section must match the owning application's bundle ID.
Apple has changed the certificates over the years, the current ones are "Developer ID Application" and "Developer ID Installer" - you shouldn't need "Installer" unless you must run an installer specifically. (Sorry I don't recognize the "ipa" acronym, if that has relevance, or if it's just a typo from "app").
To answer your second question: yes, more than one person can have the Developer ID certificate on their machine to have multiple "build machines."
The initial generation/download of the certificate must come from (I believe) the Team Agent (not an Admin), and there are special "private keys" that you need to make sure you receive as well.
It's been a while since we've done that, but the process is something like Team Agent at developer.apple.com creates the certificate and downloads/installs it, then "exports" it from Keychain Access, and the next developer imports it to their Keychain Access app.
Thank you both for the responses.
We have accomplished the Team Agent request for entitlements for PCI DriverKit - it looks like I am able to perform all other tasks necessary at my "admin" level on the developer portal.
Following ssmith_c's advice, I was able to build, sign, notarize, and deploy a dext within a hosting application, the added export and manual selection seemed to be the difference I needed.
However, I'm still encountering some issues with Xcode 15.2 when trying to make a distributable application that works with the dext (#3 of the software types listed above). One worked, and one seemed to not work. The "not working" one is crashing based on:
Exception Type: EXC_CRASH (SIGKILL (Code Signature Invalid))
Exception Codes: 0x0000000000000000, 0x0000000000000000
Termination Reason: CODESIGNING 1 Taskgated Invalid Signature
Using codesign to inspect the signature, it seems ok, but I likely am just not seeing the issue:
daniek3@MacBook-Pro Project % codesign -dv --verbose=3 AppName.app
Executable=/Users/daniek3/repository/Project/AppName.app/Contents/MacOS/AppName
Identifier=com.company.AppName
Format=app bundle with Mach-O universal (x86_64 arm64)
CodeDirectory v=20500 size=319557 flags=0x10000(runtime) hashes=9975+7 location=embedded
Hash type=sha256 size=32
CandidateCDHash sha256=94ec693fa2c5c43167f1e8005a93d6405361072f
CandidateCDHashFull sha256=94ec693fa2c5c43167f1e8005a93d6405361072fa22fd9750416d50890c41f4e
Hash choices=sha256
CMSDigest=94ec693fa2c5c43167f1e8005a93d6405361072fa22fd9750416d50890c41f4e
CMSDigestType=2
CDHash=94ec693fa2c5c43167f1e8005a93d6405361072f
Signature size=8980
Authority=Developer ID Application: Company (<Team ID>)
Authority=Developer ID Certification Authority
Authority=Apple Root CA
Timestamp=Oct 29, 2024 at 2:23:36 PM
Info.plist entries=22
TeamIdentifier=<Team ID>
Runtime Version=14.2.0
Sealed Resources version=2 rules=13 files=3
Internal requirements count=1 size=220
As noted, however, this was in Xcode 15.2, so I'll move to Xcode 16 and give it another try.
Trying the same app that is having difficulty in the last post (above) in Xcode 16.1.
From the Organizer, choosing to distribute app as "Custom->Direct Distribution->Export->Development Team (selected)->Automatically Manage Signing" gives the following two errors:
There is a problem with the request entity - You already have a current Developer ID Application Managed (with Kext) certificate or a pending certificate request.
No profiles for 'com.company.AppName' were found - Xcode couldn't find any Developer ID provisioning profiles matching 'com.company.AppName'.
Having previously developed kexts for our hardware, yes, our developer certificate has the "kext" qualifier. This should be normal for most developers moving from kext to dext development.
There absolutely is a provisioning profile for com.company.AppName - as will be shown in the next step.
If instead I use the choices:
From the Organizer, choosing to distribute app as "Custom->Direct Distribution->Export->Development Team (selected)->Manually Manage Signing" and select "Distribution Certificate: Default (..)" and "AppName.app: " I get a new error that reads:
Profile doesn't match the entitlements file's value for the com.apple.developer.driverkit.userclient-access entitlement.
This error seems poorly articulated as well, since the entitlements file requires I enter the Bundle ID of the dext that the application will communicate with, but there is no place to declare that in the developer portal when making the provisioning profile.
Should I open a technical support incident to get this more closely looked at? This app works when built in debug, but I am not getting the settings correct to be able to distribute it.
Like you, I previously encountered a similar error in Xcode 16.0 beta - entered as FB15239007 (this pertains to the kext-related developer certificate error).
I've additionally submitted FB15634854 to describe the "Profile doesn't match the entitlements file's value for the com.apple.developer.driverkit.userclient-access entitlement." error to Apple.
FWIW, I completely rebuilt the project, thinking perhaps that I had gotten into a bad customization spot with the previous one, but I am still unable to get a combination that works in Xcode 16.1 for this.
Thank you Kevin for the explanations and the tools to distinguish which element contributed to which piece. It's taken a few days, but I've figured out a few things and can hopefully get this all settled.
First, I had changed the bundle ID of my dext to what it "should be", after learning that the bundle ID ought to be an extension of the "owning" application's bundle ID.
It turns out you can't do that from an Admin role. I kept looking at the output of the security command and seeing the older bundle ID showing up for com.apple.developer.driverkit.userclient-access., which was not the updated bundle ID I was developing now.
Backing up to this point of the thread:
That leads to the other two errors, which are actually the same underlying error. First the manual error:
Profile doesn't match the entitlements file's value for the com.apple.developer.driverkit.userclient-access entitlement.
In other words, you pointed Xcode at a provisioning profile ("2") which didn't match apps signing ("1") configuration. You were actually on the right track here;
there is no place to declare that in the developer portal [what the dext bundle ID is] when making the provisioning profile.
Your Team Agent must request DriverKit entitlements and know in advance the bundle ID of the dext you intend to create. That's how the provisioning profile (2) from the developer portal knows the bundle ID of the dext that your application wants to create a user client for. Having this more clearly documented would be helpful.
Next post I'll explain the settings that I am currently using - there are still a few lingering issues which most likely need new forum threads, so I won't initiate those conversations on this thread, but I do want to wrap this up with a clear description so that others don't hit the same walls that I did.
For this explanation, I am using Xcode 15.x. The hosting application will use com.company.HostingApp as its bundle ID and com.company.HostingApp.Driver as the dext's bundle ID, and then a second app with bundle ID com.company.ClientOnlyApp which communicates with the dext but is not the hosting application.
As noted above, the driver bundle ID of com.company.HostingApp.Driver needed to be requested from the Team Agent at the developer portal. Once that's completed, you still need to create the Identifier to be used, by going to Identifiers and creating one that matches.
Dext Identifier settings:
DriverKit Allow Third-Party User Clients ("Capabilities" tab - I need this, perhaps you do not)
DriverKit ("Additional Capabilities" tab)
DriverKit Allow Any UserClient Access ("Additional Capabilities" tab)
DriverKit PCI (Primary Match) ("Additional Capabilities" tab) (I'm making a PCI driver, so I need this)
Hosting App Identifier settings:
DriverKit Communicates with Drivers ("Capabilities" tab)
System Extension ("Capabilities" tab)
DriverKit ("Additional Capabilities" tab)
DriverKit UserClient Access ("Additional Capabilities" tab)
Client Only Identifier settings:
DriverKit Communicates with Drivers ("Capabilities" tab)
DriverKit ("Additional Capabilities" tab)
DriverKit UserClient Access ("Additional Capabilities" tab)
Once these are built, you can move to Profiles and create the provisioning profiles necessary. Since the settings are covered in the Identifiers, creating a Provisioning file only required selecting Distribution - Developer ID, then selecting the App ID (aka Identifier), then under Certificate choose your Developer Team's Certificate, and finally under Additional Entitlements there is a drop-box to select DriverKit and System Extension Template for Mac Dev ID profile / Mac Direct Dist Profile (yours may not be named this, but mine is).
And in the Xcode Project -> Target -> Signing and Capabilities, I have the following selections:
Dext Target Signing and Capabilities
Automatically manage signing checked
Team
Bundle Identifier: com.company.HostingApp.Driver
DriverKit (x2)*
DriverKit (Allow Any UserClient Access)
DriverKit (PCI Primary Match) (x2)*
(*=these are due to Xcode 15's duplication of capabilities)
Hosting App Target Signing and Capabilities
Automatically manage signing checked
Team
Bundle Identifier: com.company.HostingApp
App Sandbox
Hardened Runtime
System Extension
Client Only App Target Signing and Capabilities
Automatically manage signing checked
Team
Bundle Identifier: com.company.ClientOnlyApp
App Sandbox
Hardened Runtime
System Extension
Once all of those settings are done, I can produce distributable dext software and applications that work with the dext.
Order of steps to produce distributable software:
Hosting application's Xcode's Product -> Archive menu option
Xcode's Window -> Organizer, select the hosting application archive that was just built, and press the Distribute App button
Choose Custom
Choose Developer ID - distribute directly to customers
Choose Export - sign and export without notarizing
Choose Manually manage signing
Select Distribution certificate for your Developer Team, Hosting App provisioning file for the hosting app, and Dext provisioning file for the dext.
Click Export
Use notarytool and staple to complete the notarization process.
Repeat these steps for the Client Only application as well.
Notes & Questions:
I know that I do not have DriverKit UserClient Access selected in the Xcode Project -> Target -> Signing and Capabilities selection for either the hosting application or the client only application. This seems to be in keeping with Kevin Elliot's example, and the current situation is working for me, perhaps because the dext is set to "allow any UserClient Access"?
System Extension Redistributable is an entitlement that means your dext (or other system extension) is allowed to be distributed by another Developer Team. This is not a feature that I needed, but perhaps you might.
It is unclear what a Developer Team needs to do when the team must deploy multiple dexts - it seems like the system is built for a single dext authorization (see above post where the Team Agent must register the dext bundle ID) - how would a team select between different dexts?
Hosting applications must be placed in /Applications/, and directly there, not in a sub-directory. Given that requirement, it seems that an installer package is still the best option for distributing the host application, so that it is installed properly on the user's hard drive?