Explore the core architecture of the operating system, including the kernel, memory management, and process scheduling.

Post

Replies

Boosts

Views

Activity

Understanding `EINTR`
I’ve talked about EINTR a bunch of times here on DevForums. Today I found myself talking about it again. On reading my other explanations, I didn’t think any of them were good enough to link to, so I decided to write it up properly. If you have questions or comments, please put them in a new thread here on DevForums. Use the App & System Services > Core OS topic area so that I see it. Share and Enjoy — Quinn “The Eskimo!” @ Developer Technical Support @ Apple let myEmail = "eskimo" + "1" + "@" + "apple.com" Understanding EINTR Many BSD-layer routines can fail with EINTR. To see this in action, consider the following program: import Darwin func main() { print("will read, pid: \(getpid())") var buf = [UInt8](repeating: 0, count: 1024) let bytesRead = read(STDIN_FILENO, &buf, buf.count) if bytesRead < 0 { let err = errno print("did not read, err: \(err)") } else { print("did read, count: \(bytesRead)") } } main() It reads some bytes from stdin and prints the result. Build this and run it in one Terminal window: % ./EINTRTest will read, pid: 13494 Then, in other window, stop and start the process by sending it the SIGSTOP and SIGCONT signals: % kill -STOP 13494 % kill -CONT 13494 In the original window you’ll see something like this: % ./EINTRTest will read, pid: 13494 zsh: suspended (signal) ./EINTRTest % did not read, err: 4 [1] + done ./EINTRTest When you send the SIGSTOP the process stops and the shell tells you that. But looks what happens when you continue the process. The read(…) call fails with error 4, that is, EINTR. The read man page explains this as: [EINTR] A read from a slow device was interrupted before any data arrived by the delivery of a signal. That’s true but unhelpful. You really want to know why this error happens and what you can do about it. There are other man pages that cover this topic in more detail — and you’ll find lots of info about it on the wider Internet — but the goal of this post is to bring that all together into one place. Signal and Interrupts In the beginning, Unix didn’t have threads. It implemented asynchronous event handling using signals. For more about signals, see the signal man page. The mechanism used to actually deliver a signal is highly dependent on the specific Unix implementation, but the general idea is that: The system decides on a specific process (or, nowadays, a thread) to run the signal handler. If that’s blocked inside the kernel waiting for a system call to complete [1], the system unblocks the system call by failing it with an EINTR error. Thus, every system call that can block [2] might fail with an EINTR. You see this listed as a potential error in the man pages for read, write, usleep, waitpid, and many others. [1] There’s some subtlety around the definition of system call. On traditional Unix systems, executables would make system calls directly. On Apple platforms that’s not supported. Rather, an executable calls a routine in the System framework which then makes the system call. In this context the term system call is a shortcut for a System framework routine that maps to a traditional Unix system call. [2] There’s also some subtlety around the definition of block. Pretty much every system call can block for some reason or another. In this context, however, a block means to enter an interruptible wait state, typically while waiting for I/O. This is what the above man page quote is getting at when it says slow device. Solutions This is an obvious pitfall and it would be nice if we could just get rid of it. However, that’s not possible due to compatibility concerns. And while there are a variety of mechanism to automatically retry a system call after a signal interrupt, none of them are universally applicable. If you’re working on a large scale program, like an app for Apple’s platforms, you only good option is to add code to retry any system call that can fail with EINTR. For example, to fix the program at the top of this post you might wrap the read(…) system call like so: func readQ(_ d: Int32, _ buf: UnsafeMutableRawPointer!, _ nbyte: Int) -> Int { repeat { let bytesRead = read(d, buf, nbyte) if bytesRead < 0 && errno == EINTR { continue } return bytesRead } while true } Note In this specific case you’d be better off using the read(into:retryOnInterrupt:) method from System framework. It retries by default (if that’s not appropriate, pass false to the retryOnInterrupt parameter). You can even implement the retry in a generic way. See the errnoQ(…) snippet in QSocket: System Additions. Library Code If you’re writing library code, it’s important that you handle EINTR so that your clients don’t have to. In some cases it might make sense to export a control for this, like the retryOnInterrupt parameter shown in the previous section, but it should default to retrying. If you’re using library code, you can reasonably expect it to handle EINTR for you. If it doesn’t, raise that issue with the library author. And you get this error back from an Apple framework, like Foundation or Network framework, please file a bug against the framework.
0
0
109
Oct ’24
NSProcessInfo operatingSystemVersion generates warning CFPropertyListCreateFromXMLData(): Old-style plist parser: missing semicolon in dictionary
Consider this very trivial code which accesses the operatingSystemVersion property of NSProcessInfo as documented at https://developer.apple.com/documentation/foundation/nsprocessinfo/1410906-operatingsystemversion osversion.c: #include <Foundation/Foundation.h> int main(int argc, char *argv[]) { NSOperatingSystemVersion osVersion = [[NSProcessInfo processInfo] operatingSystemVersion]; fprintf(stderr, "OS version: %ld.%ld.%ld\n", osVersion.majorVersion, osVersion.minorVersion, osVersion.patchVersion); } Compile it: /usr/bin/clang -isysroot /Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX13.1.sdk -iframework /Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX13.1.sdk/System/Library/Frameworks -x objective-c -o a.out -framework Foundation osversion.c Then run it: ./a.out It works fine and prints the OS version: OS version: 14.6.1 Run it again and pass it some arbitrary program arguments: ./a.out foo bar Still continues to work fine and prints the output: OS version: 14.6.1 Now run it again and this time pass it two program arguments, the first one being - and the second one being something of the form {x=y} ./a.out - {x=y} This time notice how it prints a couple of warning logs from CFPropertyListCreateFromXMLData before printing the output: 2024-10-11 11:18:03.584 a.out[61327:32412190] CFPropertyListCreateFromXMLData(): Old-style plist parser: missing semicolon in dictionary on line 1. Parsing will be abandoned. Break on _CFPropertyListMissingSemicolon to debug. 2024-10-11 11:18:03.585 a.out[61327:32412190] CFPropertyListCreateFromXMLData(): Old-style plist parser: missing semicolon in dictionary on line 1. Parsing will be abandoned. Break on _CFPropertyListMissingSemicolon to debug. OS version: 14.6.1 As far as I can see there's nothing wrong in the code nor the user inputs to the program. Is this some issue in the internal implementation of NSProcessInfo? Should this be reported as an issue through feedback assistant (which category)? Although this example was run on 14.6.1 of macos, the issue is reproducible on older versions too.
5
0
321
Oct ’24
IOS18 function centraliManager: didDiscoverPeripheral: advertising Data: RSSI The advertisement data does not have kCBAdDataLocalName.
iOS 17 advertisementData: { kCBAdvDataIsConnectable = 1; kCBAdvDataLocalName = CZL2; kCBAdvDataRxPrimaryPHY = 1; kCBAdvDataRxSecondaryPHY = 0; kCBAdvDataServiceUUIDs = ( FFE0 ); kCBAdvDataTimestamp = "750419647.067132"; } iOS 18 advertisementData: { kCBAdvDataIsConnectable = 0; kCBAdvDataRxPrimaryPHY = 0; kCBAdvDataRxSecondaryPHY = 0; kCBAdvDataServiceUUIDs = ( FFE0 ); kCBAdvDataTimestamp = "750420105.457082"; } What should I do if the key value pair of kCBAdvDataLocalName disappears?
1
0
233
Oct ’24
Access denied to Hypervisor redistributor register
Hi! I would like try to boot the Linux kernel with the Hypervisor framework and see how far I get. So far the kernel runs up to the point where it's trying to identify the redistributor of the Hypervisor's GICv3, but I get an exception when it's reading the memory-mapped GICR_FIDR2 register. I tried the same via hv_gic_get_redistributor_reg() and get HV_DENIED. What could be the reason for this exception? I believe I've initialized enough of the GIC for it to work. No interrupts yet, though. It is of course entirely possible I forgot to set/clear some bits, but there are several redistributor registers missing in the framework, so it's not possible to do the full initialization a hardware GIC v3 implementation needs. I assume the Hypervisor's GIC abstraction takes care of several steps internally. What are the steps to initialize the HVF's GIC? Do you have a working example? I couldn't find anything on the internet. The popular virtualization software out there all seem to bring their own emulated interrupt controller. I'm using Sequoia 15.0.1. Thank you for any hints!
3
0
236
Oct ’24
iOS 17.6.1 App crashes during Matter provisioning
The app crashes after the system Matter provisioning dialog disappears. Fatal Exception: NSInvalidArgumentException *** +[NSString stringWithUTF8String:]: NULL cString Fatal Exception: NSInvalidArgumentException 0 CoreFoundation 0x83f20 __exceptionPreprocess 1 libobjc.A.dylib 0x172b8 objc_exception_throw 2 Foundation 0x6194 +[NSString allocWithZone:] 3 Matter 0x471e30 MTRDeviceControllerStorageClasses 4 libsystem_dnssd.dylib 0x48ac CallbackWithError 5 libsystem_dnssd.dylib 0x2a10 DNSServiceProcessResult 6 libdispatch.dylib 0x3dd4 _dispatch_client_callout 7 libdispatch.dylib 0x72d8 _dispatch_continuation_pop 8 libdispatch.dylib 0x1b1c8 _dispatch_source_latch_and_call 9 libdispatch.dylib 0x19d8c _dispatch_source_invoke 10 libdispatch.dylib 0xb284 _dispatch_lane_serial_drain 11 libdispatch.dylib 0xbf64 _dispatch_lane_invoke 12 libdispatch.dylib 0x16cb4 _dispatch_root_queue_drain_deferred_wlh 13 libdispatch.dylib 0x16528 _dispatch_workloop_worker_thread 14 libsystem_pthread.dylib 0x4934 _pthread_wqthread 15 libsystem_pthread.dylib 0x10cc start_wqthread
0
0
153
Oct ’24
getattrlistbulk lists same files over and over on macOS 15 Sequoia
A customer of mine reported that since updating to macOS 15 they aren't able to use my app anymore, which performs a deep scan of selected folders by recursively calling getattrlistbulk. The problem is that the app apparently keeps scanning forever, with the number of scanned files linearly increasing to infinity. This happens for some folders on a SMB volume. The customer confirmed that they can reproduce the issue with a small sample app that I attach below. At first, I created a sample app that only scans the contents of the selected folder without recursively scanning the subcontents, but the issue didn't happen anymore, so it seems to be related to recursively calling getattrlistbulk. The output of the sample app on the customer's Mac is similar to this: start scan /Volumes/shares/Backup/Documents level 0 fileManagerCount 2847 continue scan /Volumes/shares/Backup/Documents new items 8, sum 8, errno 34 /Volumes/shares/Backup/Documents/A.doc /Volumes/shares/Backup/Documents/B.doc ... continue scan /Volumes/shares/Backup/Documents new items 7, sum 1903, errno 0 /Volumes/shares/Backup/Documents/FKV.pdf /Volumes/shares/Backup/Documents/KFW.doc /Volumes/shares/Backup/Documents/A.doc /Volumes/shares/Backup/Documents/B.doc ... which shows that counting the number of files in the root folder by using try FileManager.default.contentsOfDirectory(atPath: path).count returns 2847, while getattrlistbulk lists about 1903 files and then starts listing the files from the beginning, not even between repeated calls, but within a single call. What could this issue be caused by? (The website won't let me attach .swift files, so I include the source code of the sample app as a text attachment.) ViewController.swift
15
0
369
Oct ’24
NSFileManager contentsOfDirectoryAtPath:error: returns NSFileReadUnknownError with underlying POSIX EINTR
Our backup app (Arq) is encountering random errors for some users on macOS Sequoia. The method [NSFileManager contentsOfDirectoryAtPath:error:] returns nil with an NSError domain NSCocoaErrorDomain, code 256 ("NSFileReadUnknownError"). The NSError's NSUnderlyingError key is an NSError with domain NSPOSIXErrorDomain and code 4 (EINTR). Sometimes waiting and retrying works fine; sometimes 5 retries still fail. For some users it happens on different directories each time they try to back up. What is causing this? Are we supposed to use a different API to get directory contents these days?
1
0
258
Oct ’24
Can I hand Matter commissioning over to the Home App?
Instead of using the MatterSupport framework to commission a device, I'd much rather throw the process over to the Home App if possible. I noticed that if I open my camera app and scan a Matter QR code, that starts the process to commission the device in the Home App. That means the Home App must be able to handle a url scheme with the QR code details. Is this available for developers to deep link into the Home App? Launching the QR code as a url itself does not work so Apple Home isn't capturing the MT url scheme. Otherwise I can open Apple Home with com.apple.home:// but have no idea what params would need to be passed to that scheme. if let url = URL(string: "MT:E9.01EEI141RUX5.210") { UIApplication.shared.open(url) { result in print(result) } } Is there something in the documentation I'm missing that allows for this?
2
0
263
Oct ’24
get properties of device in kIOTerminatedNotification handler
Hello, I am facing with misunderstanding how to read usb device properties correctly. The notification 'kIOTerminatedNotification' is delivered after an IOService has been terminated. Can I use IORegistryEntryCreateCFProperties() to get properties of terminated device? I am asking because I/O Registry is a dynamic database that captures the connections of all driver and nub objects currently active. Howerver, can we say that terminated device is still active? If IORegistryEntryCreateCFProperties() can not be used, are there any other way? (e.g. using Device Interface) Thank you in advance!
0
0
176
Oct ’24
"Unable to find file provider extension with identifier" error
I’m developing a file provider extension for macOS; I’m working with xcode 16 and macOS Sequoia. I created an host application via xcode with a simple button “Add domain” that triggers the following code: let domain = NSFileProviderDomain(identifier: NSFileProviderDomainIdentifier(rawValue: "me.piranef.fileprovider"), displayName: "piranef") NSFileProviderManager.add(domain) { theError in NSLog(">>> ERROR: \(theError?.localizedDescription ?? "No error")") } Note: I provide the link to the whole project on GitHub below. Finally I added via xcode a file provider target: At this point everything should be ok to run a simple stub application that once running add a piranef file provider visible under any file manager window in finder. But the following error appears: No file provider was found with the identifier “me.piranef.MyFileProviderTester” My suspect is that despite the target has been created by xcode, some setup in some .plist or .entitlement file must be changed manually or some tricky key added to make the file provider extension visible to the hosting application. I tried to manually change some setup that appeared logical for me like: The product bundle identifier in the target -> build settings of the extension: App Groups in the .entitlements file of the extension that seems set to a placeholder file, set to the same value of the host application: An hint I got reading the readme file of the FruitBasket sample application (by Apple) is to embed without signing the extension into the main app: Done! It’s ok! To give all possible information I uploaded the whole project into my github profile at: https://github.com/fpiraneo/fileproviderstub/ Any hint is welcome; I already googled or searched in StackOverflow or even asked ChatGPT for help but with no results. Even other users are experiencing the same issue and posting on StackOverflow with no answers: "Error adding File Provider domain: No valid file provider found with identifier ‘MyApp.FinderExtensionHost’ on MacOS” on StackOverflow
3
0
242
Oct ’24
NFC (ISO7816)- Tag Connection Lost IPhone 15 & 16 variants
NFC reading starts but after BAC, during the reading of DG2 the NFC stops reading and returns with Tag connection Lost error . It almost fails everytime on iPhone 15 Pro max with the following error. What's the solution for this? This is really important because our app entirely relies on the NFC data to proceed forward. This always fails in iPhone 15 & 16. Same card works fine on older iPhone models. What change should we do or include for the newer models. Error Domain=NFCError Code=102 "Tag response error / no response" UserInfo={NSLocalizedDescription=Tag response error / no response, NSUnderlyingError=0x303bd6af0 {Error Domain=nfcd Code=29 "Tag Error" UserInfo={NSLocalizedDescription=Tag Error, NSUnderlyingError=0x303bd6910 {Error Domain=com.apple.nfstack Code=20 "No response from tag" UserInfo={NSLocalizedDescription=No response from tag}}}}}
2
1
283
Oct ’24
eslogger man page strange reference
In the man page for the eslogger tool, there is a reference to the jq tool. Postprocess the output in a shell pipeline with jq: % sudo eslogger exec | jq -r 'select(.process.executable.path == "/bin/zsh")|"(.process.audit_token.pid): (.process.executable.path) -(.event.exec.target.executable.path)"' The problem is that the jq tool is not installed by default with macOS. [Q] Isn't the idea that the man page should only reference tools that are part of the standard macOS distribution (or can be downloaded and installed by the OS when you try to run them, like with some developer tools)?
2
0
210
Oct ’24
Kernel Development Kit Missing
Hello, It seems like the Kernel Debug Kit for macOS 15.0.1 (24A348) is missing from the list of downloads at developer.apple.com. It would be great if you could add them to the list of available downloads. When trying to rebuild the kernel it fails with the following error message: Error Domain=KMErrorDomain Code=34 "Missing Developer Kit: As of macOS 13.0, you will need to install a KDK matching your build 24A348 to rebuild kernel collections." UserInfo={NSLocalizedDescription=Missing Developer Kit: As of macOS 13.0, you will need to install a KDK matching your build 24A348 to rebuild kernel collections.} But my macOS version is 15.0.1. Is there a workaround for this?
1
0
319
Oct ’24
CBPeripheral delegate callback of `peripheralIsReady(toSendWriteWithoutResponse:)` doesn't happen when app in background
My team has an app that uses BTLE heavily, and has been doing so successfully, including no issues continuing to receive data in the background and updating things in the app (for recording workouts). We have a BTLE write queue that only tries to write when the CBPeripheral.canSendWriteWithoutResponse property is true, or when we get the notification from the system in peripheralIsReady(toSendWriteWithoutResponse:). This is used as a means to rate limit data transfer, as we transfer files, as well as require that packets always arrive in the correct order due to blob encoding. However, we had a new requirement come in to periodically write data out to a connected peripheral. I noticed that as soon as the app was in the background, despite other delegate callbacks coming in, like didRecieveUpdatedValue:, neither the property canSendWriteWithoutResponse nor the delegate callback were called any longer. This meant our write queue didn't think it had permission to write, and packets would just stack up. The failure to deliver these updates didn't occur immediately after backgrounding, but did within 2-5s of backgrounding. If, when in the background, I ignore the changing of that property, and instead just write the data to the peripheral, it works! Can anyone explain why, despite other CBPeripheral callbacks happening when in the background, this one does not?
1
0
210
Oct ’24
Getting or setting UID with HCE
Hi, we are trying to get or set the NFC UID when doing HCE CardSession emulation, but can't find any way to either get the chip UID used in a session, or set it beforehand. Is that possible with a normal HCE CardSession eventStream received ADPU event? Or is another framework/product needed for this? We've done a Interoperability request (INTEROP-214), which lead us to using HCE. Our usecase is specifically interacting with EV chargers, that only support using the chips UID for identification. Can Apple Pay / Wallet be an alternative to do such low level handling?
2
0
216
Oct ’24
[macOS Sequoia] Sudden surge of "failed to retrieve app-scope key" logs and failed URL security scoping
Hello, with macOS Sequoia I've observed a sudden, substantial surge in reports about the Mac App Store version of my app Yoink no longer accepting files dropped to it, with the following message being logged in Console.app: 08:16:05.516307+0200 Yoink ---Yoink Error--- Could not create NSURL bookmark for /Users/<redacted>/Downloads/<redacted>/<redacted>.txt err: Error Domain=NSCocoaErrorDomain Code=256 "Failed to retrieve app-scope key" UserInfo={NSDebugDescription=Failed to retrieve app-scope key} The code line that causes this is the following: NSData *bmData = [fileURL bookmarkDataWithOptions:NSURLBookmarkCreationWithSecurityScope includingResourceValuesForKeys:nil relativeToURL:nil error:&err]; Sometimes a relaunch of Yoink, or a restart of the Mac resolves the issue, but only temporarily. Other times, even a restart doesn't work, but running tccutil reset All at.EternalStorms.Yoink in Terminal and then restarting the Mac works (again, temporarily). The trial version and Setapp versions of my app both work as expected, however. I myself cannot reproduce this issue, so any pointers would be greatly appreciated. (This might be considered a follow up to https://developer.apple.com/forums/thread/46583 , years back) Thanks a ton, – Matthias
4
2
504
Oct ’24
Porting VFS kext to FSKit
So if one were to start the attempt of porting an existing kext VFS filesystem, to use the new FSKit (Since presumably kexts could go away), how would that look now? Is it ready? Are there any samples out there that already works (Filesystem using FSKit) ? How is the documentation? ChatGPT did not seem to know much at all. What would be Apple's reception to that? How flexible is FSKit ? Is it locked to the idea of a mount is connected to a physical device (or partition)? Or is it more virtual, in that I will have a pool of disks, and present 1, or many, mount points?
2
0
385
Oct ’24
How to clear a "stuck" FileProvider
On some systems we are running into situations where we have an existing domain, but it remains in a stuck state where the domain exists but when attempting to interact with the domain to establish an XPC connection we get: =Error Domain=NSFileProviderInternalErrorDomain Code=0 "No valid file provider found from URL file:///Users/User/Library/CloudStorage/ProviderName-ProviderName." UserInfo={NSLocalizedDescription=No valid file provider found from URL file:///Users/User/Library/CloudStorage/Provider-Provider. Nothing that I've been able to do on an affected user account allows our app's domain to be added without facing errors. If we switch to a different user profile the domain is added and we can establish an XPC connection without any issues. So far I have tried: Removing the domain via NSFileProviderManager.removeAllDomains() Navigating to the domain in locations with the app uninstalled and deleting via the prompt within the Finder window Removing the plugin using: pluginkit -r /Applications/AppName.app/Contents/PlugIns/ProviderName.appex Removing the group container folder for the app from ~/Library/Group Containers and the app's data from ~/Library/Application Support/FileProvider/ I recognize that there is a profile (https://developer.apple.com/bug-reporting/profiles-and-logs/?platform=macos&name=Icloud) for additional logging, but having an end user install this is cumbersome. While I have encountered similar behavior myself I observed unix exception error 17 using console streaming logs though I can't be sure my issue is identical to the customers until we try and repeat the results on their systems with console streaming. macOS 15 has, deliberately, removed some of the options available in fileproviderctl to remove domains. If a fileProvider domain is in a bad state, how are we supposed to remove it? Relying only on NSFileProviderManager calls isn't helpful if these calls fail.
2
1
305
Oct ’24