Apple DTS responded and asked me to submit the issue using FeedBack Assistant because they believe this issue would be best handled by the relevant engineering team directly, as it may require changes to the OS, tools, or the frameworks being used.
More as it occurs...
Post
Replies
Boosts
Views
Activity
Still have not heard back from Apple, but have discovered a few interesting things. It looks like:
SNClassifySoundRequest(classifierIdentifier: version1)
was never meant to run on the GPU in the background and it was a fluke it worked on iOS 17 and below. We are using the built-in model provided by the Sound Analysis framework - that ones seems to be optimized to run on the GPU only and thus causes the explosion we are seeing. If you look at:
SNClassifySoundRequest(mlModel: mlModel)
where you use your own model you can specify that it run on the CPU and this might work in the background. (We don't have our own model to test with since we are doing a proof of concept first using Apple's sounds). Leaving this here incase anyone wants to try it.
// create the configuration for the model
let configuration = MLModelConfiguration()
configuration.computeUnits = .cpuOnly // force computations to use only the CPU
do {
// load your Core ML model with the configuration
let mlModel = try MLModel(contentsOf: modelURL, configuration: configuration)
// wrap the model into a Sound Analysis request
let request = try SNClassifySoundRequest(mlModel: mlModel)
// create your SNAudioStreamAnalyzer
let analyzer = SNAudioStreamAnalyzer(format: audioFormat)
// add the request to the analyzer
try analyzer.add(request, withObserver: self)
} catch {
print("Error setting up Sound Analysis: \(error)")
}
I just sent a highly detailed ticket to DTS using the WWDC21 session - 10036 sample code for audio analysis with background modes set. It does the same error we are all seeing with our own apps when the iPhone locks or the app goes to the background:
Insufficient Permission (to submit GPU work from background) (00000006:kIOGPUCommandBufferCallbackErrorBackgroundExecutionNotPermitted)
I'll reply back here as soon as I hear something.
Yes. You can monitor the microphone and do sound classification when iOS or iPadOS is locked / asleep.
I spent three days tracking this bizarre one down, besides the Error message - NSTextView's started not drawing any text. No code changes had happened other than moving to Xcode 15.4 and Sonoma. The problem didn't happen with Xcode 14 and Sonoma. XCode 15.4 and Ventura work. Hows that for a puzzle? It turns out the problem was a change that Apple made to NSView in MacOS 14. The bug was caused when Apple decided to change the default return value of NSView.clipsToBounds. By setting that to true the code works again and the Error message is gone. Here's an explanation -
https://developer.apple.com/documentation/macos-release-notes/appkit-release-notes-for-macos-14#NSView
Here is a little more on the problem - https://developer.apple.com/documentation/macos-release-notes/appkit-release-notes-for-macos-14#NSView
Have you checked the expiration time in the token you set up?
Don't try and draw the string that just won't work. They won't always or ever match up. I learned that the hard way. This was sheepishly confirmed by Apple DTS. One way to solve it is to use a NSTextView everywhere you want the editable string and set isEditable = false and isSelectable = false and let that do your drawing. When you want to edit it, either set it as editable or create a new NSTextView that has the same size and feed it your NSAttributedString.
As of Jan 2024 this one bit me. Same ambiguous error. Luckily I found this comment before having to go through the steps that Eskimo kindly suggested. Headed over to developer agreements and there was yet another agreement that needed to be signed. Something about promising that my app would be carbon neutral or some other nonsense. As u/swampert stated, it worked just fine in about 6 minutes.
Found it. TL;DR - Assets.xcassets and info.plist needed to be rebuilt for the current shipping version of Xcode (14.x). I’ll type up this debugging expedition for future travelers and aspiring Xcode engineers. As stated above the app running under the beta Sonoma release (9/12/23 and earlier) had a problem where none of the default controls in dialog boxes would be their default color blue. Even on Apple’s dialogs like NSAlert(). The buttons worked, they just did not have the correct color. NSTextFields could not be edited. The app works perfectly under Mojave through Ventura. No warnings in the build or in the console at run time. After exhausting every reasonable attempt at debugging this, I rebuilt the app from scratch starting with a new project in Xcode 14.1 and copying the files over from the existing project. After the app was built, same problems. To find the culprit I started taking source code out all the way down to just AppDelegate.swift was left and all it was doing was opening an NSAlert() in the applicationDidFinishLaunching() function. Problem still existed. Then I deleted the info.plist file and the problem went away. This opened a whole new can of worms. It appears that in Xcode 13 and later, the way Xcode interacts with info.plist changed.
This is from the release notes for Xcode 13 -
Projects created from several templates no longer require configuration files such as entitlements and Info.plist files. Configure common fields in the target’s Info tab, and build settings in the project editor. These files are added to the project when additional fields are used.
And from “Developing a Document-Based App” -
Apps must use Xcode’s Document Types editor to define the types of documents they can open. When Xcode builds your app, it places the document type information in your app’s information property list (Info.plist) file.
Using the target’s Info tab I was able to add the Document Types and Exported Type Identifiers. This editor sure seems to be a work in progress. I think the engineers working on this got pulled off mid way to go make everything at Apple be carbon neutral. Here’s a tip - in the Exported Type Identifiers editor there is a field that says “Click here to add additional exported type identifier properties”. Clicking on that or anywhere near it does nothing at all. There is a very narrow invisible box on the far left side, to the left and below the “Key” label you have to blindly click to add type identifier properties.
Added all of those and sure enough Xcode generated a info.plist file with everything in it. And boom the problem with the default buttons came back. Adding Document Types and Exported Type Identifiers seems to have caused the problem. Taking them out fixed the problem with the app. (There is no delete function in the target Info tab editor, so you have to delete them manually from info.plist). To make a lot of these edits take effect you have to stop and restart Xcode. I’m sure when everyone at Apple gets done making new emojis they will get back to working on the target editors.
Next I started adding bits of Document Types one bit at a time until the problem came back. Bingo. Adding icons to the Document Types caused the problem. At this point you must be as puzzled as me as to why adding icons to the Document Type causes default buttons to not show blue, their default color. Try and draw a straight line between that cause and effect? I looked at the JSON inside Assets.xcassets and didn’t see anything malformed - or any garbage characters.
I took an empty Assets.xcassets file from a newly created empty project and copied that to the project and then added the icons back one at a time. This compiled and everything worked as expected. Default controls are blue again in Sonoma.
The best reason for this nightmare scenario I can come up with is the app was originally written in Xcode 11.x. It’s plausible that so much water has gone under the bridge with info.plist and Assets.xcassets with Xcode 13 and 14 that Xcode 14.1 built something that Sonoma could not dissect and caused this weird problem.
Hey Apple. Maybe a little less work on carbon neutrality and a little more work on Xcode??
A little more info... The first NSAlert() is the app that does not work correctly with Sonoma. This NSAlert() has focus. Notice how the blue folder behind the button does not bleed through the button. The button is active, just not blue.
The second NSAlert() is the simple test app that only has the NSAlert(). It does not have focus, and the blue folder behind it does bleed through. (If this one has focus the button is blue).
Both apps were compiled with Xcode 15 beta (the problem happens with Xcode 14.1 as well) .
Bingo - found the problem. I broke out the archiving code that was not working and put it in a new simple app (see below). I created a simple class with a Swift Array in it, wrote the archive out with the non NSSecureCoding code. Then changed the code so the class uses NSSecureCoding and changed the de-archiving code to read that file back in using:
try newTest = NSKeyedUnarchiver.unarchivedObject(ofClasses: [Test.self], from: fileData as Data) as? Test
It failed silently like the main app failed. Then as suggested by Polyphonic I added NSArray.self to the ofClasses: parameter - ran the app again and a litany of good info was spewed to the console when the test app tried to decode the Array - the real app did not send anything to the console - it just failed silently returning a nil for the Array. Here's the console message:
NSCoderTest[11222:509325] [general] *** -[NSKeyedUnarchiver validateAllowedClass:forKey:] allowed unarchiving safe plist type ''NSNumber' (0x2140200a8) [/System/Library/Frameworks/Foundation.framework]' for key 'NS.objects', even though it was not explicitly included in the client allowed classes set: '{(
"'NSCoderTest.Test' (0x100eb0700) , "'NSArray' (0x213ff2440) [/System/Library/Frameworks/CoreFoundation.framework]"
)}'. This will be disallowed in the future.
Then I added NSNumber.self to the ofClasses: parameter and all works as it should. The main app where the code was failing was created three years ago with whatever was the current Xcode version at the time. I suspect there is something mangled in the project file that keeps it from emitting some error / warning messages to the console. Great.
The laconic documentation for unarchivedObject(ofClasses:from:) says "A set of classes, at least one of which the root object should conform to". I think that means to add every class that your class needs to de-archive. Polyphonic has an excellent explanation about why this is necessary in his reply above.
Polyphonic - thanks so much for the nudge in the right direction.
Hey Apple, maybe a little more work on the documentation and a little less work on emojis maybe?
Simple test code -
class Test: NSObject, NSSecureCoding {
static var supportsSecureCoding: Bool = true
var i: Int64 = 55
var theArray: Array<Bool>? = [true, true, true, true]
func encode(with coder: NSCoder) {
coder.encode(i, forKey: "var_i")
coder.encode(theArray, forKey: "var_a")
}
override init() { }
required init?(coder: NSCoder) {
if (coder.containsValue(forKey: "var_i")) {
i = coder.decodeInt64(forKey: "var_i")
}
if (coder.containsValue(forKey: "var_a")) {
theArray = coder.decodeObject(forKey: "var_a") as? Array<Bool>
}
}
}
func applicationDidFinishLaunching(_ aNotification: Notification) {
var newTest: Test?
let fileURL = URL(fileURLWithPath: "/Users/username/Downloads/testData", isDirectory: false)
//let t1 = Test()
//t1.i = 100
//t1.a[1] = false
//t1.a[2] = false
do {
// let theData = try NSKeyedArchiver.archivedData(withRootObject: t1, requiringSecureCoding: false)
// try theData.write(to: fileURL, options: NSData.WritingOptions.atomic)
let fileData = try NSData(contentsOf: fileURL, options: [])
// try newTest = NSKeyedUnarchiver.unarchiveTopLevelObjectWithData(fileData as Data) as? Test
try newTest = NSKeyedUnarchiver.unarchivedObject( ofClasses: [Test.self, NSArray.self],
from: fileData as Data) as? Test
}
catch {
print("well that didn't work.")
}
}
Thanks for the quick answers and explanation. In trying to suss this out I had added NSArray.self to the ofClasses: parameter and no joy.
var newParts: Parts?
newParts = try NSKeyedUnarchiver.unarchivedObject(ofClasses: [Parts.self, NSString.self, NSArray.self], from: newData) as? Parts
The Swift Array in question is a member of the class Parts() and were encoded without NSSecureCoding - would that be a factor? (these archives were created from an earlier version of the app that I'm trying to migrate to Apple's deprecation de jour world)
var pageUsesClearBackground: Array<Bool> = [true, true, true, true]
coder.encode(pageUsesClearBackground, forKey: "CB_KEY")
The plot thickens. Further down the line as things are being decoded I'm finding things that were encoded() are not decoding() - they return nil as well.
macOS 14.0 is one sharp corner I must say.
This just started happening out of the blue on a M1 Studio Mac with Monterrey 12.6.7 and XCode 14.1. The project is at least 3 years old and this warning has never shown up. One thing that is interesting is I move the project between the Studio Mac and an intel iMac every now and then. While trying to figure out what's gone wrong now - I happened to look up at the menu bar in XCode and it was saying to run the app under Rosetta. Changing that from My Mac (Rosetta) to My Mac solved the problem. No idea how that got changed.