[quote='813819022, DTS Engineer, /thread/767612?answerId=813819022#813819022']
I’m not entirely sure how this flag is getting set on your executable. I tried various mechanism to get it set in a test app, and failed.
[/quote]
Me either, I'm trying to re-create the problem in isolated form, but the quarantine flag is different.
[quote='813819022, DTS Engineer, /thread/767612?answerId=813819022#813819022']
My understanding is that:
This flag is set on the executable you’re trying to run.
That executable exists on a disk image.
If so, what does the com.apple.quarantine attribute look like for:
The root of the disk image?
The .dmg file itself?
[/quote]
Just after I download the image it looks like:
$ xattr -l /Users/luweber/Downloads/vlc-3.0.21-intel64.dmg
com.apple.macl:
com.apple.metadata:kMDItemWhereFroms: bplist00?_?https://[redacted]/download?id=abbd13b7-f1b7-4fd2-9e51-72e3d790ad83
?
com.apple.quarantine: 0087;67345866;Download Helper;
Then I mount it, but image root, nor the app bundle has a quarantine flag:
$ xattr -l /Volumes/VLC\ media\ player
com.apple.FinderInfo:
$ xattr -l /Volumes/VLC\ media\ player/VLC.app
$ xattr -l /Volumes/VLC\ media\ player/VLC.app/Contents/MacOS/VLC
Finally when I copy (drag&drop in Finder) application to /Applications:
$ xattr -l /Applications/VLC.app
com.apple.quarantine: 0187;67345866;Download\x20Helper;
Post
Replies
Boosts
Views
Activity
Thank you Quinn,
I tried the com.apple.security.files.user-selected.executable and it didn't help.
[quote='812817022, DTS Engineer, /thread/767612?answerId=812817022#812817022']
And if that doesn’t work for you then the next best option is to un-sandbox the code that creates this disk image. My standard approach for that is to embed an XPC service in the app, because the embedded XPC service can have a custom sandbox, including no sandbox.
[/quote]
None of these is actually an option for us. We would be eaten alive by security folks in the company if we would disable sandbox.
I was looking into it more and dumped the kCFURLQuarantinePropertiesKey:
{
LSQuarantineAgentName = "Download Helper";
LSQuarantineIsOwnedByCurrentUser = 1;
LSQuarantineTimeStamp = "2024-11-11 16:36:18 +0000";
LSQuarantineType = LSQuarantineTypeSandboxed;
}
That LSQuarantineTypeSandboxed caught my attention and found an older post 650470 stating:
That type indicates that the process was touched by a sandboxed process that didn't have permissions to write that file.
So maybe it's a same problem like with location services, which don't (or don't used to) consult responsible process for entitlements or something. What entitlements this could be related to, the hardened runtime stuff maybe?
[quote='812627022, DTS Engineer, /thread/767612?answerId=812627022#812627022']
Is your product distributed on the Mac App Store? Or have you sandboxed it because that’s the right thing to do?
[/quote]
It's not distributed on Mac App Store (but it may eventually get there). But we do the sandboxing because it's right thing to do given the app is basically a web browser in disguise.
It says that ticket is missing:
$ syspolicy_check distribution /Applications/VLC.app
App has failed one or more pre-distribution checks.
---------------------------------------------------------------
Notary Ticket Missing
File: VLC.app
Severity: Fatal
Full Error: A Notarization ticket is not stapled to this application.
Type: Distribution Error
When I run the stapler on it the error goes away but app still doesn't start:
$ stapler staple /Applications/VLC.app
Processing: /Applications/VLC.app
Processing: /Applications/VLC.app
The staple and validate action worked!
$ syspolicy_check distribution /Applications/VLC.app
App passed all pre-distribution checks and is ready for distribution.
I ran also the spctl before the stapler:
$ spctl --assess -v /Applications/VLC.app
/Applications/VLC.app: File created by an AppSandbox, exec/open not allowed
[quote='812085022, DTS Engineer, /thread/767612?answerId=812085022#812085022']
If you use the Finder to copy the app off the disk image to somewhere else — say the Applications folder or, if that already have an app of the same name, your home directory — will the app launch from there.
[/quote]
It doesn't help. I copied it (drag & drop in Finder) from DMG to ~/Desktop and to /Applications and it shows the same dialog.
The quarantine attribute is preserved no matter what I do with the bundle. Opening the application from context menu (secondary click on the bundle in the Finder) also yields the same result.
Hello Kevin, I found the issue but I'll reply anyway as it may be useful to other people.
Are you seeing this issue on later system versions?
No, it looks like it's only Monterey, specifically the last two minor releases which I tested on.
Is your XPC service directly presenting it's own window(s) or is this coming through your main app?
Yes, I'm starting a NSApplication on main thread and the service listener on a second thread. That's only difference to regular XPCService template running just xpc_main().
If you're not already doing so, you may want to try having your app using "beginActivityWithOptions" to ensure that it's remaining active.
I tried this both in application and xpc service and it didn't had any effect. I even tried task_policy, setting main thread to user interactive.
Particularly on a old system like this, you may find that periodically "resetting" the message (say, every minute) works better than simply holding the same message "forever".
I tried exactly this and it works. I was sending the reply block only at the start of the application to kickstart the service and boost it. Sending the block again before I want a playback seems to fix the priority problem on Monterey and doesn't have any side effects on newer systems.
I still have a question though. Is there any recommendation how to investigate xpc service priorities (or QoS in general)? I tried launchctl print pid/13422/com.my.service and it gives handful of information but nothing I could figure out the boosting was not working correctly (spawn type seems to reflect only the startup priority). Also in Instruments I was using time profiler, system trace etc. and it shown the correct application lifecycle ("Foreground") but sound was still broken.
So I found following being all over the logs when issue occurs:
HALC_ProxyIOContext::IOWorkLoop: skipping cycle due to overload
The machine is i9 with 64GB of RAM basically idling.
Edit:
Found another message which seems to lead to source of the problem:
HALS_OverloadMessage: Overload due to client running as an adaptive unboosted
So indeed it seems to be some priority problem?
Thank you Kevin, I'll file the incident in a few hours. I'll try to get some fresh sysdiagnose logs as well. We didn't collect those because most users are in managed environments so we need to arrange everything with their admins.
Is it always the same "5f45 4545" value being replaced?
No, it's same value per instance, but different for each instance.
I have no idea why that would have happened, but it would be interesting to confirm whether or not that pattern holds more broadly.
Definitely not all instances are replaced. I can still find same sequence in the file. I'll send you the originals and corrupted files so you can see for yourself. At the beginning it looked like all instances of the sequence were replaced but once we got more of these corrupted files it's clear that's not the case - we were just lucky with the first ones.
Just to clarify, was that the specific installer pkg they had, not just the same version? I want to make sure you've ruled out the possibility that the bad file wasn't already inside the installer.
I think I pretty much ruled this out. I was originally suspecting that admins were installing some modified packages so I asked if I can get those, but they were either links to our CDN or just unmodified packages.
Thank you for Kevin, I'll try to answer some questions
What were the actual modifications that occurred and what files did they occur on? You said that specific bytes were modified but what was actually changed?
The modification is always a replacement of meaningful ASCII string in Mach-O (main executable or dylibs). It's often a single sequence, but sometimes it's two where the second one is subsequence of the first. It's interesting that not all occurrences of the string are replaced in the whole file. I'm trying to make some sense out of it. It looks like the string is always replaced in both architectures in Mach-O, so it makes me think that whatever is doing that is aware of the executable format.
Related to that point, how do you "know what you know" and are you sure you understand what actually happened?
I get the zipped application bundles from customers and then compare the corrupted files to the bundles I extracted from pkg they use for install. Another clue I get is the install.log. In our postinstall we run gktool, so I see the output and it already complains there that bundle is altered.
The component package which is getting corrupted has PackageInfo setup like this:
Which forces installer to basically just replace the destination bundle instead of updating it so if the target bundle would be corrupted before installation it would not have any effect (unless it has a newer version) as installer would remove it and put in new bundle.
That all leads me to fact that new bundle has to be corrupted either during or right after it was installed but before postinstall script runs.
For you to have access to the port the kernel must have inserted an entry for that port into your task’s Mach port namespace. Which is exactly what the bootstrap service does when you successfully look up a name.
Ah now I get it! I got stuck with deceiving idea that bootstrap service is kind of "DNS" for ports, but was not thinking for a minute how the port name actually gets into the process's port namespace. Now it makes much more sense.
Thank you for connecting the dots for my dumb brain! :-)
@DTS Engineer I have a follow up question though. I understand that the bootstrap namespace is private to to application so I can't connect its XPC service by name if I don't have access to that namespace.
But what If I know the mach port? Let's say I'll be lucky and guess the port :-) ...or use launchctl to tell me:
$ launchctl print pid/12345/com.my.test.service
...
endpoints = {
"com.my.test.service" = {
port = 0xeaf13
active = 1
managed = 1
reset = 0
hide = 0
watching = 0
}
}
...
In that case I would I be able to use msg_send() to send messages directly and eventually replicating what XPC does or is there any other level of protection (e.g. what comes to mind is that launchd can check the audit session or something)?
Thank you Quinn for looking into this. I think I found the problem:
I'm having a simple scenario where main bundle contains and embedded XPCService bundle. In services NSXPCListenerDelegate I'm using the - setCodeSigningRequirement: on the incoming connection with valid code signing requirements.
The connection was always refused and I've seen the security_exception in the log. So I compared the entitlements and signing of the applications bundles and only difference between working and non-working version was obvious presence of get-task-allow on developer signed builds and different signing certificate chain. So I was wrongly assuming that difference is due to signing.
But it turned out that if I moved the problematic bundle to /Applications it immediately started to work. So it's not a signing difference but it's on-disk location difference.
I think it can come from the application.sb:
(allow file-read*
process-exec
...
(subpath "/Developer")
(subpath "/Applications"))
So when the XPCService is located under /Applications it can read the code signature from the parent bundle's executable.
I'm not sure that profile is used for XPCServices bundled in the application but it would explain the behaviour.
By “XPCService” do you mean an .xpc bundle within your app?
Yes, I do.
If so, applying a code signing requirement is kinda pointless. XPC services bundled within your app are only visible to your app.
I'm not really sure what visibility means in this case, so asked here https://developer.apple.com/forums/thread/757627. Our security team requires us to make sure no-one else can connect to in case the can guess the service's port (if that's possible).
[1] Here's the full error stack:
connection [0x13a904080] activating connection: mach=false listener=false peer=true name=com.example.app.service.peer[46207].0x13a904080 com.apple.xpc
security_exception MacOS error: -67065 com.apple.securityd
security_exception 0 Security 0x00000001943d6108 Security::CommonError::LogBacktrace() + 124 com.apple.securityd
security_exception 1 Security 0x00000001943d66bc Security::MacOSError::MacOSError(int) + 340 com.apple.securityd
security_exception 2 Security 0x00000001943d672c Security::MacOSError::throwMe(int) + 40 com.apple.securityd
security_exception 3 Security 0x00000001942d28d4 Security::CodeSigning::KernelCode::locateGuest(__CFDictionary const*) + 1140 com.apple.securityd
security_exception 4 Security 0x00000001942a7560 Security::CodeSigning::SecCode::autoLocateGuest(__CFDictionary const*, unsigned int) + 132 com.apple.securityd
security_exception 5 Security 0x00000001942b05c8 SecCodeCopyGuestWithAttributes + 156 com.apple.securityd
security_exception 6 support 0x0000000104d166e8 xpc_support_check_token + 280 com.apple.securityd
security_exception 7 libxpc.dylib 0x00000001910632e0 _xpc_connection_check_peer_requirement + 428 com.apple.securityd
security_exception 8 libxpc.dylib 0x000000019104b58c _xpc_connection_mach_event + 1056 com.apple.securityd
security_exception 9 libdispatch.dylib 0x0000000105092c64 _dispatch_client_callout4 + 20 com.apple.securityd
security_exception 10 libdispatch.dylib 0x00000001050b5994 _dispatch_mach_msg_invoke + 612 com.apple.securityd
security_exception 11 libdispatch.dylib 0x000000010509bb98 _dispatch_lane_serial_drain + 368 com.apple.securityd
security_exception 12 libdispatch.dylib 0x00000001050b70c4 _dispatch_mach_invoke + 496 com.apple.securityd
security_exception 13 libdispatch.dylib 0x000000010509bb98 _dispatch_lane_serial_drain + 368 com.apple.securityd
security_exception 14 libdispatch.dylib 0x000000010509ceb0 _dispatch_lane_invoke + 468 com.apple.securityd
security_exception 15 libdispatch.dylib 0x00000001050ac958 _dispatch_root_queue_drain_deferred_wlh + 652 com.apple.securityd
security_exception 16 libdispatch.dylib 0x00000001050abc30 _dispatch_workloop_worker_thread + 444 com.apple.securityd
security_exception 17 libsystem_pthread.dylib 0x000000010513bd40 _pthread_wqthread + 288 com.apple.securityd
security_exception 18 libsystem_pthread.dylib 0x0000000105143a94 start_wqthread + 8 com.apple.securityd
esecurity_exception UNIX error exception: 1 com.apple.securityd
security_exception 0 Security 0x00000001943d6108 Security::CommonError::LogBacktrace() + 124 com.apple.securityd
security_exception 1 Security 0x00000001943d6464 Security::UnixError::UnixError(int, bool) + 352 com.apple.securityd
security_exception 2 Security 0x00000001943d64d4 Security::UnixError::throwMe(int) + 44 com.apple.securityd
security_exception 3 Security 0x00000001943d6aec Security::UnixPlusPlus::FileDesc::close() + 0 com.apple.securityd
security_exception 4 Security 0x00000001942fcbd0 Security::CodeSigning::SingleDiskRep::fd() + 64 com.apple.securityd
security_exception 5 Security 0x00000001942dfdfc Security::CodeSigning::MachORep::MachORep(char const*, Security::CodeSigning::DiskRep::Context const*) + 528 com.apple.securityd
security_exception 6 Security 0x00000001942d223c Security::CodeSigning::KernelCode::identifyGuest(Security::CodeSigning::SecCode*, __CFData const**) + 684 com.apple.securityd
security_exception 7 Security 0x00000001942a6bc8 Security::CodeSigning::SecCode::identify() + 100 com.apple.securityd
security_exception 8 Security 0x00000001942a759c Security::CodeSigning::SecCode::autoLocateGuest(__CFDictionary const*, unsigned int) + 192 com.apple.securityd
security_exception 9 Security 0x00000001942b05c8 SecCodeCopyGuestWithAttributes + 156 com.apple.securityd
security_exception 10 support 0x0000000104d166e8 xpc_support_check_token + 280 com.apple.securityd
security_exception 11 libxpc.dylib 0x00000001910632e0 _xpc_connection_check_peer_requirement + 428 com.apple.securityd
security_exception 12 libxpc.dylib 0x000000019104b58c _xpc_connection_mach_event + 1056 com.apple.securityd
security_exception 13 libdispatch.dylib 0x0000000105092c64 _dispatch_client_callout4 + 20 com.apple.securityd
security_exception 14 libdispatch.dylib 0x00000001050b5994 _dispatch_mach_msg_invoke + 612 com.apple.securityd
security_exception 15 libdispatch.dylib 0x000000010509bb98 _dispatch_lane_serial_drain + 368 com.apple.securityd
security_exception 16 libdispatch.dylib 0x00000001050b70c4 _dispatch_mach_invoke + 496 com.apple.securityd
security_exception 17 libdispatch.dylib 0x000000010509bb98 _dispatch_lane_serial_drain + 368 com.apple.securityd
security_exception 18 libdispatch.dylib 0x000000010509ceb0 _dispatch_lane_invoke + 468 com.apple.securityd
security_exception 19 libdispatch.dylib 0x00000001050ac958 _dispatch_root_queue_drain_deferred_wlh + 652 com.apple.securityd
security_exception 20 libdispatch.dylib 0x00000001050abc30 _dispatch_workloop_worker_thread + 444 com.apple.securityd
security_exception 21 libsystem_pthread.dylib 0x000000010513bd40 _pthread_wqthread + 288 com.apple.securityd
security_exception 22 libsystem_pthread.dylib 0x0000000105143a94 start_wqthread + 8 com.apple.securityd
<Missing Description> xpc_support_check_token: identifier com.example.app status: 100001
connection [0x13a904080] invalidated because the current process cancelled the connection by calling xpc_connection_cancel() com.apple.xpc
<Missing Description> invalidate<com.example.app.service([app<application.com.example.app.271193970.274328866.AF06BFB0-EAE9-4539-AA9C-368A2584E9AE(501)>:46207])(501)>{vt: (null)}:
Also have to add that from experimentation I learned that basically once sandbox_init[1][2] is called in the parent process, a child can't init its own sandbox, any attempt to re-init will fail.
[1] Doesn't matter if triggered by entitlements via libsecinit or manually calling sandbox_init in the main()
[2] The role of com.apple.security.inherit was for long time lost on me, it seems its sole role is to prevent libsecinit to init a sandbox when com.apple.security.app-sandbox is present :)