tccd process stuck with file system filter

Hello,


We have file system filter. Recently on Catalina macOS we found that following tccd related files are stuck at open call:


  • /private/var/folders/tb/jjk74dn51p5_vgz844dpvfbw0000gn/T/com.apple.tccd/TemporaryItems/(A Document Being Saved By tccd)/keys
  • /Library/Application Support/com.apple.TCC/AdhocSignatureCache/keys


At the same time, tccd process in Activity monitor also stuck at _NSWriteDataToFileWithExtendedAttributes / _NSReadBytesFromFileWithExtendedAttributes. Please check below samples of tccd process : -


Thread_165622   DispatchQueue_24: com.apple.tcc.AdhocSignatureCache  (serial)
    + 2852 start_wqthread  (in libsystem_pthread.dylib) + 15  [0x7fff7224857b]
    +   2852 _pthread_wqthread  (in libsystem_pthread.dylib) + 290  [0x7fff7224871b]
    +     2852 _dispatch_workloop_worker_thread  (in libdispatch.dylib) + 598  [0x7fff71ffaa9e]
    +       2852 _dispatch_lane_invoke  (in libdispatch.dylib) + 363  [0x7fff71ff1452]
    +         2852 _dispatch_lane_serial_drain  (in libdispatch.dylib) + 597  [0x7fff71ff0ace]
    +           2852 _dispatch_client_callout  (in libdispatch.dylib) + 8  [0x7fff71feb50e]
    +             2852 _dispatch_call_block_and_release  (in libdispatch.dylib) + 12  [0x7fff71fea583]
    +               2852 __53-[TCCDAdhocSignatureCache getSignatureForStaticCode:]_block_invoke.114  (in tccd) + 211  [0x10c50462e]
    +                 2852 -[TCCDAdhocSignatureCache saveKeysToDirectory]  (in tccd) + 189  [0x10c502fc6]
    +                   2852 _NSWriteDataToFileWithExtendedAttributes  (in Foundation) + 1005  [0x7fff3d1d2d41]
    +                     2852 close  (in libsystem_kernel.dylib) + 10  [0x7fff721851aa]


OR


2641 Thread_5755   DispatchQueue_27: com.apple.tcc.AdhocSignatureCache  (serial)
    + 2641 start_wqthread  (in libsystem_pthread.dylib) + 15  [0x7fff688a957b]
    +   2641 _pthread_wqthread  (in libsystem_pthread.dylib) + 290  [0x7fff688a971b]
    +     2641 _dispatch_workloop_worker_thread  (in libdispatch.dylib) + 598  [0x7fff6865ba9e]
    +       2641 _dispatch_mach_invoke  (in libdispatch.dylib) + 481  [0x7fff6866263e]
    +         2641 _dispatch_lane_serial_drain  (in libdispatch.dylib) + 263  [0x7fff68651980]
    +           2641 _dispatch_mach_msg_invoke  (in libdispatch.dylib) + 435  [0x7fff68661aeb]
    +             2641 _dispatch_client_callout4  (in libdispatch.dylib) + 9  [0x7fff6864c5ae]
    +               2641 _xpc_connection_mach_event  (in libxpc.dylib) + 927  [0x7fff688e9158]
    +                 2641 _xpc_connection_call_event_handler  (in libxpc.dylib) + 56  [0x7fff688eaf68]
    +                   2641 __main_block_invoke.167  (in tccd) + 55  [0x10e76fcd5]
    +                     2641 handle  (in tccd) + 3308  [0x10e770a0d]
    +                       2641 do_TCCAccessRequest  (in tccd) + 8508  [0x10e77642c]
    +                         2641 -[TCCDAccessIdentity matchesCodeRequirementData:]  (in tccd) + 345  [0x10e787ecb]
    +                           2641 -[TCCDPlatformMacOS adhocSignStaticCode:]  (in tccd) + 169  [0x10e7a45b3]
    +                             2641 -[TCCDAdhocSignatureCache getSignatureForStaticCode:]  (in tccd) + 199  [0x10e79422d]
    +                               2641 _dispatch_lane_barrier_sync_invoke_and_complete  (in libdispatch.dylib) + 60  [0x7fff68658567]
    +                                 2641 _dispatch_client_callout  (in libdispatch.dylib) + 8  [0x7fff6864c50e]
    +                                   2641 __53-[TCCDAdhocSignatureCache getSignatureForStaticCode:]_block_invoke  (in tccd) + 312  [0x10e7943d6]
    +                                     2641 -[TCCDAdhocSignatureCache loadSignatureWithUUID:]  (in tccd) + 124  [0x10e7930db]
    +                                       2641 +[NSData(NSData) dataWithContentsOfURL:options:error:]  (in Foundation) + 61  [0x7fff3384473a]
    +                                         2641 -[NSData(NSData) initWithContentsOfFile:options:maxLength:error:]  (in Foundation) + 111  [0x7fff3383e075]
    +                                           2641 _NSReadBytesFromFileWithExtendedAttributes  (in Foundation) + 160  [0x7fff338265d6]
    +                                             2641 __open  (in libsystem_kernel.dylib) + 10  [0x7fff687e6192]
    2641 Thread_5996   DispatchQueue_11: com.apple.root.default-qos.overcommit  (concurrent)
    + 2641 start_wqthread  (in libsystem_pthread.dylib) + 15  [0x7fff688a957b]
    +   2641 _pthread_wqthread  (in libsystem_pthread.dylib) + 290  [0x7fff688a971b]
    +     2641 _dispatch_workloop_worker_thread  (in libdispatch.dylib) + 598  [0x7fff6865ba9e]
    +       2641 _dispatch_mach_invoke  (in libdispatch.dylib) + 481  [0x7fff6866263e]
    +         2641 _dispatch_lane_serial_drain  (in libdispatch.dylib) + 263  [0x7fff68651980]
    +           2641 _dispatch_mach_msg_invoke  (in libdispatch.dylib) + 435  [0x7fff68661aeb]
    +             2641 _dispatch_client_callout4  (in libdispatch.dylib) + 9  [0x7fff6864c5ae]
    +               2641 _xpc_connection_mach_event  (in libxpc.dylib) + 927  [0x7fff688e9158]
    +                 2641 _xpc_connection_call_event_handler  (in libxpc.dylib) + 56  [0x7fff688eaf68]
    +                   2641 __main_block_invoke.167  (in tccd) + 55  [0x10e76fcd5]
    +                     2641 handle  (in tccd) + 3308  [0x10e770a0d]
    +                       2641 do_TCCAccessRequest  (in tccd) + 8508  [0x10e77642c]
    +                         2641 -[TCCDAccessIdentity matchesCodeRequirementData:]  (in tccd) + 345  [0x10e787ecb]
    +                           2641 -[TCCDPlatformMacOS adhocSignStaticCode:]  (in tccd) + 169  [0x10e7a45b3]
    +                             2641 -[TCCDAdhocSignatureCache getSignatureForStaticCode:]  (in tccd) + 199  [0x10e79422d]
    +                               2641 _dispatch_sync_f_slow  (in libdispatch.dylib) + 171  [0x7fff6865840e]
    +                                 2641 __DISPATCH_WAIT_FOR_QUEUE__  (in libdispatch.dylib) + 270  [0x7fff686587ab]
    +                                   2641 _dispatch_event_loop_wait_for_ownership  (in libdispatch.dylib) + 498  [0x7fff686682fe]
    +                                     2641 _dispatch_kq_poll  (in libdispatch.dylib) + 247  [0x7fff68667844]
    +                                       2641 kevent_id  (in libsystem_kernel.dylib) + 10  [0x7fff687e6c22]


Which process have file handle : tccd or our process trying to open a file?

Please suggest document for tccd to analyze the issue.


Thanks ,

Simran

Replies

We have file system filter.

What do you mean by “file system filter”?

Share and Enjoy

Quinn “The Eskimo!”
Apple Developer Relations, Developer Technical Support, Core OS/Hardware

let myEmail = "eskimo" + "1" + "@apple.com"

Its Kauth kext which monitors file activities.


Thanks,

Simran

Deadlock is a common problem with kauth KEXTs. Looking at a backtrace of the deadlocked process is only half the story. What’s happening in your kauth KEXT at the time of the deadlock?

Share and Enjoy

Quinn “The Eskimo!”
Apple Developer Relations, Developer Technical Support, Core OS/Hardware

let myEmail = "eskimo" + "1" + "@apple.com"

Thanks for your response.


Please check following details :


1:

Kauth kext is basically providing events to user mode for scanning a file. It waits till scanning is done.

So kext is waiting for that thread (in user mode) which open a file , scan it and provide result to kauth. Here "open" call of tccd internal file (/Library/Application Support/com.apple.TCC/AdhocSignatureCache/keys) is not completed (it is stucked) and user mode thread is waiting for the same.


2:

With "Signed and Notarized" build above issue did not occur. It is generated only with the build which is partially signed and not-notarized. (Here, only kexts are signed)


Thanks,

Simran

This situation is covered by the Deadlock Avoidance section of Technote 2127 Kernel Authorization.

Share and Enjoy

Quinn “The Eskimo!”
Apple Developer Relations, Developer Technical Support, Core OS/Hardware

let myEmail = "eskimo" + "1" + "@apple.com"

Hello,


I have verified kauth for details. I have minimized location to generate events to folder "/Library/Application Support/com.apple.TCC/AdhocSignatureCache" only.

after machine start , only 2 threads are waiting : which are from the same path with process tccd (root and user mode). no other threads are blocking/ waiting.


Please check following points:

  • In my daemon open call is blocking . what are the causes for this? why it is not returning ?
  • How to identify why open is stuck ?


Thanks ,

Sheetal

In my daemon open call is blocking.

What does the in-kernel backtrace of the

open
call look like?

Share and Enjoy

Quinn “The Eskimo!”
Apple Developer Relations, Developer Technical Support, Core OS/Hardware

let myEmail = "eskimo" + "1" + "@apple.com"

Hello,


While checking kernel details , I have seen following logs of sandbox and tccd daemons, please check below:


  1. Vnode action for "/Library/Application Support/com.apple.TCC/AdhocSignatureCache/F86FE4D8-7544-446E-B7B6-8C2440A00598" in kauth
  2. TCCD:

    Error reading signature from URL: url=/Library/Application Support/com.apple.TCC/AdhocSignatureCache/F86FE4D8-7544-446E-B7B6-8C2440A00598 error=Error Domain=NSCocoaErrorDomain Code=257 "The file “F86FE4D8-7544-446E-B7B6-8C2440A00598” couldn’t be opened because you don’t have permission to view it." UserInfo={NSFilePath=/Library/Application Support/com.apple.TCC/AdhocSignatureCache/F86FE4D8-7544-446E-B7B6-8C2440A00598, NSUnderlyingError=0x7fb8b0f22dd0 {Error Domain=NSPOSIXErrorDomain Code=13 "Permission denied"}}

  3. Sandbox: <Our user daemon> (230) System Policy: deny(1) file-read-data /Library/Application Support/com.apple.TCC/AdhocSignatureCache/F86FE4D8-7544-446E-B7B6-8C2440A00598


Why sandbox denying to read tccd files ? and open call gets blocked in user mode?


Thanks,

Simran