Hi,I have come to notice that when registering several clients on several authorization events (one for ES_EVENT_TYPE_AUTH_EXEC and another for ES_EVENT_TYPE_AUTH_CREATE for example), even when I immediately call `es_respond_auth_result` - I am getting a significant performance penalty, that wasn't there when doing the same operations in a Kernel Extension.For example, compilation that used to take me less the 8 minutes, now takes me 10-11 minutes.When running instruments I can see that most of the time is actually spent inside the authorization function (es_respond_auth_result) - am I doing something wrong here?I saw the caching flag makes a small improvement (reduces compilation time to 9-10 minutes), but It's a bit of a problem to use it without knowing what's the actual effect of the caching - what parameters are actually cached? what events can I assume will not be shown again?This performance impact makes this framework unusable - what customer would be willing to take that severe impact with a product using this framework?Also opened a bug (FB7503995) about this issue.Thanks.
obenshalom
Users receive reputation points by contributing quality content to the forums.
Posts include post and replies published on the forums.
Post authors can mark replies as Accepted to indicate successful solutions.
Apple Developer Forums admins can mark replies as Apple Recommended to indicate an approved solution
Last seen
User for
Post
Replies
Boosts
Views
Activity
With EndpointSecurity framework we are able to receive a process audit token for every event that was made by it.
However, we sometimes find ourselves in need to enumerate running processes, or validate an existing process matching the one on our memory. This can also happen after our daemon is installed, and we want to query current system information.
In this case we need to be able to get a process audit token on demand, and not by an even that it had made. Is it possible to do such query? is there a plan to add something like that?
On rare occasions when changing network on a machine with network extension installed (e.g. when disconnecting laptop from Docking station and connecting to Wi-Fi), Network freezes, and a few seconds later machine crashes.
On console we can see that there was a crash in UserEventAgent:
Termination Reason:		WATCHDOG, [0x1] monitoring timed out for service
Termination Details:	 WATCHDOG, remoted connection down, inducing a crash
And on system.log we can see:
UserEventAgent[9151]: assertion failed: 19G2021: com.apple.bonjour + 6922 [54D5EA10-98BA-3AD4-94C3-182BB3700419]: 0xfffffffffffeffe0
(Which if I understand correctly, is kDNSServiceErr_Timeout).
I would have thought it indicated some issue with our Network Extension, but when I got to sample the extension when network was frozen, I could see the following stack:
Call graph:
		2833 Thread_159027: Main Thread	 DispatchQueue_<multiple>
		+ 2833 _dispatch_sig_thread	(in libdispatch.dylib) + 53	[0x7fff71ec6476]
		+	 2833 _dispatch_sigsuspend	(in libdispatch.dylib) + 36	[0x7fff71ec649a]
		+		 2833 __sigsuspend_nocancel	(in libsystem_kernel.dylib) + 10	[0x7fff72056502]
		2833 Thread_170723
		+ 2833 thread_start	(in libsystem_pthread.dylib) + 15	[0x7fff7210fb8b]
		+	 2833 _pthread_start	(in libsystem_pthread.dylib) + 148	[0x7fff72114109]
		+		 2833 Poco::ThreadImpl::runnableEntry(void*)	(in libPocoFoundation.60.dylib) + 94	[0x10327b2fe]
		+			 2833 Poco::ActiveDispatcher::run()	(in libPocoFoundation.60.dylib) + 72	[0x10327f598]
		+				 2833 Poco::NotificationQueue::waitDequeueNotification()	(in libPocoFoundation.60.dylib) + 392	[0x103243e48]
		+					 2833 Poco::EventImpl::waitImpl()	(in libPocoFoundation.60.dylib) + 70	[0x10320c8a6]
		+						 2833 _pthread_cond_wait	(in libsystem_pthread.dylib) + 698	[0x7fff72114425]
		+							 2833 __psynch_cvwait	(in libsystem_kernel.dylib) + 10	[0x7fff72053882]
		2833 Thread_180423
		+ 2833 thread_start	(in libsystem_pthread.dylib) + 15	[0x7fff7210fb8b]
		+	 2833 _pthread_start	(in libsystem_pthread.dylib) + 148	[0x7fff72114109]
		+		 2833 Poco::ThreadImpl::runnableEntry(void*)	(in libPocoFoundation.60.dylib) + 94	[0x10327b2fe]
		+			 2833 Poco::ActiveDispatcher::run()	(in libPocoFoundation.60.dylib) + 72	[0x10327f598]
		+				 2833 Poco::NotificationQueue::waitDequeueNotification()	(in libPocoFoundation.60.dylib) + 392	[0x103243e48]
		+					 2833 Poco::EventImpl::waitImpl()	(in libPocoFoundation.60.dylib) + 70	[0x10320c8a6]
		+						 2833 _pthread_cond_wait	(in libsystem_pthread.dylib) + 698	[0x7fff72114425]
		+							 2833 __psynch_cvwait	(in libsystem_kernel.dylib) + 10	[0x7fff72053882]
		2833 Thread_209573	 DispatchQueue_59: NEFilterExtensionProviderContext queue	(serial)
		+ 2833 start_wqthread	(in libsystem_pthread.dylib) + 15	[0x7fff7210fb77]
		+	 2833 _pthread_wqthread	(in libsystem_pthread.dylib) + 290	[0x7fff72110a3d]
		+		 2833 _dispatch_workloop_worker_thread	(in libdispatch.dylib) + 596	[0x7fff71ec5c09]
		+			 2833 _dispatch_lane_invoke	(in libdispatch.dylib) + 363	[0x7fff71ebc5d6]
		+				 2833 _dispatch_lane_serial_drain	(in libdispatch.dylib) + 263	[0x7fff71ebbaf6]
		+					 2833 _dispatch_source_invoke	(in libdispatch.dylib) + 2084	[0x7fff71ec84be]
		+						 2833 _dispatch_continuation_pop	(in libdispatch.dylib) + 414	[0x7fff71eb8818]
		+							 2833 _dispatch_client_callout	(in libdispatch.dylib) + 8	[0x7fff71eb6658]
		+								 2833 -[NEFilterDataExtensionProviderContext handleSocketSourceEventWithSocket:]	(in NetworkExtension) + 1115	[0x7fff3e92e818]
		+									 2833 -[NEFilterDataExtensionProviderContext handleDataCompleteForFlow:direction:reply:controlSocket:completionHandler:]	(in NetworkExtension) + 381	[0x7fff3e92bf09]
		+										 2833 -[NEFilterDataSavedMessageHandler enqueueWithFlow:context:]	(in NetworkExtension) + 183	[0x7fff3e93085c]
		+											 2833 -[NEFilterDataSavedMessageHandler executeWithFlow:context:]	(in NetworkExtension) + 294	[0x7fff3e9309a4]
		+												 2833 -[NEFilterDataSavedMessageHandler executeVerdictHandlerWithFlow:verdict:context:]	(in NetworkExtension) + 305	[0x7fff3e930b83]
		+													 2833 __114-[NEFilterDataExtensionProviderContext handleDataCompleteForFlow:direction:reply:controlSocket:completionHandler:]_block_invoke.312	(in NetworkExtension) + 133	[0x7fff3e92c03d]
		+														 2833 -[NEFilterSocketFlow createDataCompleteReply:controlSocket:direction:verdict:context:]	(in NetworkExtension) + 174	[0x7fff3e9362dc]
		+															 2833 -[NEFilterSocketFlow createDataReply:controlSocket:direction:verdict:context:]	(in NetworkExtension) + 109	[0x7fff3e936215]
		+																 2833 -[NEFilterSocketFlow writeCurrentVerdictWithMessage:controlSocket:]	(in NetworkExtension) + 366	[0x7fff3e936479]
		+																	 2833 +[NEFilterSocketFlow writeMessageWithControlSocket:drop:socketID:inboundPassOffset:inboundPeekOffset:outboundPassOffset:outboundPeekOffset:statsReportFrequency:]	(in NetworkExtension) + 158	[0x7fff3e9368eb]
		+																		 2833 write	(in libsystem_kernel.dylib) + 10	[0x7fff72053bf6]
		2833 Thread_210045
			2833 start_wqthread	(in libsystem_pthread.dylib) + 15	[0x7fff7210fb77]
				2833 _pthread_wqthread	(in libsystem_pthread.dylib) + 390	[0x7fff72110aa1]
					2833 __workq_kernreturn	(in libsystem_kernel.dylib) + 10	[0x7fff720524ce]
Which if I understand correctly, indicates extension callbacks weren't even called in the time of the sample.
I also see that memory for network extension grows through time, which might indicate some kind of leak.
My question is - is that a known issue for network extension? How can I debug network extension with instruments? It crashes immediately when I try to attach to it.
Thanks!
Hi,
I am trying to create an app that filters network events - whether to collect statistics, or to even block some specific flows.
I see that using NEFilterDataProvider I am able to only filter UDP or TCP protocols (when filtering by .any, I see I only receive UDP/TCP). For example, I wish to see the flow of a simple ping 1.1.1.1.
This of course makes the statistics partial (without ICMP packet/raw socket packets), or the flow block being bypass-able (even if with some effort), by using Raw sockets.
Is there a way to add to the filtering also ICMP and RAW flows? Should I use a different provider for those?
When using es_mute_path_prefix to mute processes under my own app directory, I see that we are still getting AUTH_EXEC events on new processes executed from that directory.
Is that the expected behavior? Should I add a check under AUTH_EXEC to mute/ignore each new process, if it matches that prefix?
Hi,
After upgrading OS to 11.3 beta, we are seeing Network Extension crashes. When removing PacketFilter from the extension, crashes stop.
Crash logs suggests issue is in the beta OS rather than on the extension itself:
Thread 2 Crashed:: Dispatch queue: com.apple.network.connections
0	 libdispatch.dylib						 0x00007fff203c92e0 dispatch_async + 153
1	 libnetwork.dylib							 0x00007fff2436fd51 nw_queue_context_async + 81
2	 libnetwork.dylib							 0x00007fff24795fd4 nw_interpose_handle_path_update_locked + 372
3	 libnetwork.dylib							 0x00007fff24795e3c __nw_interpose_start_block_invoke.3 + 108
4	 libdispatch.dylib						 0x00007fff203c4673 _dispatch_call_block_and_release + 12
5	 libdispatch.dylib						 0x00007fff203c5856 _dispatch_client_callout + 8
6	 libdispatch.dylib						 0x00007fff203cd4cc _dispatch_workloop_invoke + 2140
7	 libdispatch.dylib						 0x00007fff203d5c5d _dispatch_workloop_worker_thread + 811
8	 libsystem_pthread.dylib			 0x00007fff2056c4c0 _pthread_wqthread + 314
9	 libsystem_pthread.dylib			 0x00007fff2056b493 start_wqthread + 15
Is this a known issue?
We have also opened a ticket FB8995139
When responding to ES_ACTION_TYPE_AUTH event, there is a flag called cache we can pass to either es_respond_auth_result or es_respond_flags_result functions.
In the documentation it states caching semantics depend on specific event - but actual caching semantics to each event is never documented. This means we pass this flag, hoping not too much or too little will be considered as cached.
For example, we have learned it in the hard way that if we cache an authorization for an event executed by root, it will be cached for all users (as caching semantics apparently does not take user into account).
Are those semantics documented anywhere? It is hard to use this framework solely counting on trial-and-error.
Thanks.
Hi,
We are developing a Network Extension, containing NEFilterDataProvider and NEFilterPacketProvider.
We are seeing, not consistently, issues with network connectivity, or severe latency, in several cases when using Network Extensions:
When there is another Network Extension on the system (VPN)
When network interface is changed between LAN adapter to Wi-Fi, or the other way around.
This possibly started on macOS 11.3.
We were wondering if this (especially the latency) is caused by us filtering some crucial system network operations or processes, that we shouldn't collect statistics on, or at least try our best to refrain from doing so.
Our goal is of course not to affect regular network connectivity, whilst still being able to deny all network connections (including existing) on a specific endpoint.
Is there such list of processes? Are there things we absolutely should refrain from doing?
Thanks.
On some machines (but not all of them), starting macOS 12.1 we do not see ICMP flow anymore on NEFilterPacketProvider. Possibly this happens on machines where the Packet Filter is pre-authorized by a MDM profile.
Furthermore, since NEFilterDataProvider sees only outgoing ICMP flows, this means we are blind for any incoming ICMP traffic.
Is this a known issue? Is there any workaround for that?
Hi,
We keep getting the following crash, with an App compiled with Xcode 13, on macOS Ventura beta 6:
Crashed Thread: 0 Dispatch queue: com.apple.main-thread
Exception Type: EXC_BAD_INSTRUCTION (SIGILL)
Exception Codes: 0x0000000000000001, 0x0000000000000000
Termination Reason: Namespace SIGNAL, Code 4 Illegal instruction: 4
Terminating Process: exc handler [2484]
Application Specific Signatures:
API Misuse
Thread 0 Crashed:: Dispatch queue: com.apple.main-thread
0 libxpc.dylib 0x7ff81a6b8936 _xpc_api_misuse + 117
1 libxpc.dylib 0x7ff81a6a0a49 xpc_connection_set_target_uid + 188
2 WindowManagement 0x7ffb32635bfb -[WMClientWindowManager _createXPCConnection] + 859
3 WindowManagement 0x7ffb3263661e -[WMClientWindowManager _xpcConnection] + 174
4 WindowManagement 0x7ffb326340c6 -[WMClientWindowManager stages] + 86
5 AppKit 0x7ff81e619737 __54-[NSWMWindowCoordinator initializeStageFramesIfNeeded]_block_invoke + 35
6 libdispatch.dylib 0x7ff81a7b1aa4 _dispatch_client_callout + 8
7 libdispatch.dylib 0x7ff81a7b2c92 _dispatch_once_callout + 20
8 AppKit 0x7ff81e619712 -[NSWMWindowCoordinator initializeStageFramesIfNeeded] + 86
9 AppKit 0x7ff81db2d0a4 -[NSWindow _commonInitFrame:styleMask:backing:defer:] + 867
10 AppKit 0x7ff81db2c932 -[NSWindow _initContent:styleMask:backing:defer:contentView:] + 1067
11 AppKit 0x7ff81db2c500 -[NSWindow initWithContentRect:styleMask:backing:defer:] + 42
12 AppKit 0x7ff81db2aa1f -[NSWindowTemplate nibInstantiate] + 354
13 AppKit 0x7ff81daf8f1e -[NSIBObjectData instantiateObject:] + 222
14 AppKit 0x7ff81daf8690 -[NSIBObjectData nibInstantiateWithOwner:options:topLevelObjects:] + 476
15 AppKit 0x7ff81daed16b loadNib + 420
16 AppKit 0x7ff81daec557 +[NSBundle(NSNibLoading) _loadNibFile:nameTable:options:withZone:ownerBundle:] + 737
17 AppKit 0x7ff81daec181 -[NSBundle(NSNibLoading) loadNibNamed:owner:topLevelObjects:] + 201
18 AppKit 0x7ff81daebf5f +[NSBundle(NSNibLoading) loadNibNamed:owner:] + 394
19 AppKit 0x7ff81dadea6f NSApplicationMain + 566
20 <App name> 0x1040399c9 main + 9
21 dyld 0x7ff81a601310 start + 2432
Same build works perfectly on macOS Ventura.
This app is UI-less, notarized and has several system entitlements (if it is relevant). We thought the Xcode version might be related, but since this App is supposed to be released before Xcode 14 is out, we do not wish to build a production app using a beta IDE.
Is this a known issue? Is there something we can do about it?