Simple transparent app proxy Network Extensions on macOS

I am developing a simple transparent proxy that does forward any flow to the destination on macOS.

I set up configuration using NETransparentProxyManager and able to start AppProxy provider and get handleNewFlow: callback, however

when I do NEAppProxyProvider createTCPConnectionToEndpoint: .... to the destination endpoint, the connection establishes but stays in the waiting state NWTCPConnectionStateWaiting and the console logs the policy deny message (see below).


My app id has all entitlements and Content Filter network extension works just fine from within the same extension.

-App Groups

-Custom Network Protocol

-Network Extensions

-Personal VPN

-System Extension


Apparently OS thinks that extension does not have Network Extension privilege PRIV_NET_PRIVILEGED_NECP_MATCH: why?


What am I missing?


Sandbox: com.xxxxxxxxxxxx(42182) System Policy: deny(1) system-privilege 10006

Violation: System Policy: deny(1) system-privilege 10006

Process: com.xxxxxxxxxxxx [42182]

Path: /Library/SystemExtensions/99D00C16-EDD3-455F-B5E8-B71DDDA2BBB4/com.xxxxxxxxxxxx.CatalinaPlusTest.PacketTunnelPlus.systemextension/Contents/MacOS/com.xxxxxxxxxxxx.CatalinaPlusTest.PacketTunnelPlus

Load Address: 0x10f5a5000

Identifier: com.xxxxxxxxxxxx.CatalinaPlusTest.PacketTunnelPlus

Version: 1 (1.0)

Code Type: x86_64 (Native)

Parent Process: launchd [1]

Responsible: /Library/SystemExtensions/99D00C16-EDD3-455F-B5E8-B71DDDA2BBB4/com.xxxxxxxxxxxx.CatalinaPlusTest.PacketTunnelPlus.systemextension/Contents/MacOS/com.xxxxxxxxxxxx.CatalinaPlusTest.PacketTunnelPlus

User ID: 0


Date/Time: 2020-03-24 15:39:32.600 PDT

OS Version: Mac OS X 10.15.4 (19E264b)

Report Version: 8


MetaData: {"primary-filter-value":10006,"errno":1,"pid":42182,"signing-id":"com.xxxxx.CatalinaPlusTest.PacketTunnelPlus","platform-policy":true,"primary-filter":"privilege-id","team-id":"C489D5E8E8","process":"xxxxxx","platform-binary":false,"target":10006,"privilege-id":"PRIV_NET_PRIVILEGED_NECP_MATCH","action":"deny","hardware":"Mac","platform_binary":"no","profile-flags":0,"responsible-process-user-uuid":"FFFFEEEE-DDDD-CCCC-BBBB-AAAA00000000","responsible-process-path":"\/Library\/SystemExtensions\/99D00C16-EDD3-455F-B5E8-B71DDDA2BBB4\/com.xxxxxxxxxxxx.CatalinaPlusTest.PacketTunnelPlus.systemextension\/Contents\/MacOS\/com.xxxxxxxxxxxx.CatalinaPlusTest.PacketTunnelPlus","profile":"platform","flags":5,"apple-internal":false,"process-path":"\/Library\/SystemExtensions\/99D00C16-EDD3-455F-B5E8-B71DDDA2BBB4\/com.xxxxxxxxxxxx.CatalinaPlusTest.PacketTunnelPlus.systemextension\/Contents\/MacOS\/com.xxxxxxxxxxxx.CatalinaPlusTest.PacketTunnelPlus","build":"Mac OS X 10.15.4 (19E264b)","responsible-process-uid":0,"uid":0,"summary":"deny(1) system-privilege 10006","operation":"system-privilege"}


Thread 0 (id: 4951912):

0 libsystem_kernel.dylib 0x00007fff66fc44da __semwait_signal_nocancel + 10

1 libsystem_c.dylib 0x00007fff66ef7f38 sleep$NOCANCEL + 41

2 libdispatch.dylib 0x00007fff66e343da _dispatch_queue_cleanup2 + 156

3 libsystem_pthread.dylib 0x00007fff67080054 _pthread_tsd_cleanup + 551

4 libsystem_pthread.dylib 0x00007fff67082512 _pthread_exit + 70

5 libsystem_pthread.dylib 0x00007fff6707fe08 pthread_exit + 42

6 libdispatch.dylib 0x00007fff66e2ffce libdispatch_init + 0

7 com.xxxxxxxxxxxx.CatalinaPlusTest.PacketTunnelPlus 0x000000010f5a5f5e

8 libdyld.dylib 0x00007fff66e7dcc9 start + 1

9 com.xxxxxxxxxxxx.CatalinaPlusTest.PacketTunnelPlus 0x0000000000000001


Thread 1 (id: 4951932):

0 libsystem_kernel.dylib 0x00007fff66fc04ce __workq_kernreturn + 10

1 libsystem_pthread.dylib 0x00007fff6707db77 start_wqthread + 15


Thread 2 (id: 4951933):

0 libsystem_kernel.dylib 0x00007fff66fc2072 necp_client_action + 10

1 libnetwork.dylib 0x00007fff657c7328 nw_path_create_evaluator_for_endpoint + 760

2 Network 0x00007fff3385b2d3 -[NWPathEvaluator initWithEndpoint:parameters:] + 531

3 Network 0x00007fff3385b0a4 __41+[NWPathEvaluator sharedDefaultEvaluator]_block_invoke + 36

4 libdispatch.dylib 0x00007fff66e24658 _dispatch_client_callout + 8

5 libdispatch.dylib 0x00007fff66e257de _dispatch_once_callout + 20

6 Network 0x00007fff3385b07e +[NWPathEvaluator sharedDefaultEvaluator] + 46

7 NetworkExtension 0x00007fff33b0fecd -[NEProvider initAllowUnentitled:] + 248

8 NetworkExtension 0x00007fff339f0d92 -[NEExtensionProviderContext createWithCompletionHandler:] + 398

9 Foundation 0x00007fff2f6514f3 __NSXPCCONNECTION_IS_CALLING_OUT_TO_EXPORTED_OBJECT_S1__ + 10

10 Foundation 0x00007fff2f5db9be -[NSXPCConnection _decodeAndInvokeMessageWithEvent:flags:] + 2363

11 Foundation 0x00007fff2f592b29 message_handler + 210

12 libxpc.dylib 0x00007fff670c22bc _xpc_connection_call_event_handler + 56

13 libxpc.dylib 0x00007fff670c11cb _xpc_connection_mach_event + 934

14 libdispatch.dylib 0x00007fff66e246f8 _dispatch_client_callout4 + 9

15 libdispatch.dylib 0x00007fff66e39bc9 _dispatch_mach_msg_invoke + 435

16 libdispatch.dylib 0x00007fff66e29af6 _dispatch_lane_serial_drain + 263

17 libdispatch.dylib 0x00007fff66e3a71c _dispatch_mach_invoke + 481

18 libdispatch.dylib 0x00007fff66e29af6 _dispatch_lane_serial_drain + 263

19 libdispatch.dylib 0x00007fff66e2a609 _dispatch_lane_invoke + 414

20 libdispatch.dylib 0x00007fff66e33c09 _dispatch_workloop_worker_thread + 596

21 libsystem_pthread.dylib 0x00007fff6707ea3d _pthread_wqthread + 290

22 libsystem_pthread.dylib 0x00007fff6707db77 start_wqthread + 15


Thread 3 (id: 4951934):

0 libsystem_kernel.dylib 0x00007fff66fc4502 __sigsuspend_nocancel + 10

1 libdispatch.dylib 0x00007fff66e34476 _dispatch_sigsuspend + 0


Binary Images:

0x10f5a5000 - 0x10f5a9ff3 com.xxxxxxxxxxxx.CatalinaPlusTest.PacketTunnelPlus (1.0 - 1) <67ce2dcd-47a0-31da-8312-71c9e4fe9e4c> /Library/SystemExtensions/99D00C16-EDD3-455F-B5E8-B71DDDA2BBB4/com.xxxxxxxxxxxx.CatalinaPlusTest.PacketTunnelPlus.systemextension/Contents/MacOS/com.xxxxxxxxxxxx.CatalinaPlusTest.PacketTunnelPlus

0x7fff2f571000 - 0x7fff2f936ff8 com.apple.Foundation (6.9 - 1675.129) <9a74fa97-7f7b-3929-b381-d9514b1e4754> /System/Library/Frameworks/Foundation.framework/Versions/C/Foundation

0x7fff3385a000 - 0x7fff339b1ff3 com.apple.Network (1.0 - 1) <d1c8fdde-c822-3c40-bb26-18f24cfc8ae2> /System/Library/Frameworks/Network.framework/Versions/A/Network

0x7fff339b2000 - 0x7fff33c11ff7 com.apple.NetworkExtension (1.0 - 1) <cb7e4930-c6ec-3642-b4bf-2b9d54ba3a53> /System/Library/Frameworks/NetworkExtension.framework/Versions/A/NetworkExtension

0x7fff657a8000 - 0x7fff65c23ff5 libnetwork.dylib (1880.100.30) <9519b6f8-44e2-3f53-b995-1527c5333240> /usr/lib/libnetwork.dylib

0x7fff66e22000 - 0x7fff66e62ff0 libdispatch.dylib (1173.100.2) <eb592997-b11c-3ab3-85b1-f725f3d0b412> /usr/lib/system/libdispatch.dylib

0x7fff66e63000 - 0x7fff66e99fff libdyld.dylib (750.5) <d2a07ef5-a64b-3692-be13-89daa2ec5e80> /usr/lib/system/libdyld.dylib

0x7fff66ecd000 - 0x7fff66f54fff libsystem_c.dylib (1353.100.2) <4f5eed22-4d46-3f04-8c64-c492cdad70eb> /usr/lib/system/libsystem_c.dylib

0x7fff66fbe000 - 0x7fff66feaff7 libsystem_kernel.dylib (6153.101.6) <e76440e1-d1e8-3d9a-8b47-d01f554ff1c4> /usr/lib/system/libsystem_kernel.dylib

0x7fff6707c000 - 0x7fff67086fff libsystem_pthread.dylib (416.100.3) <a8514582-e000-3854-911a-0a73d2c79600> /usr/lib/system/libsystem_pthread.dylib

0x7fff670b5000 - 0x7fff670eaffe libxpc.dylib (1738.100.39) <32b0e31e-9da3-328b-a962-bc9591b93537> /usr/lib/system/libxpc.dylib

Accepted Reply

That may not apply to your issue, but make sure that you don't have includeAllNetworks set to true (which would sound logical in the first place but causes all sorts of weird failures) in the NETunnelProviderProtocol instance you pass to the NETransparentProxyManager while configuring the proxy in the main app.

Doing so causes a networking loop back into the transparent proxy that gets NECP deny messages, which really do not explain the base issue at all. Reported as FB7468866.

Replies

That may not apply to your issue, but make sure that you don't have includeAllNetworks set to true (which would sound logical in the first place but causes all sorts of weird failures) in the NETunnelProviderProtocol instance you pass to the NETransparentProxyManager while configuring the proxy in the main app.

Doing so causes a networking loop back into the transparent proxy that gets NECP deny messages, which really do not explain the base issue at all. Reported as FB7468866.

I was also facing this issue (deny(1) system-privilege 10006) with PacketTunnel on macOS and includeAllNetworks was the culprit. Thanks to your answer, I was able to resolve the issue (only after wasting more than a day). Has there been any update on FB7468866 to fix this issue?
There was an update to the documentation that was posted recently for includeAllNetworks. Notice that this is not supported for NETransparentProxyManager.


Matt Eaton
DTS Engineering, CoreOS
meaton3@apple.com
Thanks for the update Matt.

However, I received this issue with Packet Tunnel. Whenever includeAllNetworks is enabled, if PacketTunnelProvider tries to apply a split tunnel rule, the rule fails to apply (but no error returned) and packet tunnel receives all the traffic as if its full tunnel and wildcard match domain.

Now I understand that when includeAllNetworks is enabled, split tunnel rules conflict with this setting and should not be used. But there is no clear documentation or error when settings split tunnel rules. There is just a cryptic message in Console.app System Policy: deny(1) system-privilege 10006 which is easy to miss. It does not even specify the problem.

So, I think it would be nicer to have documentation mention this conflict and its impact. It would be even nicer to have NETunnelProvider.setTunnelNetworkSettings method return an error in its completion block when such conflicts take place.

Now I understand that when includeAllNetworks is enabled, split tunnel rules conflict with this setting and should not be used. But there is no clear documentation or error when settings split tunnel rules.
So, I think it would be nicer to have documentation mention this conflict and its impact.

I agree. I think this is a great enhancement request for documentation on this matter. Please respond back with the Feedback ID when you have done so.


Matt Eaton
DTS Engineering, CoreOS
meaton3@apple.com
Thanks Matt. Here is the feedback ID: FB9108197

Thanks Matt. Here is the feedback ID: FB9108197

Thank you, I see you bug report internally.


Matt Eaton
DTS Engineering, CoreOS
meaton3@apple.com