Hi, I want to write an application and a system extension to redirect the outgoing network traffic through the application. My design is as follows: Create a thread as proxy server listener on a port (e.g., 1234). So App waits for connections on 127.0.0.1:1234 2. Whenever outgoing traffic is encountered on a specific port, say 567, this traffic should be diverted to proxy listener in the application. Application takes care of sending it out after the processing. Can somebody help me in pointing the relevant APIs in Transparent Proxy and Packet Filter. If any sample code is available, please share with me.

I have the similar scenario. Can you point me to the APIs for NEAppProxyprovider and NEPacketTunnelProvider which can be used to divert the ip/port of outgoing packets

Thanks a lot Matt!! I modified the SimpleFirewall program into simple transparent proxy using NETransparentproxyManager and NEAppProxyProvider. Transparent proxy. It catches the outbound network traffic as per defined NENetworkRule. handleNewFlow is also called. In handleNewFlow, I opened connection to remote server using NWTCPconnection and once the connection status is connected => I called openWithLocalEndpoint. For data transfer, I am using tcpflow.read() and conn.write() methods for reading from caught TCP flow and writing on connection to remote server. It is reading the data properly (I am printing the whole message in logs), but I am getting the following error in console logs while writing on remote connection and nothing is received on remote server. "Error Domain=kNWErrorDomainPOSIX Code=89 "Operation canceled" UserInfo={NSDescription=Operation canceled}" Can you please help me resolving this issue.

I am also facing the same issue of getting into loop. I am not clear about the method of using audit token from flow's metadata. If you have a working example, can you please share. Is there any simple way of avoiding the capture of tcp flow coming from specific process(in our case it is launch agent).

Thanks Matt for your reply. I got the looping issue resolved. I have a generic question on macOS network extension - transparent proxy provider. Is it a must to implement transparent proxy provider as a system extension inside an app? With this approach, we are observing 2 popups , one popup for system extension, one popup while adding VPN configuration (for monitoring network). The other issue is user can see the transparent proxy in the Network Preferences, which is weird. This is something different when compared to NKE behavior. We are looking for ways to avoid these 3 things. Can we avoid above popups and proxy inside network preferences? Before trying anything, I would like a guidance whether trying that approach worth or not. We are thinking to try implementing transparent proxy provider inside a launchdaemon? Can we implement transparent proxy provider inside a launchdaemon? or if system extension is the only option, then how can. we avoid above 2 popups and proxy visible in network preference?

Matt, You mentioned that "The container app and Network System Extension for your NETransparentProxyProvider will also need a Sandbox". But in my sample, I have got a network system extension inside an app without sandbox. It is working as expected. One point to note here - disabled SIP on my system. Does disabling SIP lift the sandbox requirement of app and network system extension?

I have couple of queries: Is there any way we can disable or grey out the connect/disconnect option for transparent proxy provider in network preferences? The transparent proxy provider is shown as VPN application in network preferences. Will this be impacted when other VPNs are installed and running on the system?

Thanks Matt for the reply! When we add/update IMAP account settings and save, then the flow is going through accountsd (either 993 or 143) which can not be captured by transparent proxy. Once the mail account is configured and saved (either it is 143 or 993), if we send a mail from Mail.app, then Mail app is making connection to Mail Server and that is being captured. So in brief, Mail.app makes connection (any SMTP or IMAP ports) => proxy catches that flow. Steps to reproduce the specific scenario are: #1: Add an IMAP mail account on IMAP SSL port 993 and SMTP non-SSL port 25 and save it. #2: Install transparent proxy and it is running with the configured network rules to catch any SMTP and IMAP traffic. #3. Send mail from Mail.app and it gets captured. #4: While transparent proxy is up and running, open IMAP mail account settings and edit settings of IMAP from SSL to nonSSL and port from 993 to 143 and click Save. When Save is clicked, the connection which goes out is from accountsd and not from Mail.app. This is not being captured. My network rules are as below: (NENetworkRule *)createNeRuleFor:(NSString *)port { [[NENetworkRule alloc] initWithRemoteNetwork:[NWHostEndpoint endpointWithHostname:@"0.0.0.0" port:port]                                            remotePrefix:0                                            localNetwork:nil                                            localPrefix:0                                            protocol:NENetworkRuleProtocolTCP                                            direction:NETrafficDirectionOutbound]; } settings.includedNetworkRules = @[         [self createNeRuleFor:@"25"],         [self createNeRuleFor:@"587"],         [self createNeRuleFor:@"110"],         [self createNeRuleFor:@"995"],         [self createNeRuleFor:@"143"],         [self createNeRuleFor:@"993"],     ]; Query#1: Is there any restriction on the ability of network transparent proxies on catching network traffic from some system processes such as accountsd process?

Opened feedback request, ID is FB8935382

Thanks Matt for your immediate response. We use NEAppProxyProvider as we still use Xcode11.x. Per your suggestion, we will try using NETransparentProxyProvider and check the behaviour.

Hi,  I am observing the similar crash in sysextd while updating a network system extension on macOS12.4. The update failed, I still see old system extension. Please find the crash report below: Translated Report (Full Report Below) Process: sysextd [1214] Path: /System/Library/Frameworks/SystemExtensions.framework/Versions/A/Helpers/sysextd Identifier: sysextd Version: ??? Code Type: X86-64 (Native) Parent Process: launchd [1] User ID: 0 Date/Time: 2022-06-21 18:22:30.1240 +0530 OS Version: macOS 12.4 (21F79) Report Version: 12 Bridge OS Version: 6.5 (19P5071) Anonymous UUID: 492561C8-C69A-A8BF-3340-633DA3ABC374 Time Awake Since Boot: 1600 seconds System Integrity Protection: enabled Crashed Thread: 1 Dispatch queue: sysextd.extension_manager Exception Type: EXC_BAD_INSTRUCTION (SIGILL) Exception Codes: 0x0000000000000001, 0x0000000000000000 Exception Note: EXC_CORPSE_NOTIFY Termination Reason: Namespace SIGNAL, Code 4 Illegal instruction: 4 Terminating Process: exc handler [1214] Thread 0: 0 libsystem_kernel.dylib 0x7ff81cd1e1aa __semwait_signal_nocancel + 10 1 libsystem_c.dylib 0x7ff81cc4db18 nanosleep$NOCANCEL + 185 2 libsystem_c.dylib 0x7ff81cc4da25 sleep$NOCANCEL + 41 3 libdispatch.dylib 0x7ff81cbaf75f _dispatch_queue_cleanup2 + 158 4 libsystem_pthread.dylib 0x7ff81cd53567 _pthread_tsd_cleanup + 487 5 libsystem_pthread.dylib 0x7ff81cd55b89 _pthread_exit + 70 6 libsystem_pthread.dylib 0x7ff81cd5333d pthread_exit + 42 7 libdispatch.dylib 0x7ff81cbaafb9 dispatch_main + 96 8 sysextd 0x10d40a2a1 0x10d401000 + 37537 9 sysextd 0x10d4098f9 0x10d401000 + 35065 10 dyld 0x1137ef51e start + 462 Thread 1 Crashed:: Dispatch queue: sysextd.extension_manager 0 libswiftCore.dylib 0x7ff82a22e245 assertionFailure(:_:file:line:flags:) + 421 1 sysextd 0x10d436803 0x10d401000 + 219139 2 sysextd 0x10d43a407 0x10d401000 + 234503 3 sysextd 0x10d437308 0x10d401000 + 221960 4 sysextd 0x10d43736f 0x10d401000 + 222063 5 Foundation 0x7ff81dcde088 NSXPCCONNECTION_IS_CALLING_OUT_TO_EXPORTED_OBJECT_S1 + 10 6 Foundation 0x7ff81dc88738 -[NSXPCConnection _decodeAndInvokeMessageWithEvent:flags:] + 2347 7 Foundation 0x7ff81dc3f31e message_handler + 206 8 libxpc.dylib 0x7ff81ca94b70 _xpc_connection_call_event_handler + 56 9 libxpc.dylib 0x7ff81ca93956 _xpc_connection_mach_event + 1413 10 libdispatch.dylib 0x7ff81cb9e3b1 _dispatch_client_callout4 + 9 11 libdispatch.dylib 0x7ff81cbb7041 _dispatch_mach_msg_invoke + 445 12 libdispatch.dylib 0x7ff81cba41cd _dispatch_lane_serial_drain + 342 13 libdispatch.dylib 0x7ff81cbb7b77 _dispatch_mach_invoke + 484 14 libdispatch.dylib 0x7ff81cba41cd _dispatch_lane_serial_drain + 342 15 libdispatch.dylib 0x7ff81cba4dfd _dispatch_lane_invoke + 366 16 libdispatch.dylib 0x7ff81cbaeeee _dispatch_workloop_worker_thread + 753 17 libsystem_pthread.dylib 0x7ff81cd51fd0 _pthread_wqthread + 326 18 libsystem_pthread.dylib 0x7ff81cd50f57 start_wqthread + 15 Thread 2: 0 libsystem_kernel.dylib 0x7ff81cd1e1d2 __sigsuspend_nocancel + 10 1 libdispatch.dylib 0x7ff81cbaf81b _dispatch_sigsuspend + 36 2 libdispatch.dylib 0x7ff81cbaf7f7 _dispatch_sig_thread + 49 Thread 3: 0 libsystem_pthread.dylib 0x7ff81cd50f48 start_wqthread + 0 Thread 4: 0 libsystem_pthread.dylib 0x7ff81cd50f48 start_wqthread + 0 Thanks, Ravi

Hi Eskimo, Thanks for your reply. We still observe the issue even after restart. Sharing some more information here: The current output of 'systemextensionsctl list' looks like this (same as before update too) admin@admins-MacBook-Pro-2 ~ % systemextensionsctl list                                            1 extension(s) --- com.apple.system_extension.network_extension enabled active teamID bundleID  (version) name [state] '* *   ABC my.sysext.bundleid (4.3.2.123/4.3.2.123) MySystemExtension [activated enabled] Suppose we are trying to update to 4.3.2.567 and it is failing.

Thanks Matt for the reply. If the code signing is problem, how the installation has been successful? However, for the analysis, I am sharing further information below: System extension installation location: /Applications/SampleGUI.app/Contents/Applications/SampleSysExtHost.app/Contents/Library/SystemExtensions/my.sysext.bundleid.systemextension SampleSysExtHost.app ===> system extension container app my.sysext.bundleid.systemextension ===> network system extension. Please find the container app and system extension code signing and entitlements details in attached file: CodeSigning Details

Sharing the details again: **ContainerApp codesign output:** devs-iMac:Applications dev$ codesign -d --entitlements - SampleSysExtHost.app Executable=/Applications/SampleGUI.app/Contents/Applications/SampleSysExtHost.app/Contents/MacOS/SampleSysExtHost <?xml version="1.0" encoding="UTF-8"?> <!DOCTYPE plist PUBLIC "-//Apple//DTD PLIST 1.0//EN" "http://www.apple.com/DTDs/PropertyList-1.0.dtd"> <plist version="1.0"> <dict> <key>com.apple.application-identifier</key> <string>[TEAM_ID].[APP_ID]</string> <key>com.apple.developer.networking.networkextension</key> <array> <string>app-proxy-provider-systemextension</string> </array> <key>com.apple.developer.system-extension.install</key> <true/> <key>com.apple.developer.team-identifier</key> <string>[TEAM_ID]</string> <key>com.apple.security.get-task-allow</key> <false/> </dict> </plist> **ContainerApp provisioning profile output:** devs-iMac:Contents dev$ security cms -D -i embedded.provisionprofile  <?xml version="1.0" encoding="UTF-8"?> <!DOCTYPE plist PUBLIC "-//Apple//DTD PLIST 1.0//EN" "http://www.apple.com/DTDs/PropertyList-1.0.dtd"> <plist version="1.0"> <dict> <key>AppIDName</key> <string>[Profile AppID Name]</string> <key>ApplicationIdentifierPrefix</key> <array> <string>[TEAM_ID]</string> </array> <key>CreationDate</key> <date>[DATE]</date> <key>Platform</key> <array> <string>OSX</string> </array> <key>IsXcodeManaged</key> <false/> <key>DeveloperCertificates</key> <array> <data>[DATA]</data> </array> <key>Entitlements</key> <dict> <key>com.apple.developer.system-extension.install</key> <true/> <key>com.apple.developer.networking.networkextension</key> <array> <string>packet-tunnel-provider-systemextension</string> <string>app-proxy-provider-systemextension</string> <string>content-filter-provider-systemextension</string> <string>dns-proxy-systemextension</string> <string>dns-settings</string> </array> <key>com.apple.application-identifier</key> <string>[TEAM_ID].[APP_ID]</string> <key>keychain-access-groups</key> <array> <string>[TEAM_ID].*</string> </array> <key>com.apple.developer.team-identifier</key> <string>[TEAM_ID]</string> </dict> <key>ExpirationDate</key> <date>[DATE</date> <key>Name</key> <string>[PROFILE NAME]</string> <key>ProvisionsAllDevices</key> <true/> <key>TeamIdentifier</key> <array> <string>[TEAM_ID]</string> </array> <key>TeamName</key> <string>[TEAM_NAME]</string> <key>TimeToLive</key> <integer>6570</integer> <key>UUID</key> <string>[UUID]</string> <key>Version</key> <integer>1</integer> </dict> </plist> **System extension codesign output:** devs-iMac:SystemExtensions dev$ codesign -d --entitlements - my.sysext.bundleid.systemextension <?xml version="1.0" encoding="UTF-8"?> <!DOCTYPE plist PUBLIC "-//Apple//DTD PLIST 1.0//EN" "http://www.apple.com/DTDs/PropertyList-1.0.dtd"> <plist version="1.0"> <dict> <key>com.apple.application-identifier</key> <string>[TEAM_ID].[APP_ID]</string> <key>com.apple.developer.networking.networkextension</key> <array> <string>app-proxy-provider-systemextension</string> </array> <key>com.apple.developer.team-identifier</key> <string>[TEAM_ID]</string> <key>com.apple.security.application-groups</key> <array> <string>[TEAM_ID].com.test.appgroup</string> </array> <key>com.apple.security.get-task-allow</key> <false/> </dict> </plist> **System extension provisioning profile output:** devs-iMac:Contents dev$ security cms -D -i embedded.provisionprofile <?xml version="1.0" encoding="UTF-8"?> <!DOCTYPE plist PUBLIC "-//Apple//DTD PLIST 1.0//EN" "http://www.apple.com/DTDs/PropertyList-1.0.dtd"> <plist version="1.0"> <dict> <key>AppIDName</key> <string>[NAME]</string> <key>ApplicationIdentifierPrefix</key> <array> <string>[TEAM_ID]</string> </array> <key>CreationDate</key> <date>[DATE]</date> <key>Platform</key> <array> <string>OSX</string> </array> <key>IsXcodeManaged</key> <false/> <key>DeveloperCertificates</key> <array> <data>[DATA]</data> </array> <key>Entitlements</key> <dict> <key>com.apple.developer.networking.networkextension</key> <array> <string>packet-tunnel-provider-systemextension</string> <string>app-proxy-provider-systemextension</string> <string>content-filter-provider-systemextension</string> <string>dns-proxy-systemextension</string> <string>dns-settings</string> </array> <key>com.apple.application-identifier</key> <string>[TEAM_ID].[APP_ID]</string> <key>keychain-access-groups</key> <array> <string>[TEAM_ID].*</string> </array> <key>com.apple.developer.team-identifier</key> <string>[TEAM_ID]</string> </dict> <key>ExpirationDate</key> <date>[DATE]</date> <key>Name</key> <string>[PROFILE_NAME]</string> <key>ProvisionsAllDevices</key> <true/> <key>TeamIdentifier</key> <array> <string>[TEAM_ID]</string> </array> <key>TeamName</key> <string>[TEAM_NAME]</string> <key>TimeToLive</key> <integer>6570</integer> <key>UUID</key> <string>[UUID]</string> <key>Version</key> <integer>1</integer> </dict> </plist>

Because of this upgrade issue, we are unable to uninstall the system extension. Deactivation request is failing. The only remaining way to uninstall the system extension is disabling SIP which should not be done in general. Can you please have a look at the above the signing related information and let us know what to do next.