iOS 10.3 breaks IKEv2?

We have a VPN app in the app store which has worked great until our users updated to iOS 10.3. As an aside, macOS also broke with the latest update, 10.12.4, but I'll focus on iOS in this post unless directed otherwise.


From devices system logs on iPhone 6 running iOS 10.3:

nesessionmanager(NetworkExtension)[398] <Notice>: NESMIKEv2VPNSession[APPNAME:6736EF81-160B-40BD-BA71-B65C486EEB2D]: status changed to connecting
nesessionmanager(NetworkExtension)[398] <Error>: Plugin com.apple.neplugin.IKEv2 does not have a bundle URL
APPNAME(CFNetwork)[393] <Notice>: TIC TCP Conn Event [4:0x17019a820]: 3 Err(50)
APPNAME(CFNetwork)[393] <Notice>: TIC TCP Conn Event [3:0x17419aea0]: 3 Err(50)
APPNAME(CFNetwork)[393] <Notice>: TIC TCP Conn Cancel [3:0x17419aea0]
APPNAME(libsystem_network.dylib)[393] <Error>: nw_endpoint_handler_add_write_request [3.1 <IP>:443 failed socket-flow (satisfiable)] cannot accept write requests
APPNAME(CFNetwork)[393] <Notice>: TIC TCP Conn Cancel [4:0x17019a820]
APPNAME(CFNetwork)[393] <Notice>: TIC TCP Conn Destroyed [4:0x17019a820]
APPNAME(libsystem_network.dylib)[393] <Error>: nw_endpoint_handler_add_write_request [4.1 <IP>:443 failed socket-flow (satisfiable)] cannot accept write requests
APPNAME(libsystem_network.dylib)[393] <Error>: tcp_connection_write_eof_block_invoke Write close callback received error: [22] Invalid argument
APPNAME(CFNetwork)[393] <Notice>: TIC TCP Conn Destroyed [3:0x17419aea0]
kernel(Sandbox)[0] <Notice>: SandboxViolation: nesessionmanager(398) deny(1) file-issue-extension target: /System/Library/Frameworks/NetworkExtension.framework/PluginIKEv2.vpnplugin class: com.apple.vpn-plugin
nesessionmanager(NetworkExtension)[398] <Error>: sendInitCommand: failed to create a com.apple.vpn-plugin sandbox extension for /System/Library/Frameworks/NetworkExtension.framework/PluginIKEv2.vpnplugin
nesessionmanager(NetworkExtension)[398] <Notice>: NESMIKEv2VPNSession[APPNAME:6736EF81-160B-40BD-BA71-B65C486EEB2D]: Skip a start command from APPNAME[393]: session in state connecting
kernel(Sandbox)[0] <Notice>: SandboxViolation: neagent(400) deny(1) process-exec* /Developer/usr/libexec/neagent
nehelper(libsystem_network.dylib)[96] <Notice>: checkInterfaceSettings checking interface settings for (
    ipsec3
)
configd[32] <Notice>: network changed
kernel[0] <Notice>: SIOCPROTODETACH_IN6: pdp_ip0 error=6
kernel[0] <Notice>: SIOCPROTODETACH_IN6: pdp_ip1 error=6
securityd[97] <Notice>: cert[1]: AnchorTrusted =(leaf)[force]> 0
neagent(Security)[400] <Notice>:  [root AnchorTrusted]
securityd[97] <Notice>: cert[0]: AnchorTrusted =(leaf)[force]> 0
neagent(Security)[400] <Notice>:  [leaf AnchorTrusted]
neagent(EAP8021X)[400] <Notice>: server certificate not trusted status 1001 -9807
neagent(NetworkExtension)[400] <Error>: Failed to process IKE Auth (EAP) packet
nesessionmanager(NetworkExtension)[398] <Notice>: NESMIKEv2VPNSession[APPNAME:6736EF81-160B-40BD-BA71-B65C486EEB2D]: status changed to disconnecting
kernel[0] <Notice>: SIOCPROTODETACH_IN6: ipsec3 error=6
configd[32] <Notice>: network changed
nesessionmanager(NetworkExtension)[398] <Notice>: NESMIKEv2VPNSession[APPNAME:6736EF81-160B-40BD-BA71-B65C486EEB2D]: status changed to disconnected, last stop reason Plugin initiated
nesessionmanager(NetworkExtension)[398] <Notice>: NESMIKEv2VPNSession[APPNAME:6736EF81-160B-40BD-BA71-B65C486EEB2D]: Received a stop command from APPNAME[393] with reason 1



The same logs are produced whether I run the app store version or build from Xcode. Several errors are listed but I'm leaning towards an entitlements problem given line 3 regarding the SandboxViolation. Alternatively the requirements for VPN server certs have been tightened?


From this guide I was able to extract the .app file's entitlements:

<key>com.apple.developer.networking.networkextension</key>
  <array>
    <string>app-proxy-provider</string>
    <string>content-filter-provider</string>
    <string>packet-tunnel-provider</string>
  </array>
<key>com.apple.developer.networking.vpn.api</key>
  <array>
    <string>allow-vpn</string>
  </array>


In Xcode I also see both Personal VPN and Network Extensions enabled with all the lines 'checked' and, similarly, our App ID in the developer portal shows both as enabled for both development and production.


I also get this back from the Network Extension API on startVPNTunnel():


Error Domain=VPN Code=1 "(null)"


What other information can I provide? I'm still digging through other related posts to find something to latch onto. Thanks for looking!




Edit 2: Would the combination of (1) an on-demand profile and (2) the requirement for clients to authenticate via client certificates lead to the need for the

com.apple.managed.vpn.shared
requirement? We clearly do not have that (seen in the entitelment list above) and I can see how a request would fail via SandboxException in that case even if it wasn't in previous iOS/macOS versions. I see eskimo's note on this here in note #9: https://forums.developer.apple.com/thread/67613. I just don't want to consume a TSI if that's not the issue.

Replies

It sounds like you’re setting up Personal VPN, that is, using your own to configure the built-in IKEv2 VPN transport. If so, the first step in debugging an issue like this is to try replicating it with a configuration profile. If the problem reproduces with a configuration profile, none of your code is in play and this warrants a compatibility bug report and escalation via AppleCare. If, OTOH, things work with the configuration profile but fail when you set up the same configuration via the NEVPNManager API, you should open a DTS tech support incident and we can investigate that.

[is there a] need for the com.apple.managed.vpn.shared requirement?

No. The

com.apple.managed.vpn.shared
is only needed by custom VPN transport (Network Extension providers), and even there it’s only needed if you want to integrate with configuration via configuration profile (typically in enterprise environments). There’s no way that the built-in IKEv2 transport could possible need this entitlement (it already has all the access it needs).

Share and Enjoy

Quinn “The Eskimo!”
Apple Developer Relations, Developer Technical Support, Core OS/Hardware

let myEmail = "eskimo" + "1" + "@apple.com"

I'm experiencing this behavior too, but only on iOS 10.3.2, NOT on 10.3.


On top of our own app, all other apps I have tried so far using IKEv2 (about 10 VPN apps in the store), fail.


Could this be related? Find logs below.



Mar 30 09:27:19 iPhone nesessionmanager(NetworkExtension)[211] <Error>: Plugin com.apple.neplugin.IKEv2 does not have a bundle URL
Mar 30 09:27:19 iPhone kernel(Sandbox)[0] <Notice>: SandboxViolation: nesessionmanager(211) deny(1) file-issue-extension target: /System/Library/Frameworks/NetworkExtension.framework/PluginIKEv2.vpnplugin class: com.apple.vpn-plugin
Mar 30 09:27:19 iPhone nesessionmanager(NetworkExtension)[211] <Error>: sendInitCommand: failed to create a com.apple.vpn-plugin sandbox extension for /System/Library/Frameworks/NetworkExtension.framework/PluginIKEv2.vpnplugin
Mar 30 09:27:19 iPhone nesessionmanager(NetworkExtension)[211] <Error>: +[NEProcessInfo copyUUIDsFromExecutable:]: cannot open /usr/libexec/neagent: No such file or directory
Mar 30 09:27:19 iPhone nesessionmanager(NetworkExtension)[211] <Error>: +[NEProcessInfo copyUUIDsForExecutable:]_block_invoke: failed to get UUIDs for /usr/libexec/neagent
Mar 30 09:27:19 iPhone nesessionmanager(NetworkExtension)[211] <Notice>: NESMIKEv2VPNSession[MyVPNApp:9CF0CB97-20F3-4FB1-9710-7DCDA332B453]: Skip a start command from nsurlsessiond[127]: session in state connecting
Mar 30 09:27:19 iPhone nesessionmanager(NetworkExtension)[211] <Error>: com.apple.neplugin.IKEv2[inactive]: XPC error received in response to init command: (null)
Mar 30 09:27:19 iPhone nesessionmanager(NetworkExtension)[211] <Error>: com.apple.neplugin.IKEv2[inactive]: dropping a message because the current state is not "started" (0)
Mar 30 09:27:19 iPhone nesessionmanager(NetworkExtension)[211] <Notice>: NESMIKEv2VPNSession[MyVPNApp:9CF0CB97-20F3-4FB1-9710-7DCDA332B453]: status changed to disconnecting
Mar 30 09:27:19 iPhone nesessionmanager(NetworkExtension)[211] <Notice>: NESMIKEv2VPNSession[MyVPNApp:9CF0CB97-20F3-4FB1-9710-7DCDA332B453]: status changed to disconnected, last stop reason Plugin failed

Thanks for your help so far, eskimo. You're a rockstar.


After manually installing a matching config from Settings -> General -> VPN I still get a failure with:

nesessionmanager(NetworkExtension)[398] <Notice>: NESMIKEv2VPNSession[Test:6309C3CA-6B4C-4878-9721-335BDBB82814]: Received a start command from Preferences[637]
nesessionmanager(NetworkExtension)[398] <Notice>: NESMIKEv2VPNSession[Test:6309C3CA-6B4C-4878-9721-335BDBB82814]: status changed to connecting
nesessionmanager(NetworkExtension)[398] <Error>: Plugin com . apple . neplugin . IKEv2 does not have a bundle URL
kernel(Sandbox)[0] <Notice>: SandboxViolation: nesessionmanager(398) deny(1) file-issue-extension target: /System/Library/Frameworks/NetworkExtension.framework/PluginIKEv2.vpnplugin class: com . apple . vpn-plugin
nesessionmanager(NetworkExtension)[398] <Error>: sendInitCommand: failed to create a com .apple .vpn-plugin sandbox extension for /System/Library/Frameworks/NetworkExtension.framework/PluginIKEv2.vpnplugin
kernel(Sandbox)[0] <Notice>: SandboxViolation: neagent(799) deny(1) process-exec* /Developer/usr/libexec/neagent
nehelper(libsystem_network.dylib)[96] <Notice>: checkInterfaceSettings checking interface settings for (
    ipsec3
)
kernel[0] <Notice>: SIOCPROTODETACH_IN6: pdp_ip0 error=6
kernel[0] <Notice>: SIOCPROTODETACH_IN6: pdp_ip1 error=6
neagent(NetworkExtension)[799] <Error>: pfkey received SA is NULL
configd[32] <Notice>: network changed
backboardd(MultitouchHID)[56] <Notice>: [HID] [MT] MTSimpleHIDManager::handleInputDetectionState Dispatching proximity event [0x0]
securityd[97] <Notice>: cert[1]: AnchorTrusted =(leaf)[force]> 0
neagent(Security)[799] <Notice>:  [root AnchorTrusted]
securityd[97] <Notice>: cert[0]: AnchorTrusted =(leaf)[force]> 0
neagent(Security)[799] <Notice>:  [leaf AnchorTrusted]
neagent(EAP8021X)[799] <Notice>: server certificate not trusted status 1001 -9807
neagent(NetworkExtension)[799] <Error>: Failed to process IKE Auth (EAP) packet
nesessionmanager(NetworkExtension)[398] <Notice>: NESMIKEv2VPNSession[Test:6309C3CA-6B4C-4878-9721-335BDBB82814]: status changed to disconnecting


Not sure why Preferences would have that SandboxViolation so it's looking like the server's trust chain is at fault?


Running /usr/bin/nscurl --ats-diagnostics against our CA returns PASS on all tests (from both macOS 10.12.3 and 10.12.4 machines). Also, again, this cert was working perfectly fine before iOS 10.3 (still working on un-updated devices) and it broke on macOS 10.12.4 as well.


What information can I add?

Thanks for your help so far, Eskimo. Not sure why my posts are being moderated but I'm adding this as a quick update until the more detailed post shows up. In short, a manual configuration via iOS Settings pane results in the same behavior. Hopefully my more-detailed post shows up soon but, until then, I'm not sure how to move forward and we're really in a pinch for a solution to this.


Edit for more details:

(Same devicelogs as my initial post but initated from Preferences and not our app.)



Not sure why Preferences would have that SandboxViolation so it's looking like the server's trust chain is at fault?


Running /usr/bin/nscurl --ats-diagnostics against our CA returns PASS on all tests (from both macOS 10.12.3 and 10.12.4 machines). Also, again, this cert was working perfectly fine before iOS 10.3 (still working on un-updated devices) and it broke on macOS 10.12.4 as well.


What information can I add?



Edit: Filed a bug report 31357710

Not sure why my posts are being moderated …

Yeah, DevForums’s spam filter can be a bit twitchy. In this context it doesn’t really slow things down because I can approve your posts.

Edit: Filed a bug report 31357710

Thanks for that.

After manually installing a matching config from Settings -> General -> VPN I still get a failure …

You really need to follow-up with AppleCare here. I can help you with API issues but the above makes it clear that this is not an API issue.

… so it's looking like the server's trust chain is at fault?

I don’t understand this well enough to comment on that statement. However, I will say that 10.3 does include a change with regards trust evaluation, namely, that if you install a custom CA’s root certificate on your device you must explicitly enable it in Settings > General > About > Certificate Trust Settings. You should definitely check that.

Share and Enjoy

Quinn “The Eskimo!”
Apple Developer Relations, Developer Technical Support, Core OS/Hardware

let myEmail = "eskimo" + "1" + "@apple.com"

We are experiencing the same issue in iOS 10.3.2 beta. Did you find a solution yet or is it something in iOS that changed?

Thanks!

The 10.3.2 beta issue is covered by this thread.

Share and Enjoy

Quinn “The Eskimo!”
Apple Developer Relations, Developer Technical Support, Core OS/Hardware

let myEmail = "eskimo" + "1" + "@apple.com"

Everyone.. IKEv2 simply does not work on IOS 10.3.2, for sure. Get an LT2P, probably your best option.

IKEv2 simply does not work on IOS 10.3.2, for sure. Get an LT2P, probably your best option.

This is not good advice. 10.3.2 hasn’t shipped yet, so saying that it “does not work” is somewhat premature. The correct response to problems with pre-release system software is to file a bug about the problem (and, if you do that, please post your bug number, just for the record).

Additionally, L2TP is not Apple’s preferred VPN transport, so encouraging folks to use that is poor form.

Share and Enjoy

Quinn “The Eskimo!”
Apple Developer Relations, Developer Technical Support, Core OS/Hardware

let myEmail = "eskimo" + "1" + "@apple.com"

Edit 2: Would the combination of (1) an on-demand profile and (2) the requirement for clients to authenticate via client certificates lead to the need for the com.apple.managed.vpn.shared requirement?

I recommend you start a new thread for your new question. It’s very hard to have a threaded discussion when you add new questions to older posts.

Share and Enjoy

Quinn “The Eskimo!”
Apple Developer Relations, Developer Technical Support, Core OS/Hardware

let myEmail = "eskimo" + "1" + "@apple.com"