Simple tunnel build fails - NEAppProxyErrorDomain

Hi,


I downloaded the last version of SimpleTunnel.


I have Xcode 8.1.


When I try to build that project I receive 2 times the error "Use of undeclared type 'NEAppProxyErrorDomain'" in ClientAppProxyConnection file, in these lines:


func handleErrorCondition(_ flowError: NEAppProxyErrorDomain? = nil, notifyServer: Bool = true) {

func closeConnection(_ direction: TunnelConnectionCloseDirection, flowError: NEAppProxyErrorDomain?) {


I tried to clean the project and rebuild them several times, and I deleted the entire project and downloaded again, etc...


What can I do?


Regards,

Answered by OriolOMM in 195183022

I installed again Xcode 8.0 and works well, the problem is only with xcode 8.1.


I reported that bug.

Accepted Answer

I installed again Xcode 8.0 and works well, the problem is only with xcode 8.1.


I reported that bug.

I reported that bug.

What was the bug number?

Share and Enjoy

Quinn “The Eskimo!”
Apple Developer Relations, Developer Technical Support, Core OS/Hardware

let myEmail = "eskimo" + "1" + "@apple.com"

29217001

29217001

Thanks.

I’m not sure why the code compiled previously — I’d have to look at how the Xcode 8.0 imported these declarations and I don’t currently have Xcode 8.0 installed — but the fix for Xcode 8.1 is pretty straightforward: the

flowError
parameter of both
handleErrorCondition(_:notifyServer:)
and
closeConnection(_:flowError:)
should be of type
NEAppProxyFlowError.Code
.

Note This will just make the code compile. Realistically, this part of SimpleTunnel could do with a rework to move away from NSError and towards Swift’s

Error
protocol. And it looks like the way that the Network Extension framework errors are imported could be better, even in Swift 3.

Share and Enjoy

Quinn “The Eskimo!”
Apple Developer Relations, Developer Technical Support, Core OS/Hardware

let myEmail = "eskimo" + "1" + "@apple.com"

Hi, I just ended up here while trying to make the various SimpleTunnel targets build by updating to the latest Swift3 Syntax (I guess that the project

was updated in April, prior to the various DIspatch and UnsafeRowPointer API changes).

I though it may have been easier to continue on this post given the subject, but I am happy to open a new question if you think it is more appropriate.


I think I've managed to update most of the code but I'm currently stuck with a few cases on UnsafeRawPointer migration to call the socket C API (UDPServerConnection.swift in the tunnel_server target):


Previously this compiled:


let bytesRead = withUnsafeMutablePointer(&socketAddress) {

recvfrom(UDPSocket, UnsafeMutablePointer<Void>(response), response.count, 0, UnsafeMutablePointer($0), &socketAddressLength)

}


In the lastest Swift3 release, the necxt-to-last argemunet of the call to recvfrom,UnsafeMutablePointer($0), throws a compiler error. I have changed access to what it should represent the a pointer to the socket address using 'withMemoryReboudTo' and that brought me to replacing the above with:


let bytesRead = withUnsafeMutablePointer(to: &socketAddress) {

$0.withMemoryRebound(to: sockaddr_storage.self, capacity: 1, {

recvfrom(UDPSocket, UnsafeMutableRawPointer(mutating: response), response.count, 0, $0, &socketAddressLength)

})

}

But now it comes the tricky part: the compiler now warns me that it 'Cannot convert valure of type 'UnsafeMutablePointer<_> to expected argument type <UnsafeMutablePointer<sockaddr>!>''.

Effectively, the recvfrom interface expects a value of type UnsafeMutablePointer<sockaddr>! as the next to last parameter, whilst $0 is of type sockadd_storage. The same applies for other calls to sendTo where it complains about passing a sokaddr_in to an argument expecting type sockaddr.

As far as I can see, this was the same in the pre-update code above and it has never been an issue.

From my little understanding of both Swift-s RawPointer's and the C socket API ( oh well, that doesn't help but everybody has to start somwhere..) and extensive poking around, it looks like sockddr_storage is meant to fit sockaddr_in or sockaddr and the C functions would be accepting any of those struct to then typecast them to the correct type:

http://stackoverflow.com/questions/16010622/reasoning-behind-c-sockets-sockaddr-and-sockaddr-storage

As a consequence, it would have been prefectly fine to call those functions with a pointer to any of those structs as long as the function was able to typecast to the correct type as runtime.

Is it possible that the compiler moaning above is related to the fact that we cannot use the new 'typed' UnsafePointers as an argument to the

expected 'UNsafeMutablePointer<sockaddr>'?

Or am I completely misunderstanding the issue here and the problem lies in the way I've converted to code?

As a side note, have I missed a more up-to-date version of the sample code built with xCode 8.1 I could use / compare my chaanges to?


Thanks in advance

Calling BSD Sockets from Swift 3 is tricky because of the changes in SE-0107 UnsafeRawPointer API, which puts Swift’s type aliasing rules on a firm footing. I strongly recommend you read the UnsafeRawPointer Migration doc before going further. Specifically, it has a Socket API Helper that shows a good way to deal with this.

Share and Enjoy

Quinn “The Eskimo!”
Apple Developer Relations, Developer Technical Support, Core OS/Hardware

let myEmail = "eskimo" + "1" + "@apple.com"

Thanks Eskimo, I did go through the UnsafeWarPointer migration docs before tackling the conversion of the project to the latest Swift3, but I still cannot figure out the last remining issues. I will dig into your Socket API Helper as suggested before anything else though.

Thanks, will let you know how it goes.


Edit:

    static func fromSockAddr<ReturnType>(_ body: @escaping (_ sa: UnsafeMutablePointer<sockaddr>, _ saLen: inout socklen_t) throws -> ReturnType) rethrows -> (ReturnType, sockaddr_storage) {
        /
        var ss = sockaddr_storage()
        /
        var saLen = socklen_t(MemoryLayout<sockaddr_storage>.size)
        let result = try withUnsafePointer(to: &ss) {
            try $0.withMemoryRebound(to: sockaddr.self, capacity: 1) {
                try body($0, &saLen)
            }
        }
        return (result, ss)
    }


Apparently the helper function(s) suffer from the same compiler error I described above: in the example here, it fails to compile at line 09 with the usual Cannot convert value of type 'UnsafePointer<_>' to expected argument type 'UnsafeMutablePointer<sockaddr>'.

Any idea?

Ah, interesting. This code was broken by a post-Swift 3.0 change. To fix it, on line 6, replace

withUnsafePointer
with
withUnsafeMutablePointer
.

What happened here is that, in Swift 3.0, both the

withUnsafePointer(to:_:)
and
withUnsafeMutablePointer(to:_:)
routines called the supplied closure with a mutable pointer. This obviously makes no sense; only the second should be a mutable pointer, while the first should obviously be a immutable pointer. We fixed this in at some point, but my socket helpers were relying on the previous incorrect behaviour.

Share and Enjoy

Quinn “The Eskimo!”
Apple Developer Relations, Developer Technical Support, Core OS/Hardware

let myEmail = "eskimo" + "1" + "@apple.com"

Thanks, that explains it! I manged to make it build and run, now fighting with entitlements and provisioning profiles for the sample app but that's a different level of fun 🙂.


Edit: after a few hours spent with figthing the XCode's "The executable was fund with invalid entitlements", I have noticed this post related to missing com.apple.managed.vpn.shared entitlements when generating the Netwrok Extension entitlements via the website: https://forums.developer.apple.com/thread/67894.

Likely to be connected to my issue and therefore shall I wait for an answer on that before banging my head aagainst the wall any further?


Further on this, I did follow your Debugging Entitlement Issues and my provisioning profile entitlements seems to make sense and be consistent with the entitlements listed in the .app and .appex products of the template.


Provisioning profile's entitlements:


  <key>Entitlements</key>
  <dict>
      <key>com.apple.developer.networking.networkextension</key>
      <array>
            <string>app-proxy-provider</string>
            <string>content-filter-provider</string>
            <string>packet-tunnel-provider</string>
            </array>
      <key>keychain-access-groups</key>
      <array>
            <string>$MyTeamID$.*</string>
      </array>
      <key>get-task-allow</key>
      <true/>
      <key>application-identifier</key>
      <string>$MyTeamID$.com.manni.SimpleTunnel</string>
      <key>com.apple.developer.team-identifier</key>
      <string>$MyTeamID$</string>
      <key>com.apple.developer.networking.vpn.api</key>
      <array>
            <string>allow-vpn</string>
      </array>
  </dict>


and when I run codesign -dentitlements SimpleTunnel.app:


<?xml version="1.0" encoding="UTF-8"?>
<!DOCTYPE plist PUBLIC "-/
<plist version="1.0">
<dict>
  <key>application-identifier</key>
  <string>$MyTeamID$.com.manni.SimpleTunnel</string>
  <key>com.apple.developer.networking.networkextension</key>
  <array>
  <string>packet-tunnel-provider</string>
  <string>app-proxy-provider</string>
  <string>content-filter-provider</string>
  </array>
  <key>com.apple.developer.networking.vpn.api</key>
  <array>
       <string>allow-vpn</string>
  </array>
  <key>com.apple.developer.team-identifier</key>
  <string>$MyTeamID$</string>
  <key>get-task-allow</key>
  <true/>
  <key>keychain-access-groups</key>
  <array>
       <string>$MyTeamID$.com.apple.managed.vpn.shared</string>
  </array>
</dict>
</plist>

You shouldn’t need the

com.apple.managed.vpn.shared
keychain access group to get a VPN provider up and limping. It’s only necessary if you want to support managed VPN, where the device manager installs a client identity as part of a configuration profile such that your VPN provider can access it.
<string>$MyTeamID$.com.apple.managed.vpn.shared</string>

This is definitely not correct. The whole point of the

com.apple.managed.vpn.shared
keychain access group is that it’s not prefixed by your Team ID, and thus it’s shared by all VPN providers and the configuration profile machinery.

Pasted in below in a dump of the entitlements and provisioning profiles for my testbed VPN app. I don’t have time right now to compare it with what you posted, but I figured your might find it helpful.

Share and Enjoy

Quinn “The Eskimo!”
Apple Developer Relations, Developer Technical Support, Core OS/Hardware

let myEmail = "eskimo" + "1" + "@apple.com"
$ codesign -d --entitlements :- QNEPacketTunnel.app
Executable=…/QNEPacketTunnel.app/QNEPacketTunnel
<?xml version="1.0" encoding="UTF-8"?>
<!DOCTYPE plist PUBLIC "-//Apple//DTD PLIST 1.0//EN" "http://www.apple.com/DTDs/PropertyList-1.0.dtd">
<plist version="1.0">
<dict>
    <key>application-identifier</key>
    <string>VR9NTVC6BB.com.example.apple-samplecode.QNE-iOS.PacketTunnel</string>
    <key>com.apple.developer.networking.HotspotHelper</key>
    <true/>
    <key>com.apple.developer.networking.networkextension</key>
    <array>
        <string>packet-tunnel-provider</string>
        <string>app-proxy-provider</string>
        <string>content-filter-provider</string>
    </array>
    <key>com.apple.developer.team-identifier</key>
    <string>VR9NTVC6BB</string>
    <key>com.apple.security.application-groups</key>
    <array>
        <string>group.com.example.apple-samplecode.QNE-iOS</string>
    </array>
    <key>get-task-allow</key>
    <true/>
</dict>
</plist>
$ security cms -D -i QNEPacketTunnel.app/embedded.mobileprovision
<?xml version="1.0" encoding="UTF-8"?>
<!DOCTYPE plist PUBLIC "-//Apple//DTD PLIST 1.0//EN" "http://www.apple.com/DTDs/PropertyList-1.0.dtd">
<plist version="1.0">
<dict>
    <key>AppIDName</key>
    <string>QNE iOS PacketTunnel</string>
    <key>ApplicationIdentifierPrefix</key>
    <array>
    <string>VR9NTVC6BB</string>
    </array>
    <key>CreationDate</key>
    <date>2016-11-10T14:52:24Z</date>
    <key>Platform</key>
    <array>
        <string>iOS</string>
    </array>
    <key>DeveloperCertificates</key>
    <array>
        <data>…</data>
    </array>
    <key>Entitlements</key>
    <dict>
        <key>keychain-access-groups</key>
        <array>
            <string>VR9NTVC6BB.*</string>     
            <string>com.apple.managed.vpn.shared</string>
        </array>
        <key>get-task-allow</key>
        <true/>
        <key>application-identifier</key>
        <string>VR9NTVC6BB.com.example.apple-samplecode.QNE-iOS.PacketTunnel</string>
        <key>com.apple.security.application-groups</key>
        <array>
            <string>group.com.example.apple-samplecode.QNE-iOS</string>
        </array>
        <key>com.apple.developer.team-identifier</key>
        <string>VR9NTVC6BB</string>
        <key>com.apple.developer.networking.networkextension</key>
        <array>
            <string>packet-tunnel-provider</string>
            <string>app-proxy-provider</string>
            <string>content-filter-provider</string>
        </array>
        <key>com.apple.developer.networking.HotspotHelper</key>
        <true/>
    </dict>
    <key>ExpirationDate</key>
    <date>2017-11-10T14:52:24Z</date>
    <key>Name</key>
    <string>QNE iOS PacketTunnel</string>
    <key>ProvisionedDevices</key>
    <array>
        <string>…</string>
    </array>
    <key>TeamIdentifier</key>
    <array>
        <string>VR9NTVC6BB</string>
    </array>
    <key>TeamName</key>
    <string>MyTeamName</string>
    <key>TimeToLive</key>
    <integer>365</integer>
    <key>UUID</key>
    <string>cd82ad10-817d-4625-9af4-db637ac9194c</string>
    <key>Version</key>
    <integer>1</integer>
</dict>
</plist>
$ codesign -d --entitlements :- QNEPacketTunnel.app/PlugIns/QNEPacketTunnelProvider.appex
Executable=…/QNEPacketTunnel.app/PlugIns/QNEPacketTunnelProvider.appex/QNEPacketTunnelProvider
<?xml version="1.0" encoding="UTF-8"?>
<!DOCTYPE plist PUBLIC "-//Apple//DTD PLIST 1.0//EN" "http://www.apple.com/DTDs/PropertyList-1.0.dtd">
<plist version="1.0">
<dict>
    <key>application-identifier</key>
    <string>VR9NTVC6BB.com.example.apple-samplecode.QNE-iOS.PacketTunnel.Provider</string>
    <key>com.apple.developer.networking.HotspotHelper</key>
    <true/>
    <key>com.apple.developer.networking.networkextension</key>
    <array>
        <string>packet-tunnel-provider</string>
        <string>app-proxy-provider</string>
        <string>content-filter-provider</string>
    </array>
    <key>com.apple.developer.team-identifier</key>
    <string>VR9NTVC6BB</string>
    <key>com.apple.security.application-groups</key>
    <array>
        <string>group.com.example.apple-samplecode.QNE-iOS</string>
    </array>
    <key>get-task-allow</key>
    <true/>
</dict>
</plist>
$ security cms -D -i QNEPacketTunnel.app/PlugIns/QNEPacketTunnelProvider.appex/embedded.mobileprovision
<?xml version="1.0" encoding="UTF-8"?>
<!DOCTYPE plist PUBLIC "-//Apple//DTD PLIST 1.0//EN" "http://www.apple.com/DTDs/PropertyList-1.0.dtd">
<plist version="1.0">
<dict>
    <key>AppIDName</key>
    <string>QNE iOS PacketTunnel Provider</string>
    <key>ApplicationIdentifierPrefix</key>
    <array>
    <string>VR9NTVC6BB</string>
    </array>
    <key>CreationDate</key>
    <date>2016-11-10T14:52:52Z</date>
    <key>Platform</key>
    <array>
        <string>iOS</string>
    </array>
    <key>DeveloperCertificates</key>
    <array>
        <data>…</data>
    </array>
    <key>Entitlements</key>
    <dict>
        <key>keychain-access-groups</key>
        <array>
            <string>VR9NTVC6BB.*</string>     
            <string>com.apple.managed.vpn.shared</string>
        </array>
        <key>get-task-allow</key>
        <true/>
        <key>application-identifier</key>
        <string>VR9NTVC6BB.com.example.apple-samplecode.QNE-iOS.PacketTunnel.Provider</string>
        <key>com.apple.security.application-groups</key>
        <array>
            <string>group.com.example.apple-samplecode.QNE-iOS</string>
        </array>
        <key>com.apple.developer.team-identifier</key>
        <string>VR9NTVC6BB</string>
        <key>com.apple.developer.networking.networkextension</key>
        <array>
            <string>packet-tunnel-provider</string>
            <string>app-proxy-provider</string>
            <string>content-filter-provider</string>
        </array>
        <key>com.apple.developer.networking.HotspotHelper</key>
        <true/>
    </dict>
    <key>ExpirationDate</key>
    <date>2017-11-10T14:52:52Z</date>
    <key>Name</key>
    <string>QNE iOS PacketTunnel Provider</string>
    <key>ProvisionedDevices</key>
    <array>
        <string>…</string>
    </array>
    <key>TeamIdentifier</key>
    <array>
        <string>VR9NTVC6BB</string>
    </array>
    <key>TeamName</key>
    <string>MyTeamName</string>
    <key>TimeToLive</key>
    <integer>365</integer>
    <key>UUID</key>
    <string>4e5e0ff8-cef6-4078-9223-15115673212a</string>
    <key>Version</key>
    <integer>1</integer>
</dict>
</plist>

Interesting. Apart from fixing the

com.apple.managed.vpn.shared
keychain issue you outlined, I've double checked your dump with mines and there is a difference that seems a good hint.

The network extension entitlements below are present in my product's entitlement (codesign -dentitlements SimpleTunnel.app), in the embedded provisioning profile (security cms -D -i SimpleTunnel.app/embedded.mobileprovision), in the PacketTunnel.appex's entitlements in the PlugIns dir (codesign -d --entitlements :- SimpleTunnel.app/PlugIns/PacketTunnel.appex) but NOT in the latter's embedded provisioning profile.


        <key>com.apple.developer.networking.networkextension</key>
        <array>
            <string>packet-tunnel-provider</string>
            <string>app-proxy-provider</string>
            <string>content-filter-provider</string>


That is to say, I don't get the above in the embedded provisioning profile when I run your last step, security cms -D -i SimpleTunnel.app/PlugIns/PacketTunnel.appex/embedded.mobileprovision.

Any idea you may have is as alwaysgreatly appreciated, and thanks for your continuous assistance!

All fixed, resolution below in case it may help anyone else with similar problems.


Disregard the edit in my post above: I must have been half asleep but today I have noticed that the problem was not a difference in the embedded provisioning profile between the copied extensions in the PlugIns dir and the ones in the original build locations.

All of my app extensions where not signed with the correct provisioning profile, so that pointed towards an issue in my Certificated and Profiles: I originally let XCode create the AppIds for all of the extension automatically, and I discovered that they did not have the Netwrok Extensions entitlements on by default. After turning that on, I generated an explicit Provisioning Profile for each extension's AppIDs and used that to sign each target in XCode (disabling therefore the 'Automatically manage signing' option). Et voila, the app is running.


P.s: pretty sure the guys at Apple are alreaedy working on updating the sample code to work with XCode 8.1 and the laster Swift 3, but in case it may be helpful I have a version that compiles here (although may benefit from a review).


Thanks again,

Matteo

Hi Eskimo, a piece of advice, if you have some time before Christmas comes and brings us joy...or tunneling desperation.


I've managed to build and run the SimpleTunnel project in XCode 8.1 after sorting out the conversion to the latest Wift syntax and the provisioning profiles issues you've kindy helped with.

I can now run the server (and telnet into its IP address:port without any issue), run and debug the PacketTunnel extension which exchanges its "Hello message" with the app, estabilishes a tunnel with the server (server acceps the new connection)...and then nothing happens after the client sends a "connection open" message to the server. The ViewController's status is stuck on "Connecting", and no further communication / state change seems to happen between the client and the server across the tunnel (which is probably the reason why the label remains on "Connecting" as I assume that we are waitin for a server response to the "connection open" message to consider the connection effectively working).


I am wondering whether this may be caused by a bug in my code updates for Swift 3 (I've had and fixed cases of the extension crashing silently on slighly mismatching signatures before, eg: Completion blocks expecting NSError? and being passed and Error? and similar), or if there's anything you could point me towards, for instance some common configuration scenario you may have ancountered before with the project showing these symptoms.


Updated:

Looking at the eschange of TCP control packets in WireShark, when I toggle the VPN slider I can see the following:


[SYN] | Tunnel Client (iPhone) -> Tunnel Server (Mac)

[SYN-ACK] | Server -> Client

[ACK] | Client -> Server


... at this point the TCP connection is estabilished, as reflected by connection!.state in the debugger..but the status label stays on "Connecting".

The next packet is the Windows size update one:


[TCP Windows Update] | Server -> Client

At thsi point, the client sends the "Open" command through the channel and we see the request to send data coming thorugh and acknowledged:


[PSH-ACK] | Client -> Server

[ACK] | Server -> Client


...unfortunately, this is were it all ends. No more communication, data sent etc.. and the status text still left at "Connecting".


EDIT: I managed to route the traffic through the tunnel to the server in the end. I am using the "old" (Swift 2.x) code for the Project, built via XCode 8.2 with "Use lgacy Swift" enabled. That only required a few minor tweaks to update to Swift 2.3 and it works as expected. Still have to fix some nat configuration as I the server does not seem to be able to route the DNS response back to the tunnel interface (and therefore Safari on my iPhone cannot browse), but at least I can see the traffic running through.

Do you a link to the source that compiles? Still the code online has not been updated to the latest Xcode to build correctly. Thanks Matt

Sorry for hijacking this thread. I am fighting with SimpleTunnel sample as well.

I have managed to make it build and run. I did not do any other (config.plist, Entitlements, whatever) changes. Do I need to change the servers config.plist? Is there anything more I have to do? Create and save certificate or such?

The issue I am heaving is that when switching the "VPN" swith on to enable it:

  • targetManager.connection.startVPNTunnel() is called and does not throw
  • NSNotification.Name.NEVPNStatusDidChange informs us new state is "connecting"
  • NSNotification.Name.NEVPNStatusDidChange informs us new state is "disconnected"

The tunnel_server is running on my MacBook. SimpleTunnel.app on iPhone (device).

Thanks and sorry again!

Simple tunnel build fails - NEAppProxyErrorDomain
 
 
Q