10 Replies
      Latest reply on Sep 4, 2018 11:47 PM by eskimo
      Ainassine Level 1 Level 1 (0 points)

        Hello,

        Recently, for my study at school, I need to develop an app which could tag each data flow(TCP or UDP) with which app it comes from. In that case we can get a <data flow, app name> tuple, and this could do a lot help for our study.

        I'm wondering that is there any good suggestions to make that possible? Does NetworkExtension know which app the data flow come from when it handle with the data flows? I've googled and searched for answers, but the result are not satisfying.

        I've noticed that on my iPhone8 with iOS 11, the System offers us the information of how much cellular data each app has used. Will this API be opened to us or is there any alternative methods to implement that function?

         

        Thank you if you could help me!

        • Re: Does NetworkExtension know which app the data flow comes from?
          eskimo Apple Staff Apple Staff (9,845 points)

          Does NetworkExtension know which app the data flow come from when it handle with the data flows?

          Yes, at least in some cases.  This information is available when you enable per-app VPN, which you can do in two ways:

          • Using an app proxy provider — In this case the identity of the app is available in the metaData property of the NEAppProxyFlow.

          • Using a packet tunnel provider in per-app VPN mode — In this case the metadata is per packet, accessible via the metadata property of NEPacket.

          Be aware that there are significant restrictions on setting up per-app VPN.  You can find a summary of these in this post.

          Share and Enjoy

          Quinn “The Eskimo!”
          Apple Developer Relations, Developer Technical Support, Core OS/Hardware
          let myEmail = "eskimo" + "1" + "@apple.com"

          • Re: Does NetworkExtension know which app the data flow comes from?
            eskimo Apple Staff Apple Staff (9,845 points)

            Oh, one other thing to note is that an RVI packet trace includes app information in the packet metadata, so you can get this information without writing any code on iOS.  See QA1176 Getting a Packet Trace for information on RVI.

            Share and Enjoy

            Quinn “The Eskimo!”
            Apple Developer Relations, Developer Technical Support, Core OS/Hardware
            let myEmail = "eskimo" + "1" + "@apple.com"

              • Re: Does NetworkExtension know which app the data flow comes from?
                Ainassine Level 1 Level 1 (0 points)

                Thank you for your reply, and it really helps a lot. Here I have a new question. It seems that i need to have a MDM server to enable per-app VPN for my app, if I want to release it at app store? Am i right? BTW, for the best result, we hope that our app is able to generate a log of the network flows on devices having our app installed, and the most important thing is we have the app name each flow blengs to noted in our log. Looking forward to your reply.

                  • Re: Does NetworkExtension know which app the data flow comes from?
                    eskimo Apple Staff Apple Staff (9,845 points)

                    The exact requirements for per-app VPN are complex, which is why I pointed you to this post which explains them in detail.  However, to address your specific question:

                    It seems that i need to have a MDM server to enable per-app VPN for my app, if I want to release it at app store? Am i right?

                    If you are:

                    • Working on iOS

                    • Signing your app for something other than Development (that is, Distribution, TestFlight, In-House (Enterprise))

                    then, yes, you can only set up per-app VPN via MDM.

                    Share and Enjoy

                    Quinn “The Eskimo!”
                    Apple Developer Relations, Developer Technical Support, Core OS/Hardware
                    let myEmail = "eskimo" + "1" + "@apple.com"

                      • Re: Does NetworkExtension know which app the data flow comes from?
                        Ainassine Level 1 Level 1 (0 points)

                        Your answer helps me a lot. Thank you!

                        Now I'm using sample code SimpleTunnel to learn about how to use app-proxy. I've finished doing the following items:

                             1. Compiling and installing SimpleTunnel on my iPhone and tunnel_server on my Macbook. Both of them are running well and I can connect my iPhone to my Macbook though SimpleTunnel.

                             2. Installing a configuration profile with "com.apple.vpn.managed.applayer" on my iPhone, if you want to check the configuration profile I can paste it later.

                        The question comes that when I add a NETestingAPP rule into SimpleTunnel's info.plist, in which I write like this:

                        ```

                          <key>NETestAppMapping</key>

                          <dict>

                            <key>83A8720B-2BD5-4B7E-921D-3D9E312B4ACD</key>

                            <array>

                              <string>com.tencent.xin</string>

                            </array>

                          </dict>

                        ```

                        Through this rule, I want that app could send and receive data flows through SimpleTunnel, but that app just lost connection to Internet after I rerun SimpleTunnel.

                        Should I do some other things to make connection available?

                         

                        Looking forward to your repley!

                        • Re: Does NetworkExtension know which app the data flow comes from?
                          Ainassine Level 1 Level 1 (0 points)

                          Hey,

                           

                          And while studying SimpleTunnel, I found my Xcode debug console displays too little debug information. SimpleTunnelLog() and print() called in extensions are not recorded at the debug console. I tried to use console.app on my Mac, but it's not that convenient. Any suggestions about that?

                           

                          Yours,

                          Ainassine.

                            • Re: Does NetworkExtension know which app the data flow comes from?
                              eskimo Apple Staff Apple Staff (9,845 points)

                              This is no longer a problem.

                              Glad to hear it.

                              SimpleTunnelLog and print called in extensions are not recorded at the debug console.

                              Right.  This is because Xcode has troubles capturing stdout from processes that you attach to.  I recommend that you adopt os_log and then use the Mac’s Console app.  By adopting os_log you can set a subsystem and category, which allows for easier filtering.

                              Share and Enjoy

                              Quinn “The Eskimo!”
                              Apple Developer Relations, Developer Technical Support, Core OS/Hardware
                              let myEmail = "eskimo" + "1" + "@apple.com"

                                • Re: Does NetworkExtension know which app the data flow comes from?
                                  Ainassine Level 1 Level 1 (0 points)

                                  Hi,

                                  eskimo,

                                   

                                  Sorry to bother you again, and thank you for you advice.

                                  Thanks to your help, I've finished a large part of my project. I'm using an AppProxy together with a PacketTunnel to handle with my target app's network flow. In my AppProxy, I could figure out which app the network flow comes from according to NEAppProxyFlow's metaData property, and I could find that in the next step, AppProxy has to make it clear whether this NEAppProxyFlow is a TCP flow or a UDP flow. After this step, my question comes.

                                  I have to get as much information of network flows as possible, in which source host ip and port, destination host ip and port should be contained. For NEAppProxyTCPFlow, I could get its remoteEndpoint, while for NEAppProxyUDPFlow, I could get its localEndpoint, which means I cannot get src and dst host information for a flow at the same time. Is there any solutions for me to get remoteEndpoint and localEndpoint information simultaneously?

                                  And another question, is there any solutions to figure out what protocol is used in NEAppProxyFlow? HTTP or SSH for example.

                                   

                                  Looking forward to your reply!

                                   

                                  Yours sincerely,

                                  JQ.

                                    • Re: Does NetworkExtension know which app the data flow comes from?
                                      eskimo Apple Staff Apple Staff (9,845 points)

                                      OK, ‘easy’ question first:

                                      And another question, is there any solutions to figure out what protocol is used in NEAppProxyFlow? HTTP or SSH for example.

                                      There’s two approaches you can use for this:

                                      • You can use the remote port number as a rough guide (HTTP on port 80, SSH on port 22, and so on).  This isn’t definitive because it’s not uncommon for folks to run protocols on the ‘wrong’ port (HTTP on port 8080, for example).

                                      • Beyond that you’ll have to ‘sniff’ the traffic on the flow to see what protocol it looks like.  This is easy for HTTP and SSH, and not so easy for various other protocols.  Specifically, all protocols within TLS, like HTTPS and IMAPS, look alike on the wire.


                                      For NEAppProxyTCPFlow, I could get its remoteEndpoint, while for NEAppProxyUDPFlow, I could get its localEndpoint, which means I cannot get src and dst host information for a flow at the same time.

                                      You can to consider each protocol separately:

                                      • For TCP, the system only assigns a local address to the TCP connection once the connection is in place.  If you wait for some traffic to flow over the connection, does localEndpoint then start returning a value?

                                      • For UDP, it’s most common for a UDP flow to not be connected, that is, for the flow to allow outgoing packets to any endpoint.  The specific endpoint is then set on a packet-by-packet basis.

                                        UDP does have the notion of connected sockets, that is, you can call a connect on a UDP socket to fix the destination endpoint.  I don’t know if that’s reflected at the app proxy layer; you would have to create a small test project that does that and then see if that causes remoteEndpoint to be populated.

                                      Share and Enjoy

                                      Quinn “The Eskimo!”
                                      Apple Developer Relations, Developer Technical Support, Core OS/Hardware
                                      let myEmail = "eskimo" + "1" + "@apple.com"