17 Replies
      Latest reply: Feb 5, 2017 2:10 PM by eskimo RSS
      igorland Level 1 Level 1 (0 points)

        I am trying to download a binary file from an http website.

         

        func saveFile(from links:[String])
        {
                print("CHECK START METHOD")
             
                if (!links.isEmpty)
                {
                    print("CHECK START DOWNLOAD ",links[0])
                 
                    let url : NSURL! = NSURL(string: links[0])    /
                 
                    let request : NSURLRequest = URLRequest(url: url as URL) as NSURLRequest
                    let session = URLSession.shared
                 
                    var dataReceived = [UInt8]()
                            
                    let task = session.dataTask(with: request as URLRequest) {(data, response, error) -> Void in
                     
                        if (data != nil)
                        {
                            dataReceived = [UInt8](data!)
                            print("CHECK 1 URL: ",dataReceived.count)
                        }
                        else
                        {
                            print("ERROR: No data returned")
                        }
                     
                        DispatchQueue.main.async
                            {
                                print("COMPLETION")
                        }
                    }
                    task.resume()
                }
        
                else
                {
                    // Deal with the error
                }
            }
        

         

        What is weird is that I do not get either " CHECK 1 URL: " or "No data returned" printed. Neither do I get into the main thread. I cannot check whether I downloaded data. Although I can see in the console:


        CHECK START METHOD

        CHECK START DOWNLOAD

         

        and the link to the URL, which I can download using regular Internet browser (so the link is correct).  Funny enough, the above code seems to work when dowloading a text file but not binary data.

         

        Am I doing something wrong?

         

        Thanks a lot!

        • Re: Downloading binary file from web returns nothing
          KMT Level 8 Level 8 (7,310 points)

          What is the file type configuration for your site?

           

          What happens if you zip the binary file and try to download that file, instead?

           

          What does the webserver log have to say?

          • Re: Downloading binary file from web returns nothing
            eskimo Apple Staff Apple Staff (6,250 points)

            Whatever else you do, you should rework how you handle the parameters to your completion handler.  Here’s a typical example:

            if let error = error {
                … `error` now contains the transport error
            } else {
                let response = response as! HTTPURLResponse
                let data = data!
                if response.statusCode != 200 {
                    … server-side error …
                } else if response.mimeType != "application/json" {
                    … always a good idea to check this …
                } else {
                    … now you can look at `data` …
                }
            }
            

            Specifically:

            • You must not look at response or data until you’ve check error.

            • A transport error means that NSURLSession was unable to send the request to, or receive the response from, the server.

            • A server-side error means that the request made it to the server, something went wrong on the server, and a response came back describing that problem.

            • It’s a good idea to check the MIME type of the response to handle cases where the request got processed by the wrong server.  This can happen with various middleboxen, captive networks, firewalls, ‘smart’ gateways, and so on.  This is especially important when using HTTP rather than HTTPS.

            ps This code is unnecessarily convoluted:

            let url : NSURL! = NSURL(string: links[0])
            let request : NSURLRequest = URLRequest(url: url as URL) as NSURLRequest
            

            You could just have this:

            let url = URL(string: links[0])!
            let request = URLRequest(url: url)
            …
            let task = session.dataTask(with: request) …
            

            Right now you’re bouncing between Foundation types (NSURL and so on) and Swift Foundation types (URL), which is just extra code that add no value.

            Share and Enjoy

            Quinn “The Eskimo!”
            Apple Developer Relations, Developer Technical Support, Core OS/Hardware
            let myEmail = "eskimo" + "1" + "@apple.com"

              • Re: Downloading binary file from web returns nothing
                igorland Level 1 Level 1 (0 points)

                Thank you, KMT and Eskimo. I have uploaded the file to my website and changed the code as per the Eskimo's email.

                 

                func saveFile(from links:[String]) -> Int
                    {
                        print("CHECK START METHOD")
                        var finalResult = 1
                        if (!links.isEmpty)
                        {
                            print("CHECK START DOWNLOAD")
                            let url = URL(string: address)
                            if url == nil
                            {
                                print("URL not created")
                                finalResult = 2
                            }
                            let request = URLRequest(url: url!)
                            let session = URLSession.shared
                            var dataReceived = [UInt8]()
                            let task = session.dataTask(with: request) {(data, response, error) -> Void in
                  
                                if let error = error
                                {
                                    print("Transport error")
                                    finalResult = 2
                                }
                                else
                                {
                                    let response = response as! HTTPURLResponse
                                    let data = data!
                              
                                    if response.statusCode != 200
                                    {
                                        print("Server-side error")
                                        finalResult = 2
                                    }
                                    else
                                    {
                                        dataReceived = [UInt8](data)
                                        print("CHECK URL DATA COUNT: ",data.count)
                                        print("CHECK URL DATARECEIVED COUNT: ",dataReceived.count)
                                        finalResult = 0
                                    }
                                }
                  
                                DispatchQueue.main.async
                                    {
                                        print("COMPLETION")
                                }
                            }
                            task.resume()
                        }
                        else
                        {
                            finalResult = 2
                            print("ERROR1")
                        }
                        return finalResult;
                    }
                

                 

                Neither do I understand why it does not download the file, nor do I get why I receive no errors. Just this:

                CHECK START METHOD

                CHECK START DOWNLOAD

                FINAL RESULT:  1

                This is getting insanely complicated...

                  • Re: Downloading binary file from web returns nothing
                    igorland Level 1 Level 1 (0 points)

                    I have no idea how to attach a link to the website where the file is located. The forum does not allow me to do it.

                    • Re: Downloading binary file from web returns nothing
                      eskimo Apple Staff Apple Staff (6,250 points)

                      The code you posted won’t compile by itself because line 8 references address.  If you fix that I should be able to offer more concrete input.

                      The code also shows that you’re still having problems with the async nature of networking.  saveFile(from:) can’t possibly return 0 because your network request runs asynchronously.  So, saveFile(from:) runs, starts the network request, and then returns the current value of finalResult (if things go well this would be 1).  Later on, when the network request completes, your completion handler runs and sets finalResult to 0 (or 2), but no one can ‘see’ that value because saveFile(from:) has already returned.

                      How you fix this depends on the context in which your code is running.  If you’re downloading a file to display in your app, you would typically display a loading indicator until the download has finished, at which point you’d transition to displaying the contents of the file.  You make that transition inside the code you run on the main thread (line 45 in your latest snippet).

                      Share and Enjoy

                      Quinn “The Eskimo!”
                      Apple Developer Relations, Developer Technical Support, Core OS/Hardware
                      let myEmail = "eskimo" + "1" + "@apple.com"

                        • Re: Downloading binary file from web returns nothing
                          igorland Level 1 Level 1 (0 points)

                          Eskimo. Sorry if I did not make it explicit. The address was given in my previous post (a link to a website).

                           

                          The intention is to implement the code by downloading the file and then passing the result to another class using a delegate in line 45.

                           

                          DispatchQueue.main.async
                          {
                              let dataProcessed = doStuffWithData(data)
                              delegate.doOtherStuff(dataProcessed)
                          }
                          

                           

                          I am currently testing it using a Unit Test, so at this moment I, at least, want it to let me know if the file has been downloaded. Therefore I wonder, why I do not even see the console printing "COMPLETION" when downloading has finished. Isn't it supposed to? And why can't I see my output in lines 37 and 38? Should it be more just:

                           

                          let myObject = MyClass()
                          myObject.saveFile(myArray)
                          

                           

                          to call the method and see the console printing?

                           

                          Thank you so much for bearing with me!

                            • Re: Downloading binary file from web returns nothing
                              eskimo Apple Staff Apple Staff (6,250 points)

                              I am currently testing it using a Unit Test, so at this moment I, at least, want it to let me know if the file has been downloaded. Therefore I wonder, why I do not even see the console printing "COMPLETION" when downloading has finished.

                              I’m not 100% sure but problems like this usually result from this sequence:

                              1. saveFile(from:) starts the request, which runs asynchronously

                              2. saveFile(from:) returns to the unit test code

                              3. The unit test code returns to the unit test infrastructure

                              4. The process running the unit tests terminates, preventing the async request that was started in step 1 from every completing

                              If you’re unit testing async stuff you should learn about XCTest’s async support; see the Writing Tests of Asynchronous Operations section of Testing with Xcode for details.

                              Share and Enjoy

                              Quinn “The Eskimo!”
                              Apple Developer Relations, Developer Technical Support, Core OS/Hardware
                              let myEmail = "eskimo" + "1" + "@apple.com"

                                • Re: Downloading binary file from web returns nothing
                                  igorland Level 1 Level 1 (0 points)

                                  Thank you. After all, it was a problem with Unit Test. Ran it in an application and works as a charm. Thanks a lot to everybody!

                                  • Re: Downloading binary file from web returns nothing
                                    igorland Level 1 Level 1 (0 points)

                                    Eskimo and KMT. May I ask one more thing, please? The code works fine. However, I have an array of different web links. What I want to do is to iterate through these links, and if the first one is a valid link and contains the necessary data (which I am checking by identifying the size of the binary file), I want to extract the data, pass it to my delegate and break the iteration. Instead, the code iterates through all links and extracts data for each of them. If any of the links are not valid, it iterates through the links but does not extract data or pass it to the delegate even if one of them is good. This is my code:

                                     

                                    for link in links
                                    {
                                       let url = URL(string: link)
                                       let request = URLRequest(url: url!)
                                       let session = URLSession.shared
                                                   
                                       delegate?.startActivityIndicator()
                                                   
                                       var dataReceived = [UInt8]()
                                    
                                       let task = session.dataTask(with: request) {(data, response, error) -> Void in
                                         if error != nil
                                         {
                                           print("request transport error")
                                         }
                                         else
                                         {
                                           let response = response as! HTTPURLResponse
                                           let data = data!
                                                           
                                           noerror: if response.statusCode == 200
                                           {
                                             dataReceived = [UInt8](data)
                                                               
                                             if dataReceived.count > 100
                                             {
                                               break noerror
                                             }
                                           }
                                           else
                                           {
                                             print("request server-side error")
                                           }
                                        }
                                                       
                                        DispatchQueue.main.async
                                        {
                                          self.delegate?.stopActivityIndicator()
                                          // Process data and pass to the delegate
                                        }
                                     }
                                     task.resume()
                                    }
                                    

                                     

                                    I wonder now if this is even possible with asynchronous downloading.

                                     

                                    Thank you so much!

                                      • Re: Downloading binary file from web returns nothing
                                        KMT Level 8 Level 8 (7,310 points)

                                        This is where I bring up zipped files, again. Especially with async d/ls.

                                         

                                        If your live-download processing is brittle, delaying such activity so you can alternatively deal with it as a zip may provide a more stable result. I'm not saying this is a sure cure that specifically applies to your case, just that you may want to try it and see if it can act as another arrow in your quiver.

                                        • Re: Downloading binary file from web returns nothing
                                          eskimo Apple Staff Apple Staff (6,250 points)

                                          I wonder now if this is even possible with asynchronous downloading.

                                          It’s definitely possible, but you have to introduce some shared state.  The basic outline would be something like this:

                                          class DownloadEngine {
                                          
                                              let urls: [URL]
                                          
                                              init(urls: [URL]) {
                                                  self.urls = urls
                                              }
                                          
                                              init(urls: [String]) {
                                                  self.urls = urls.map { URL(string: $0)! }
                                              }
                                          
                                              func start() {
                                                  self.startNext()
                                              }
                                          
                                              private func startNext() {
                                                  guard self.urlIndex != self.urls.count else {
                                                      NSLog("report failure")
                                                      return
                                                  }
                                                  let url = urls[self.urlIndex]
                                                  self.urlIndex += 1
                                                  NSLog("start %@", "\(url)")
                                                  DispatchQueue.main.asyncAfter(deadline: .now() + 2.0) {
                                                      guard true /* error */ else {
                                                          NSLog("report failure")
                                                          return
                                                      }
                                                      guard true /* response is acceptable */ else {
                                                          NSLog("report success")
                                                          return
                                                      }
                                                      self.startNext()
                                                  }
                                              }
                                          
                                              private var urlIndex = 0
                                          }
                                          

                                          If you run it like this:

                                          self.downloadEngine = DownloadEngine(urls: ["http://foo.example.com", "http://bar.example.com", "http://baz.example.com"])
                                          self.downloadEngine.start()
                                          

                                          you’ll see output like this:

                                          2017-02-03 09:41:42.710 xxsi[7206:351235] start http://foo.example.com
                                          2017-02-03 09:41:44.711 xxsi[7206:351235] start http://bar.example.com
                                          2017-02-03 09:41:46.897 xxsi[7206:351235] start http://baz.example.com
                                          2017-02-03 09:41:49.096 xxsi[7206:351235] report failure
                                          

                                          There’s two things to note here:

                                          • I’m using asyncAfter(…) to simulate an async network request

                                          • I’m targeting the main queue in order to keep everything serialised

                                          IMPORTANT This last point is critical.  Because DownloadEngine holds shared state, you have to ensure that access to that state is serialised.  In my case I’m using the main queue for that.  If I were doing real networking, I’d probably create a serial OperationQueue and then set up my URLSession to use that as its delegate queue.

                                          Share and Enjoy

                                          Quinn “The Eskimo!”
                                          Apple Developer Relations, Developer Technical Support, Core OS/Hardware
                                          let myEmail = "eskimo" + "1" + "@apple.com"

                                            • Re: Downloading binary file from web returns nothing
                                              igorland Level 1 Level 1 (0 points)

                                              Thank you, Eskimo. What do you think of something like this?

                                               

                                              var listOfLinks = [String]()
                                              var queue = OperationQueue()
                                              private var urlIndex = 0
                                              
                                              
                                              init()
                                              {
                                                      self.listOfLinks = [String](repeating: "", count: 3)
                                                      self.listOfLinks[0] = "badHttpLink"
                                                      self.listOfLinks[1] = "goodHttpLink"
                                                      self.listOfLinks[2] = "badHttpLink"
                                                      saveFile()
                                              }
                                              
                                              
                                              func saveFile()
                                                  {
                                                      if (!self.listOfLinks.isEmpty)
                                                      {
                                                          let urls = self.listOfLinks.map { URL(string: $0)! }
                                              
                                                          queue = OperationQueue()
                                                  
                                                          guard self.urlIndex != urls.count else
                                                          {
                                                              NSLog("report failure")
                                                              return
                                                          }
                                                  
                                                          let url = urls[self.urlIndex]
                                                          self.urlIndex += 1
                                                  
                                                          queue.addOperation { () -> Void in
                                              
                                                              let request = URLRequest(url: url)
                                                              let session = URLSession.shared
                                                      
                                                              var dataReceived = [UInt8]()
                                                      
                                                              let task = session.dataTask(with: request) {(data, response, error) -> Void in
                                                          
                                                                  if error != nil
                                                                  {
                                                                      print("request transport error")
                                                                  }
                                                                  else
                                                                  {
                                                                      let response = response as! HTTPURLResponse
                                                                      let data = data!
                                                              
                                                                      if response.statusCode == 200
                                                                      {
                                                                          dataReceived = [UInt8](data)
                                                                      }
                                                                      else
                                                                      {
                                                                          print("request server-side error")
                                                                      }
                                                                  }
                                                          
                                                                  OperationQueue.main.addOperation(
                                                                      {
                                                                          if dataReceived.count <= 1000
                                                                          {
                                                                              self.saveFile()
                                                                          }
                                                                          else
                                                                          {
                                                                             // Other methods to process data
                                                                          }
                                                                      }
                                                                  )
                                                              }
                                                              task.resume()
                                                          }
                                                      }
                                                      else
                                                      {
                                                          print("Links to the NOAA server do not exist")
                                                      }
                                                  }
                                              
                                              
                                              
                                              
                                              

                                               

                                              It seems to be working. However, I am wondering whether I am missing any other hidden flaws and it could be further improved somehow.

                                               

                                              Many thanks for your highly valuable support!

                                                • Re: Downloading binary file from web returns nothing
                                                  eskimo Apple Staff Apple Staff (6,250 points)

                                                  You seems to be setting up two queues, one global one (line 2) and another local one (line 22).  I recommend you use one global one.

                                                  When creation an OperationQueue, if you want the work to be serialised, you have to set the maxConcurrentOperationCount property to 1.

                                                  It’s a good idea to give your queue a name, which shows up in the debugging, crash reports, and so on.

                                                  You should create your own session for these requests.  As part of doing that you can supply your queue to the session, which will cause the session to run completion handlers and delegate callbacks on that queue, making them serialised with respect to other code you run on that queue.

                                                  Share and Enjoy

                                                  Quinn “The Eskimo!”
                                                  Apple Developer Relations, Developer Technical Support, Core OS/Hardware
                                                  let myEmail = "eskimo" + "1" + "@apple.com"