Downloading binary file from web returns nothing

I am trying to download a binary file from an http website.


func saveFile(from links:[String])
{
        print("CHECK START METHOD")
     
        if (!links.isEmpty)
        {
            print("CHECK START DOWNLOAD ",links[0])
         
            let url : NSURL! = NSURL(string: links[0])    /
         
            let request : NSURLRequest = URLRequest(url: url as URL) as NSURLRequest
            let session = URLSession.shared
         
            var dataReceived = [UInt8]()
                    
            let task = session.dataTask(with: request as URLRequest) {(data, response, error) -> Void in
             
                if (data != nil)
                {
                    dataReceived = [UInt8](data!)
                    print("CHECK 1 URL: ",dataReceived.count)
                }
                else
                {
                    print("ERROR: No data returned")
                }
             
                DispatchQueue.main.async
                    {
                        print("COMPLETION")
                }
            }
            task.resume()
        }

        else
        {
            // Deal with the error
        }
    }


What is weird is that I do not get either " CHECK 1 URL: " or "No data returned" printed. Neither do I get into the main thread. I cannot check whether I downloaded data. Although I can see in the console:

CHECK START METHOD

CHECK START DOWNLOAD


and the link to the URL, which I can download using regular Internet browser (so the link is correct). Funny enough, the above code seems to work when dowloading a text file but not binary data.


Am I doing something wrong?


Thanks a lot!

Accepted Reply

I am currently testing it using a Unit Test, so at this moment I, at least, want it to let me know if the file has been downloaded. Therefore I wonder, why I do not even see the console printing "COMPLETION" when downloading has finished.

I’m not 100% sure but problems like this usually result from this sequence:

  1. saveFile(from:)
    starts the request, which runs asynchronously
  2. saveFile(from:)
    returns to the unit test code
  3. The unit test code returns to the unit test infrastructure

  4. The process running the unit tests terminates, preventing the async request that was started in step 1 from every completing

If you’re unit testing async stuff you should learn about XCTest’s async support; see the Writing Tests of Asynchronous Operations section of Testing with Xcode for details.

Share and Enjoy

Quinn “The Eskimo!”
Apple Developer Relations, Developer Technical Support, Core OS/Hardware

let myEmail = "eskimo" + "1" + "@apple.com"

Replies

What is the file type configuration for your site?


What happens if you zip the binary file and try to download that file, instead?


What does the webserver log have to say?

Whatever else you do, you should rework how you handle the parameters to your completion handler. Here’s a typical example:

if let error = error {
    … `error` now contains the transport error
} else {
    let response = response as! HTTPURLResponse
    let data = data!
    if response.statusCode != 200 {
        … server-side error …
    } else if response.mimeType != "application/json" {
        … always a good idea to check this …
    } else {
        … now you can look at `data` …
    }
}

Specifically:

  • You must not look at

    response
    or
    data
    until you’ve check
    error
    .
  • A transport error means that NSURLSession was unable to send the request to, or receive the response from, the server.

  • A server-side error means that the request made it to the server, something went wrong on the server, and a response came back describing that problem.

  • It’s a good idea to check the MIME type of the response to handle cases where the request got processed by the wrong server. This can happen with various middleboxen, captive networks, firewalls, ‘smart’ gateways, and so on. This is especially important when using HTTP rather than HTTPS.

ps This code is unnecessarily convoluted:

let url : NSURL! = NSURL(string: links[0])
let request : NSURLRequest = URLRequest(url: url as URL) as NSURLRequest

You could just have this:

let url = URL(string: links[0])!
let request = URLRequest(url: url)
…
let task = session.dataTask(with: request) …

Right now you’re bouncing between Foundation types (NSURL and so on) and Swift Foundation types (

URL
), which is just extra code that add no value.

Share and Enjoy

Quinn “The Eskimo!”
Apple Developer Relations, Developer Technical Support, Core OS/Hardware

let myEmail = "eskimo" + "1" + "@apple.com"

Thank you, KMT and Eskimo. I have uploaded the file to my website and changed the code as per the Eskimo's email.


func saveFile(from links:[String]) -> Int
    {
        print("CHECK START METHOD")
        var finalResult = 1
        if (!links.isEmpty)
        {
            print("CHECK START DOWNLOAD")
            let url = URL(string: address)
            if url == nil
            {
                print("URL not created")
                finalResult = 2
            }
            let request = URLRequest(url: url!)
            let session = URLSession.shared
            var dataReceived = [UInt8]()
            let task = session.dataTask(with: request) {(data, response, error) -> Void in
  
                if let error = error
                {
                    print("Transport error")
                    finalResult = 2
                }
                else
                {
                    let response = response as! HTTPURLResponse
                    let data = data!
              
                    if response.statusCode != 200
                    {
                        print("Server-side error")
                        finalResult = 2
                    }
                    else
                    {
                        dataReceived = [UInt8](data)
                        print("CHECK URL DATA COUNT: ",data.count)
                        print("CHECK URL DATARECEIVED COUNT: ",dataReceived.count)
                        finalResult = 0
                    }
                }
  
                DispatchQueue.main.async
                    {
                        print("COMPLETION")
                }
            }
            task.resume()
        }
        else
        {
            finalResult = 2
            print("ERROR1")
        }
        return finalResult;
    }


Neither do I understand why it does not download the file, nor do I get why I receive no errors. Just this:

CHECK START METHOD

CHECK START DOWNLOAD

FINAL RESULT: 1

This is getting insanely complicated...

I have no idea how to attach a link to the website where the file is located. The forum does not allow me to do it.

For blacklisted urls, you have to break them...like this:


h t t p : / / cnn.com

The code you posted won’t compile by itself because line 8 references

address
. If you fix that I should be able to offer more concrete input.

The code also shows that you’re still having problems with the async nature of networking.

saveFile(from:)
can’t possibly return 0 because your network request runs asynchronously. So,
saveFile(from:)
runs, starts the network request, and then returns the current value of
finalResult
(if things go well this would be 1). Later on, when the network request completes, your completion handler runs and sets
finalResult
to 0 (or 2), but no one can ‘see’ that value because
saveFile(from:)
has already returned.

How you fix this depends on the context in which your code is running. If you’re downloading a file to display in your app, you would typically display a loading indicator until the download has finished, at which point you’d transition to displaying the contents of the file. You make that transition inside the code you run on the main thread (line 45 in your latest snippet).

Share and Enjoy

Quinn “The Eskimo!”
Apple Developer Relations, Developer Technical Support, Core OS/Hardware

let myEmail = "eskimo" + "1" + "@apple.com"

Eskimo. Sorry if I did not make it explicit. The address was given in my previous post (a link to a website).


The intention is to implement the code by downloading the file and then passing the result to another class using a delegate in line 45.


DispatchQueue.main.async
{
    let dataProcessed = doStuffWithData(data)
    delegate.doOtherStuff(dataProcessed)
}


I am currently testing it using a Unit Test, so at this moment I, at least, want it to let me know if the file has been downloaded. Therefore I wonder, why I do not even see the console printing "COMPLETION" when downloading has finished. Isn't it supposed to? And why can't I see my output in lines 37 and 38? Should it be more just:


let myObject = MyClass()
myObject.saveFile(myArray)


to call the method and see the console printing?


Thank you so much for bearing with me!

I am currently testing it using a Unit Test, so at this moment I, at least, want it to let me know if the file has been downloaded. Therefore I wonder, why I do not even see the console printing "COMPLETION" when downloading has finished.

I’m not 100% sure but problems like this usually result from this sequence:

  1. saveFile(from:)
    starts the request, which runs asynchronously
  2. saveFile(from:)
    returns to the unit test code
  3. The unit test code returns to the unit test infrastructure

  4. The process running the unit tests terminates, preventing the async request that was started in step 1 from every completing

If you’re unit testing async stuff you should learn about XCTest’s async support; see the Writing Tests of Asynchronous Operations section of Testing with Xcode for details.

Share and Enjoy

Quinn “The Eskimo!”
Apple Developer Relations, Developer Technical Support, Core OS/Hardware

let myEmail = "eskimo" + "1" + "@apple.com"

Thank you. After all, it was a problem with Unit Test. Ran it in an application and works as a charm. Thanks a lot to everybody!

Eskimo and KMT. May I ask one more thing, please? The code works fine. However, I have an array of different web links. What I want to do is to iterate through these links, and if the first one is a valid link and contains the necessary data (which I am checking by identifying the size of the binary file), I want to extract the data, pass it to my delegate and break the iteration. Instead, the code iterates through all links and extracts data for each of them. If any of the links are not valid, it iterates through the links but does not extract data or pass it to the delegate even if one of them is good. This is my code:


for link in links
{
   let url = URL(string: link)
   let request = URLRequest(url: url!)
   let session = URLSession.shared
               
   delegate?.startActivityIndicator()
               
   var dataReceived = [UInt8]()

   let task = session.dataTask(with: request) {(data, response, error) -> Void in
     if error != nil
     {
       print("request transport error")
     }
     else
     {
       let response = response as! HTTPURLResponse
       let data = data!
                       
       noerror: if response.statusCode == 200
       {
         dataReceived = [UInt8](data)
                           
         if dataReceived.count > 100
         {
           break noerror
         }
       }
       else
       {
         print("request server-side error")
       }
    }
                   
    DispatchQueue.main.async
    {
      self.delegate?.stopActivityIndicator()
      // Process data and pass to the delegate
    }
 }
 task.resume()
}


I wonder now if this is even possible with asynchronous downloading.


Thank you so much!

This is where I bring up zipped files, again. Especially with async d/ls.


If your live-download processing is brittle, delaying such activity so you can alternatively deal with it as a zip may provide a more stable result. I'm not saying this is a sure cure that specifically applies to your case, just that you may want to try it and see if it can act as another arrow in your quiver.

I wonder now if this is even possible with asynchronous downloading.

It’s definitely possible, but you have to introduce some shared state. The basic outline would be something like this:

class DownloadEngine {

    let urls: [URL]

    init(urls: [URL]) {
        self.urls = urls
    }

    init(urls: [String]) {
        self.urls = urls.map { URL(string: $0)! }
    }

    func start() {
        self.startNext()
    }

    private func startNext() {
        guard self.urlIndex != self.urls.count else {
            NSLog("report failure")
            return
        }
        let url = urls[self.urlIndex]
        self.urlIndex += 1
        NSLog("start %@", "\(url)")
        DispatchQueue.main.asyncAfter(deadline: .now() + 2.0) {
            guard true /* error */ else {
                NSLog("report failure")
                return
            }
            guard true /* response is acceptable */ else {
                NSLog("report success")
                return
            }
            self.startNext()
        }
    }

    private var urlIndex = 0
}

If you run it like this:

self.downloadEngine = DownloadEngine(urls: ["http://foo.example.com", "http://bar.example.com", "http://baz.example.com"])
self.downloadEngine.start()

you’ll see output like this:

2017-02-03 09:41:42.710 xxsi[7206:351235] start http://foo.example.com
2017-02-03 09:41:44.711 xxsi[7206:351235] start http://bar.example.com
2017-02-03 09:41:46.897 xxsi[7206:351235] start http://baz.example.com
2017-02-03 09:41:49.096 xxsi[7206:351235] report failure

There’s two things to note here:

  • I’m using

    asyncAfter(…)
    to simulate an async network request
  • I’m targeting the main queue in order to keep everything serialised

IMPORTANT This last point is critical. Because

DownloadEngine
holds shared state, you have to ensure that access to that state is serialised. In my case I’m using the main queue for that. If I were doing real networking, I’d probably create a serial
OperationQueue
and then set up my
URLSession
to use that as its delegate queue.

Share and Enjoy

Quinn “The Eskimo!”
Apple Developer Relations, Developer Technical Support, Core OS/Hardware

let myEmail = "eskimo" + "1" + "@apple.com"

Thank you, Eskimo. What do you think of something like this?


var listOfLinks = [String]()
var queue = OperationQueue()
private var urlIndex = 0


init()
{
        self.listOfLinks = [String](repeating: "", count: 3)
        self.listOfLinks[0] = "badHttpLink"
        self.listOfLinks[1] = "goodHttpLink"
        self.listOfLinks[2] = "badHttpLink"
        saveFile()
}


func saveFile()
    {
        if (!self.listOfLinks.isEmpty)
        {
            let urls = self.listOfLinks.map { URL(string: $0)! }

            queue = OperationQueue()
    
            guard self.urlIndex != urls.count else
            {
                NSLog("report failure")
                return
            }
    
            let url = urls[self.urlIndex]
            self.urlIndex += 1
    
            queue.addOperation { () -> Void in

                let request = URLRequest(url: url)
                let session = URLSession.shared
        
                var dataReceived = [UInt8]()
        
                let task = session.dataTask(with: request) {(data, response, error) -> Void in
            
                    if error != nil
                    {
                        print("request transport error")
                    }
                    else
                    {
                        let response = response as! HTTPURLResponse
                        let data = data!
                
                        if response.statusCode == 200
                        {
                            dataReceived = [UInt8](data)
                        }
                        else
                        {
                            print("request server-side error")
                        }
                    }
            
                    OperationQueue.main.addOperation(
                        {
                            if dataReceived.count <= 1000
                            {
                                self.saveFile()
                            }
                            else
                            {
                               // Other methods to process data
                            }
                        }
                    )
                }
                task.resume()
            }
        }
        else
        {
            print("Links to the NOAA server do not exist")
        }
    }


It seems to be working. However, I am wondering whether I am missing any other hidden flaws and it could be further improved somehow.


Many thanks for your highly valuable support!

You seems to be setting up two queues, one global one (line 2) and another local one (line 22). I recommend you use one global one.

When creation an

OperationQueue
, if you want the work to be serialised, you have to set the
maxConcurrentOperationCount
property to 1.

It’s a good idea to give your queue a name, which shows up in the debugging, crash reports, and so on.

You should create your own session for these requests. As part of doing that you can supply your queue to the session, which will cause the session to run completion handlers and delegate callbacks on that queue, making them serialised with respect to other code you run on that queue.

Share and Enjoy

Quinn “The Eskimo!”
Apple Developer Relations, Developer Technical Support, Core OS/Hardware

let myEmail = "eskimo" + "1" + "@apple.com"