How do I download data from web and pass it them to TableViewController?

As a newbie in the Swift programming language as well as programming for iOS, I am trying to make a very simple application.


I just want to create a different class who download the data from a website (parse and get what I want) and finally show it them to the user. Why does this in Swift seem so difficult? Is it or not?


In the end I have managed to download ( with URLSession.shared.dataTask ), parse ( with SwiftSoup ) and keep the data who I want in a swift class that I designed and is responsible for this process. So, I have my class "Announcements", in which I have implemented various methods to get what I want. This class works.


Below is the code of responsible class for downloading and parsing the data:


import Foundation
import SwiftSoup


class Announcements
{
    private var annTitles = [String]()

    // https://stackoverflow.com/questions/25407447/getting-data-from-a-website-in-swift
    // https://stackoverflow.com/questions/24016142/how-to-make-an-http-request-in-swift
    private func getHTML(aURL: String, completion: ( (String) -> (Void) )?)
    {
                let url = URL(string: aURL)!
                let task = URLSession.shared.dataTask(with: url)
                {
                    (data, response, error) in
                    guard let data = data else { return }
        //            print(String(data: data, encoding: .utf8)!)
                    let tempHTML = String(data: data, encoding: .utf8)!

                    completion?(tempHTML)
                }
                task.resume()
    }
    

      
    public func getAnnouncements()
    {
        self.getHTML(aURL: "https://www.my.site/news.php")
        { result in
            if result.isEmpty
            {
                print("Announcements could not be downloaded!")
            }
            else
            {
                print("Successful download of announcements.")
                self.parseHTML(html: result)
            }
            // Default
        }
    }

   

    private func parseHTML(html : String)
    {
        do
        {
            let doc: Document = try SwiftSoup.parse( html )
            let table: Elements = try doc.getElementsByClass( "table" )
           
            for tr in try! table.select("tr")[1...100]
            {
                let td : Elements = try tr.select("td")
                self.annTitles.append( try td[0].text() )
            }
//            self.viewData()
        }
        catch Exception.Error(let type, let message)
        {
            print(type)
            print(message)
        }
        catch
        {
            print("SwiftSoup parsing data error!")
        }
     
        print("All good!")
    }

   

    public func getTitles() -> [String]
    {
        return self.annTitles
    }

   

    public func viewData()
    {
        for  i in 0...99
        {
            print( self.annTitles[i] )
        }
    }


}


How can I call it and get the data it holds the above class in the main TableViewController?

I want a simple list on the user's screen to display a title in each cell.


Below is the code from (GUI) UITableViewController :


//
//  TableViewController.swift
//  uoiAnnouncements
//

import UIKit

class TableViewController: UITableViewController {

    var announcements : Announcements?
    
    override func viewDidLoad() {
        super.viewDidLoad()

        // Uncomment the following line to preserve selection between presentations
        // self.clearsSelectionOnViewWillAppear = false

        // Uncomment the following line to display an Edit button in the navigation bar for this view controller.
        // self.navigationItem.rightBarButtonItem = self.editButtonItem
        
        self.announcements = Announcements()
        self.announcements?.getAnnouncements()
        self.announcements?.viewData() 
    }



    // MARK: - Table view data source

//    override func numberOfSections(in tableView: UITableView) -> Int {
//        // #warning Incomplete implementation, return the number of sections
//        return 0
//    }



    override func tableView(_ tableView: UITableView, numberOfRowsInSection section: Int) -> Int {
        // #warning Incomplete implementation, return the number of rows
        return 100 // Const because only 100 are initially announcements.
    }


    
    override func tableView(_ tableView: UITableView, cellForRowAt indexPath: IndexPath) -> UITableViewCell {
        let cell = tableView.dequeueReusableCell(withIdentifier: "announcementCell", for: indexPath)

        // Configure the cell...
        self.reloadInputViews()

        cell.textLabel?.text = self.announcements?.getTitles()[indexPath.row] // <---- Here is the problem!
        //  ** Announcements not downloaded yet ** !


        
        return cell
    }


}


The problem is how downloading of announcements is an asynchronous process that doesn't run on the main thread - and I don't know when it ends -!

How can I tell the main "window"/screen/TableViewController to wait?

Can I know when the data has been downloaded?

Can I send a signal, put a semaphore or something?



Thank you very much for your time!

Replies

Generally that is a very open-ended question. It all depends. It sounds like you have found some 3rd party libraries to help you avoid having to learn any of those important details.


As far as loading the data goes, everything is asynchronous. You never, ever, wait for anything. Semaphores may be useful in some very specific contexts, but that is an advanced topic. You just issue your load request. When it completes, you (safely) update the data that your table is based upon. Then you reload the table (safely and in the main thread). If you wanted to be really clever and had lots of data to update, or only a little bit of changed data, you could (even more safely) update your data and (even more carefully and always in the main thread) insert/delete/update just the individual rows that have changed.

I know that this question is first general and theoretical and then practical. Personally I would like to discuss both parties and see in the end how it would be better to implement my code. Thanks for the time, the mood and the views/knowledge you share with us.


Personally I am interested in the details and knowledge. The only third-party library I use (if I'm not mistaken) is "SwiftSoup" for html parsing. Could I do some other parsing of the HTML code with pure Swift ? Of course with out regex.


Almost everything in mobile phones, is asynchronous. Ok.

I know, of course in this example the use of semaphores is wrong. It's not a distributed programming problem, but the "connection" of backend with the graphical user interaction environment. And I'm sure in iOS you have ways to implement it this.

What are the ways to do this on iOS ?


To make my whole class "Announcements" a thread ( extended ) ?

Play with the following code?


DispatchQueue.global(qos: .background).async
{
    print("This is run on the background queue")

    DispatchQueue.main.async 
    {
        print("This is run on the main queue, after the previous code in outer block")
    }
}


But the problem is that in Swift and in general to iOS programming ( in which I am still newbie and have not fully understood it ) that the online requests are all by default asynchronous!

For example, I've done something similar with Python 3 using the "urllib" library which does not create a parallel thread for the request, just waiting for the response to the main thread. Look at this very simple code :



try :
    allPage = urllib.request.urlopen('https://www.my.site/news.php')

except urllib.error.URLError :
    print("something goes wrong.... or raise an exception")
    raise

print(allPage)


Ok. Then, in use, I would create a parallel thread, where when the download was finished and I had the data in the data sets how I wanted, then I updated the graphical interface. So I think the main problem is that the "URLSession" create by default a parallel thread.


And I ask you, aren't 3 lines of code more beautiful than 13 or 23 in the "viewDidLoad" method on TableViewController class?

Because I have seen that most people execute the parallel request (URLSession) within the "viewDidLoad" method.


"You just issue your load request. When it completes, you (safely) update the data that your table is based upon.

Then you reload the table (safely and in the main thread)."


Because sometimes really talk is cheap. Can you show me the code for the above description (based on my code above)?



"If you wanted to be really clever and had lots of data to update, or only a little bit of changed data, you could (even more safely) update your data and (even more carefully and always in the main thread) insert/delete/update just the individual rows that have changed."

I like this approach and I also too thought it. But first I'd like to make my code in the simplest form above and then play with more fun stuff.



I would really like to see an experienced/professional and beautiful approach to this issue. For this I make my question here.

An experienced or professional on iOS cannot stand out the graphical interface from backend?

My object oriented approach is failure ?


I really want your views.

Thank you.

I'm afraid you are on your own with the HTML parsing. Is that your only source of data? At best HTML parsing is very difficult. In many/most cases, it is flat-out impossible. Many web sites today are dynamically generated with Javascript. You can't parse that. I have heard of some libraries (in Python) that will simulate an entire browser environment and allow you to scrape DOM content. Good luck with that.


Of couse 3 lines are going to be easier than 13 or 23. I don't know where you got those numbers from. A properly implemented solution to a data-driven table view would be more like 2300 lines. But you have to remember that a command-line Python script is doing nothing except waiting for data. End users don't like apps to lock up while they are busy waiting for data. What if the data never shows up at all? The app is locked up forever? And the python isn't doing a table view either. It's just more work.


Alas, you are going to have to figure it out for yourself.

I have no problem with parsing the HTML code. As I mentioned above, downloading the page and parsing it works smoothly and without any problems. I had done this from the beginning very easily ( thanks to SwiftSoup ). The problem was that I wanted to completely separate the code of this work from the user interface.

I just did not know how to check from the graphical interface (TableViewController), when the data has been downloaded and if it has been downloaded, to show it to the user.


Eventually I managed to do exactly what I wanted with the code below :


import Foundation  
import SwiftSoup  
  
  
class Announcements  
{  
    private var annTitles = [String]()  
  
    // https://stackoverflow.com/questions/25407447/getting-data-from-a-website-in-swift  
    // https://stackoverflow.com/questions/24016142/how-to-make-an-http-request-in-swift  
    private func getHTML(aURL: String, completion: ( (String) -> (Void) )?)  
    {  
                let url = URL(string: aURL)!  
                let task = URLSession.shared.dataTask(with: url)  
                {  
                    (data, response, error) in  
                    guard let data = data else { return }  
        //            print(String(data: data, encoding: .utf8)!)  
                    let tempHTML = String(data: data, encoding: .utf8)!  
  
                    completion?(tempHTML)  
                }  
                task.resume()  
    }  
      
  
        
    public func getAnnouncements(completion: @escaping (Bool?) -> Void)  
    {  
        self.getHTML(aURL: "https://www.my.site/news.php")  
        { result in  
            if result.isEmpty  
            {  
                debugPrint("Announcements could not be downloaded!")
                completion(false)
            }  
            else  
            {  
                debugPrint("Successful download of announcements.")  
                self.parseHTML(html: result)
                completion(true)
            }  
            // Default  
        }  
    }  
  
     
  
    private func parseHTML(html : String)  
    {  
        do  
        {  
            let doc: Document = try SwiftSoup.parse( html )  
            let table: Elements = try doc.getElementsByClass( "table" )  
             
            for tr in try! table.select("tr")[1...100]  
            {  
                let td : Elements = try tr.select("td")  
                self.annTitles.append( try td[0].text() )  
            }  
//            self.viewData()  
        }  
        catch Exception.Error(let type, let message)  
        {  
            print(type)  
            print(message)  
        }  
        catch  
        {  
            print("SwiftSoup parsing data error!")  
        }  
       
        print("All good!")  
    }  
  
     
  
    public func getTitles() -> [String]  
    {  
        return self.annTitles  
    }  
  
     
  
    public func viewData()  
    {  
        for  i in 0...99  
        {  
            print( self.annTitles[i] )  
        }  
    }  
  
  
} 




//  
//  TableViewController.swift  
//  uoiAnnouncements  
//  
  
import UIKit  
  
class TableViewController: UITableViewController {  
  
    @IBOutlet var announcementsTableView: UITableView! // Linked from UI.
    
    var announcements : Announcements?
    
    let announceCellIdentifier = "announcementCell"




    override func viewDidLoad() {  
        super.viewDidLoad()  
  
        // Uncomment the following line to preserve selection between presentations  
        // self.clearsSelectionOnViewWillAppear = false  
  
        // Uncomment the following line to display an Edit button in the navigation bar for this view controller.  
        // self.navigationItem.rightBarButtonItem = self.editButtonItem  
          
        self.announcements = Announcements()  
        self.announcements?.getAnnouncements()  
        { (announcements) in
            if announcements!
            {
                DispatchQueue.main.async
                {
                    print("Okay! I have them! Let's go!!")
                    self.announcementsTableView.reloadData()
                }
            }
        }
  
    }  
  
  
  
    // MARK: - Table view data source  
  
//    override func numberOfSections(in tableView: UITableView) -> Int {  
//        // #warning Incomplete implementation, return the number of sections  
//        return 0  
//    }  
  
  
  
    override func tableView(_ tableView: UITableView, numberOfRowsInSection section: Int) -> Int {  
        // #warning Incomplete implementation, return the number of rows  
        return 100 // Const because only 100 are initially announcements.  
    }  
  
  
      
    override func tableView(_ tableView: UITableView, cellForRowAt indexPath: IndexPath) -> UITableViewCell {  
        let cell = tableView.dequeueReusableCell(withIdentifier: announceCellIdentifier, for: indexPath)  
  
        // Configure the cell...  
        self.reloadInputViews()  
        
        // If they have not yet downloaded, the arrays are empty! ;)
        let titles = self.announcements?.getTitles()
        if titles!.isEmpty == false // So, Only if we have data..drop them on the user's screen!!
        {
            cell.textLabel?.text = titles?[indexPath.row]
        }
        return cell  
    }  
  
  
} 


In essence the real change is in the "getAnnouncements" method, which is now a completion function ( I pronounce it correctly? - so there are called these functions? ) and returns a boolean value when it is completed.


How do you view about my implementation?

Do you think my code is correct?


I am very happy because my application now works as I wanted from the beginning and the backed code is separated from UI.

That approach looks ok to me, if the model data can only change once when it’s initially loaded.


It’s a more general issue when the model data can change more than once, due to user actions in other parts of the app or external actions (maybe you publish a new announcement or something). So personally I usually use notifications (NotificationCenter class) for this sort of thing. In viewDidLoad I subscribe to add/update/delete notifications published by my model manager object, and insert/reload/remove table rows as needed in the table view as the model data changes.


Keeping the model data separate from the view, and understanding when it can change and what should happen when it does, is extremely important. You do seem to have that so you’re well on your way I think.