Multi-label text classification in coreML

I've wondered the same and, as a result of exploration, conclude that one has to create separate models for each label type (e.g. sentiment, subject) and then run separate predictions. If I needed a single output (array of labels) I'd probably use Combine to collect the results and notify the recipient (UI?) when all are ready.

Regards, Michaela

Dear Michaela,

Thanks for your answer.

What I would implement is a multi classification with the same label type. Something like:

Training data:

(some text1):[label1, label2, label3, label4]

(some text2):[label1, label3]

(some text3):[label2, label3]

Output (max 2 labels):

(some textX) >> [label3, label4]

What do you think? Thanks in advance,

M

Hello Manuel,

For the same label type, I don't see a method in MLTextClassifier https://developer.apple.com/documentation/createml/mltextclassifier for associating an array of labels for a given set of text (string). So, presumably, one would have to repeat the text in the training data with each label, i.e. some text1 : label1, some text1: label2, etc

For output, MLTextClassifier typically predicts one label for each text string, although there is a method predictionsWithConfidence(from: for predicting multiple possible labels and their confidence scores.

CreateML and CoreML are all about identifying patterns, not looking up knowledge databases. So, if I were classifying a business by name alone, I might know that "The Jolly Swagman" is a restaurant, but there's nothing in the text that would enable such a prediction to be made. We therefore need to provide sufficient text for patterns to be evident and to be careful when manually classifying training data so as not to use knowledge, nor impose bias.

I might do some more exploring and experimenting in the next few days if I get the time.

Best wishes, Michaela

UPDATE: The documentation (referred to above) is confusing. To make predictions, we have to initialise an NLModel with an MLTextClassifier then call the predictedLabel or predictedLabelHypotheses method. Here's my (partial) code:

// Load previously trained and saved category (label) model
func loadCategoryModel() {
        let modelURL = docsURL!.appendingPathComponent("CategoryModel.mlmodelc")
        let modelConfig = MLModelConfiguration()
        modelConfig.parameters = [:]
        MLModel.load(contentsOf: modelURL, configuration: modelConfig) { result in
            switch result {
            case let .success(model):
                DispatchQueue.main.async {
                    self.mlModel = model
                }
                self.getlatestData()
            case let .failure(error):
                print("ERROR: loading model failed \(error)")
            }
        }
    }
func predictCategory(_ venue: String) -> String {
// trying to predict category of an entity from the venue name (usually business name) - demo code
        do {
            let categoryPredictor = try NLModel(mlModel: mlModel!)
            let predictedLabel = categoryPredictor.predictedLabel(for: venue) ?? "n/a"
            let possibleLabels = categoryPredictor.predictedLabelHypotheses(for: venue, maximumCount: 2)
            print(venue,"Predictions: ",predictedLabel,possibleLabels)
// prints ==> "Costco Warehouse Predictions:  General Store ["Pharmacy": 0.17649094106070892, "General Store": 0.2769075597746391]"
            return predictedLabel
        } catch {
                print("ERROR: prediction failed")
                return "unknown"
        }
    }

So, for your needs, you could train your data with multiple labels (as per earlier suggestion) then call predictedLabelHypotheses and check the probabilities provided, using those outputs (labels) that pass whatever threshold you determine.

Regards, Michaela

Dear Michaela,

Thanks a lot for your feedback.

I will try the "predictionsWithConfidence(from:" approach because, as you wrote, I need a multi-label classification.

The feature I would implement is a function that suggests to me relevant tags to apply while I'm writing a text.

So I give to the model as training data some text blocks that I've manually tagged and when I write a new text block similar to a previously existing one the system suggests to me similar tags.

I would as well updating continuously the model at runtime adding the newly added tagged text blocks to the training set in order to gradually improve the quality of the suggestions.

Best,

M