11 Replies
      Latest reply on Jul 18, 2019 9:18 AM by etata
      Robby Level 1 Level 1 (0 points)

        I am creating a model from an MLDataTable as such:

         

                    if let dataTable = try? MLDataTable(dictionary: priorityData) {

                        let (trainingCSVData, testCSVData) = dataTable.randomSplit(by: 0.8, seed: 0)

                        print( "Training Data = " + trainingCSVData.description)

                      

                        do {

                            let deleteProbabilityClassifier = try MLClassifier(trainingData: dataTable, targetColumn: "DeleteProbability")

                            // evaluate it

                            let metrics = deleteProbabilityClassifier.evaluation(on: testCSVData)

                          

                            let modelMetadata = MLModelMetadata(author: "Me", shortDescription: "Model for DeleteProbability Prediction", version: "1.1")

                            let modelPath = homeDirURL.appendingPathComponent(name + "-deleteProbabilityPredictor.mlmodel", isDirectory: false)

                            do {

                                try deleteProbabilityClassifier.write(to: modelPath, metadata: modelMetadata)

                                print("Success - DeleteProbability Model written" + modelPath.absoluteString )

                                print(metrics)

                                return true

                            }

                            catch {

                                print("Failure to write DeleteProbabilityClassifier - " + modelPath.absoluteString)

                            }

                        }

                        catch {

                            print("ML Classifier failed")

                            print("ML Classsifier threw error = " + error.localizedDescription)

                        }

                    }

                    else {

                        print("Failed - MLDataTable ")

                        print(type(of: priorityData))

                    }

         

        What do I need to do to update this code to generate an updatable model?  Currently I am running this code within a playground.  I do not see any where to set the isUpdatable to true here?

         

        Any help would be greatly appreciated.

         

        Thanks,

        Rob

        • Re: Is it possible to create and updatable model in code?
          kerfuffle Level 3 Level 3 (110 points)

          This does not appear to be possible with the Create ML APIs currently.

           

          However, you can load the mlmodel in a Python script and use coremltools to make the model updatable.

           

          If you really want to do this from within Swift, it is still possible but you'll need to use the protobuf API to directly load the mlmodel definition. There is a chapter in my book Core ML Survival Guide that explains how to modify mlmodel files from Swift using this method.

            • Re: Is it possible to create and updatable model in code?
              Robby Level 1 Level 1 (0 points)

              Thanks very much for your quick reply.  I hope I do not come off as sounding like an ad for your book, but I did purchase the book.

               

              Could you point me to where in the book I should look to use swift to convert?

               

              I have a swift playground which currently pulls from an external SQL DB to build the model and would like to be able to update that playground to build the updatable model.

               

              Thanks again for all the help and I wish you well with the book.

                • Re: Is it possible to create and updatable model in code?
                  kerfuffle Level 3 Level 3 (110 points)

                  There is a chapter in the advanced section named "Using Protobuf Without coremltools". It shows how to read the mlmodel file in Swift, make changes, and save the result as a new mlmodel file.

                   

                  The book currently doesn't have any info on making models updatable -- I'm still writing those chapters. But you can check out the examples in the coremltoolsrepo that show how to make models updatable.

                   

                  What you'll need to do is slightly different from what's shown in the coremltools examples, because it involves writing protobuf objects by hand. But the idea is the same. Please also read the chapters on the mlmodel file format and "Using the Spec to Edit Models".

                   

                  You'll have to go pretty low-level for this, but it's definitely possible to do.

                   

                    • Re: Is it possible to create and updatable model in code?
                      Robby Level 1 Level 1 (0 points)

                      Thanks, I will look into the "Using Protobuf Without coremltools" section and the coremltools examples and do not mind any low level programming.

                       

                      Would you recommend a different approach if I already have a playground to create the models?  Maybe there is a simple python script to modify existung models to be updatable?

                       

                      I would think many people would be looking to make their models updatable with iOS 13 coming and doing so should become regular practice.

                       

                      Thanks again for all the help and the quick answers.

                        • Re: Is it possible to create and updatable model in code?
                          kerfuffle Level 3 Level 3 (110 points)

                          It depends what your goal is. If you want to do everything from the playground (or a macOS app) then modifying the protobuf from Swift is the right approach.

                           

                          But if you don't mind running a Python script after doing training, then that's the much easier approach. See here: h t t p s ://github.com/apple/coremltools/tree/master/examples/updatable_models

                            • Re: Is it possible to create and updatable model in code?
                              Robby Level 1 Level 1 (0 points)

                              Thanks the updatable model examples look to be compiled python (I'm more iOS ObjC/Swift proficient) and I can not seem to see what is going on inside.

                               

                              I did go and try to make a model updatable as such:

                               

                                      if let url = Bundle.main.url(forResource: "Predictor", withExtension: "bin"), let data = try? Data(contentsOf: url),

                                          var model = try? CoreML_Specification_Model(serializedData: data) {

                                          /* do other stuff with model */

                                          print(model.description_p)

                                          print(model.pipelineClassifier)

                                          model.isUpdatable = true

                                          print(model.description_p)

                                          print(model.pipelineClassifier)

                                          if let data = try? model.serializedData() {

                                              try? data.write(to: url)

                                          }

                                      }

                               

                              When I rename and pull in that model, I get the following error:

                               

                              validator error: Last model in an updatable pipeline model should be marked as updatable.

                               

                              I will go and dig into this error, but maybe there is a tutorial as to how to make a model updatable?  Any links or sugestions would help.

                               

                              Thanks,

                              Rob

                                • Re: Is it possible to create and updatable model in code?
                                  Robby Level 1 Level 1 (0 points)

                                  I fixed the problem arrising from "validator error: Last model in an updatable pipeline model should be marked as updatable."

                                   

                                  And now XCode gives me a new error:

                                   

                                  validator error: This model type is not supported for on-device update.

                                   

                                  Sound like this error is unsurmountable.  Is there a list of supported types for on-device update?  Possibly I could rework the model to be one of these types, or is it possible that as of a later XCode beta this type will be supported for on-device update?

                                   

                                  Thanks again.

                                   

                                  Here's the code which got me thus far:

                                   

                                          if let url = Bundle.main.url(forResource: "predictor", withExtension: "bin"), let data = try? Data(contentsOf: url),

                                              var model = try? CoreML_Specification_Model(serializedData: data) {

                                              /* do other stuff with model */

                                              model.isUpdatable = true

                                              for i in 0..<model.pipelineClassifier.pipeline.models.count {

                                                  model.pipelineClassifier.pipeline.models[i].isUpdatable = true

                                              }

                                              if let data = try? model.serializedData() {

                                                  let writableUrl = URL.init(fileURLWithPath: "/Users/Me/Documents/predictor.mlmodel")

                                                  try? data.write(to: writableUrl)

                                              }

                                          }

                                    • Re: Is it possible to create and updatable model in code?
                                      kerfuffle Level 3 Level 3 (110 points)

                                      Oh duh, I could have saved you some trouble. Only neural networks and k-NN models can be made updatable (also when they are in a pipeline). Since you trained an MLClassifier, I guess it made a GLM model or something, which is not updatable indeed. I doubt they'll add that to Core ML 3 before the final release.

                                        • Re: Is it possible to create and updatable model in code?
                                          Robby Level 1 Level 1 (0 points)

                                          Thanks for the help.  After looking through the Core ML headers, I found the following list of updatable model types:

                                           

                                            /// Following model types support on-device update:

                                            ///

                                            /// - NeuralNetworkClassifier

                                            /// - NeuralNetworkRegressor

                                            /// - NeuralNetwork

                                            /// - KNearestNeighborsClassifier

                                           

                                          Since the MLClassifier did not create any of these, I need to use a different method to create the model.

                                           

                                          I believe it would be best if I could keep using the playground I have which pulls data from a remote SQL DB and creates the model.  So, is there anything similiar to the MLClassifier which I can use in Swift to create such a model?

                                           

                                          Thanks for all the help and the 2 books.  I'll keep reading but there is a lot to digest.  Any short cut or pointers would be appreciated.

                                            • Re: Is it possible to create and updatable model in code?
                                              kerfuffle Level 3 Level 3 (110 points)

                                              It depends on your use case, but if you wanted to train this kind of classifier in Swift and use it as a Core ML model, then I would start with the model that Create ML trained, read it into Swift using the protobuf API, write your own code for training it (which is relatively straightforward for this kind of model), then put the weights back into the mlmodel file and save it to disk.

                                               

                                              If doing it in Swift isn't the primary concern, I'd probably use scikit-learn to train the model and then coremltools to convert it to Core ML.

                                                • Re: Is it possible to create and updatable model in code?
                                                  etata Level 1 Level 1 (0 points)

                                                  Has this worked for you? I tried creating a KNeighborsClassifier using scikit and then convert it to an updatable MLmodel but it doesn't looks like the converter can make it updatable. I noticed the keras coremltools converted has a "respect trainable" parameters that enables on device training for the MLmodel, but it doesn't look like the sklearn converter has anything similar...

                                                   

                                                  Here is the code I was playing with

                                                   

                                                  from sklearn.neighbors import KNeighborsClassifier
                                                  import pandas as pd
                                                  import coremltools
                                                  
                                                  
                                                  model = KNeighborsClassifier(n_neighbors=1)
                                                  data = pd.read_csv('dataCSV.csv', header=0)
                                                  
                                                  headers = ['Rating','Time','ResID','UserID']
                                                   
                                                  data.columns = headers
                                                  
                                                  model.fit(data[['Time', 'ResID', 'UserID']], data['Rating'])
                                                  
                                                  coreml_model = coremltools.converters.sklearn.convert(model)
                                                  
                                                  coreml_model.save('classifier.mlmodel')