Hello everybody,
For the past week I have been struggling to run inference on a classifier I built using Google's AutoML Vision tool.
At first I thought everything would go smoothly because Google allows to export a CoreML version of the final model. I assumed I would only need to use Apple's CoreML library to make it work. When I export the model Google provides a .mlmodel file and a dict.txt file with the classification labels. For the current model I have 100 labels.
This is my Swift code to run inference on the model.
private lazy var classificationRequest: VNCoreMLRequest = {
				do {
						let classificationModel = try VNCoreMLModel(for: NewGenusModel().model)
						let request = VNCoreMLRequest(model: classificationModel, completionHandler: { [weak self] request, error in
								self?.processClassifications(for: request, error: error)
						})
						request.imageCropAndScaleOption = .scaleFit
						return request
				}
				catch {
						fatalError("Error! Can't use Model.")
				}
		}()
		func classifyImage(receivedImage: UIImage) {
				let orientation = CGImagePropertyOrientation(rawValue: UInt32(receivedImage.imageOrientation.rawValue))
				if let image = CIImage(image: receivedImage) {
						DispatchQueue.global(qos: .userInitiated).async {
								let handler = VNImageRequestHandler(ciImage: image, orientation: orientation!)
								do {
										try handler.perform([self.classificationRequest])
								}
								catch {
										fatalError("Error classifying image!")
								}
						}
				}
		}
The problem started when I tried to pass a UIImage to run inference on the model. The input type of the original model was MultiArray (Float32 1 x 224 x 224 x 3). Using Coremltools library I was able to convert the input type to Image (Color 224 x 224) using Python.
This worked and here is my code:
import coremltools
import coremltools.proto.FeatureTypes_pb2 as ft
spec = coremltools.utils.load_spec("model.mlmodel")
input = spec.description.input[0]
input.type.imageType.colorSpace = ft.ImageFeatureType.RGB
input.type.imageType.height = 224
input.type.imageType.width = 224
coremltools.utils.save_spec(spec, "newModel.mlmodel")
My problem now is with the output type. I want to be able to access the confidence of the classification as well as the result label of the classification. Again using coremltools I was able to to access the output description and I got this.
name: "scores"
type {
	multiArrayType {
		dataType: FLOAT32
	}
}
I am trying to change it this way:
f = open("dict.txt", "r")
labels = f.read()
class_labels = labels.splitlines()
print(class_labels)
class_labels = class_labels[1:]
assert len(class_labels) == 57
for i, label in enumerate(class_labels):
	if isinstance(label, bytes):
		class_labels[i] = label.decode("utf8")
classifier_config = ct.ClassifierConfig(class_labels)
output = spec.description.output[0]
output.type = ft.DictionaryFeatureType
Unfortunately this is not working and I can't find information only that can help me... This I don't know what to do next.
Thank you for your help!
Post
Replies
Boosts
Views
Activity
Hello everybody,
I used Google Cloud Platform to create a Machine learning model to perform computer vision. I downloaded the CoreML model from the cloud platform website and followed the instructions in the Google Tutorial for iOS model deployment.
This is my code currently.
class Classification {
private lazy var classificationRequest: VNCoreMLRequest = {
do {
let model = try VNCoreMLModel(for: AutoML().model)
let request = VNCoreMLRequest(model: model, completionHandler: { [weak self] request, error in
if let classifications = request.results as? [VNClassificationObservation] {
print(classifications.first ?? "No classification!")
}
})
request.imageCropAndScaleOption = .scaleFit
return request
}
catch {
fatalError("Error! Can't use Model.")
}
}()
func classifyImage(receivedImage: UIImage) {
let orientation = CGImagePropertyOrientation(rawValue: UInt32(receivedImage.imageOrientation.rawValue))
if let image = CIImage(image: receivedImage) {
DispatchQueue.global(qos: .userInitiated).async {
let handler = VNImageRequestHandler(ciImage: image, orientation: orientation!)
do {
try handler.perform([self.classificationRequest])
}
catch {
fatalError("Error classifying image!")
}
}
}
}
My code executes and I receive this:
<VNClassificationObservation: 0x600002091d40> A7DBD70C-541C-4112-84A4-C6B4ED2EB7E2 requestRevision=1 confidence=0.332127 "CICAgICAwPmveRIJQWdsYWlzX2lv"
I receive a confidence value but I don't receive a label string.
Is there any step I am not taking?
With the model there is also a dict.txt file. Is there anything I have to do with that and that I am not doing?
Thank you!
Hello everybody,
I am building a Map and I am using a map view that I want to be always present in the interface. However, the user can share to show its location or not.
The Map view requires a region. But I want to show an empty map view with just the grid on with a message.
Do you know if it is possible to achieve that? Thank you!
let span = MKCoordinateSpan(latitudeDelta: CLLOcationDegrees, longitudeDelta: CLLOcationDegrees)
let region = MKCoordinateRegion(center: observationCoordinates, span: span)
				
let annotation =	MKPointAnnotation()
annotation.coordinate = observationCoordinates
				
uiView.setRegion(region, animated: true)
uiView.addAnnotation(annotation)
Hello everybody,
I am trying to navigate from one view to another using NavigationView and NavigationLink.
I can't figure out what I am doing wrong, but the way I am doing it I get a warning saying:
"Result of NavigationLink<Label, Destination> is unused".
In fact, I can't navigate to the view I want to open.
Here is my code so far:
NavigationView {
VStack {
Button(action: {
let observation = self.createObservation()
self.records.addObservation(observation)
self.isPresented.toggle()
NavigationLink(destination: ObservationDetails(observation: observation).environmentObject(self.records)) {
EmptyView()
}
}) {
Text("Classify")
}
}
Thank you for your help!
Hello everyone,
I want to use a map view to display an annotation but when I place it on the view it always stays in at the center of the map view.
I want to give it some offset so that it goes a little bit upwards.
This is the code I have so far. Could you please help me?
let span = MKCoordinateSpan(latitudeDelta: CLLOcationDegrees, longitudeDelta: CLLOcationDegrees)
let region = MKCoordinateRegion(center: observationCoordinates, span: span)
let annotation = MKPointAnnotation()
annotation.coordinate = observationCoordinates
uiView.setRegion(region, animated: true)
uiView.addAnnotation(annotation)
Thank you for your help.
Hello everyone,I am working on a Playground and I can change the font type of my label. I have tried doing it in several different ways and I just can't do it.var clueLabel = UILabel()
clueLabel.font = UIFont(name: "MarkerFelt-Wide", size: 45)
clueLabel.font = UIFont.systemFont(ofSize: 45, weight: .bold)
clueLabel.textColor = UIColor.black
clueLabel.textAlignment = .center
view.addSubview(clueLabel)What am I doing wrong? I want to use the font Marker Felt.Thank you
Hello everybody,I have an animation to create ballons floating from the bottom to the top of the screen. Despite producing the behaviour I want, at some point, when the ballons are floating upwards the objectView I am using one ballon gets printed at the top left corner of the screen and does not move or disappear.Has this ever happend to any of you?func presentVictoryView(word: String) {
blackView.isHidden = false
for _ in 0 ... 30 {
let objectView = UIView()
objectView.translatesAutoresizingMaskIntoConstraints = false
objectView.frame = CGRect(x: 0, y: 0, width: 20, height: 100)
objectView.backgroundColor = .clear
//objectView.alpha = CGFloat(0.9)
objectView.isHidden = false
let ballon = UILabel()
ballon.translatesAutoresizingMaskIntoConstraints = false
ballon.frame = CGRect(x: 0, y: 0, width: 20, height: 100)
ballon.backgroundColor = .clear
//ballon.alpha = CGFloat(0.9)
ballon.text = ""
ballon.font = UIFont.systemFont(ofSize: 60)
objectView.addSubview(ballon)
NSLayoutConstraint.activate([
ballon.centerXAnchor.constraint(equalTo: objectView.centerXAnchor),
ballon.centerYAnchor.constraint(equalTo: objectView.centerYAnchor)
])
blackView.addSubview(objectView)
let randomXOffset = Int.random(in: -120 ..< 200)
let path = UIBezierPath()
path.move(to: CGPoint(x: 160 + randomXOffset, y: 1000))
path.addCurve(to: CGPoint(x: 100 + randomXOffset, y: -300), controlPoint1: CGPoint(x: 300 - randomXOffset, y: 600), controlPoint2: CGPoint(x: 70 + randomXOffset, y: 300))
let animation = CAKeyframeAnimation(keyPath: "position")
animation.path = path.cgPath
animation.repeatCount = 1
animation.duration = Double(arc4random_uniform(40) + 30) / 10
//animation.timeOffset = Double(arc4random_uniform(50))
objectView.layer.add(animation, forKey: "animate position along path")
}
newGame()
}I have tried a lot of things... I don't know what I am doing wrong...Thank you!
Hello everybody,I am having trouble in updating the text of a label. This only happens when I am about to call another function which is very strange.Here is my code:func updateWordLabelAndCheckVictory(letter: String, indexArray: [Int]) {
if let currentLabel = wordLabel.text {
var wordArray = Array(currentLabel)
for i in indexArray {
wordArray[i] = Character(letter)
}
wordLabel.text = String(wordArray)
if (game.checkVictory(label: wordLabel.text!)) {
currentGame += 1
presentVictoryView()
}
}
}I update the label in line 11. When the condition in line 13 is met the presentVictoryView method (line 15) executes before the label is updated... I don't really know what I am doing wrong. If I comment line 15 and don't call the presentVictoryView method the label is updated as usual...Could you please help me?Thank you.
Hello everybody,I am using a button trigger in Playgrounds to pass the button title to a function in another class.@objc func letterButtonPressed(_ sender: UIButton) {
sender.isHidden = true
hiddenButtons.append(sender)
letter = sender.currentTitle?.uppercased()
if letter != nil {
game.checkLetterInWord(letter: letter!)
}
}In this function above I check if the letter is nil first, so I don't undertstand why the function finds nil...public func checkLetterInWord(letter: String!) -> Bool {
print(letter!)
let wordCharacters = Array(word)
print(wordCharacters)
return true
}What am I doing wrong? Thank you for you help!
Hello!I want to verify a property of a tagBarItem but the navigation controller needs to be accessed on the main queue.However, I have a blocking call that I want to be processed in the background.The problem is that in line 15 when I access the value of petitionString it is empty.What am I doing wrong?performSelector(inBackground: #selector(fetchJSON), with: nil)@objc func fetchJSON() {
var petitionURLString: String = ""
DispatchQueue.main.async {
[weak self] in
if self?.navigationController?.tabBarItem.tag == 0 {
petitionURLString = "https://www.hackingwithswift.com/samples/petitions-1.json"
}
else {
petitionURLString = "https://www.hackingwithswift.com/samples/petitions-2.json"
}
}
if let url = URL(string: petitionURLString) {
if let data = try? Data(contentsOf: url) { //Este método é chamado de blocking call bloqueia a execução de código até todos os dados estarem descarregados.
//É seguro fazer parsing dos dados mas como pode dar erro usamos try?
parseData(json: data)
return
}
}
showError()
}
Hello everyone,I am trying to edit a tab bar controller to display more items. I am using a UIWindow but it just does not work. I have followed some tutorials and one thing I noticed is that in everyone of the the window property is already there.In my version of Xcode it is not. I have to declare it myself.Can you help me? Here is the code I am using.It always prints "exits with error". 😟class AppDelegate: UIResponder, UIApplicationDelegate {
var window: UIWindow?
//Window em que a aplicação está a ser executada.
//Função chamada pelo iOS quando a app termina de carregar.
func application(_ application: UIApplication, didFinishLaunchingWithOptions launchOptions: [UIApplication.LaunchOptionsKey: Any]?) -> Bool {
if let tabBarController = window?.rootViewController as? UITabBarController {
//Neste projeto, se repararmos no storyboard o primeiro ecrã que vemos faz referência a um tab bar controller, por isso está tudo dentro dele.
//Por isso é que o default view controller é um tab bar controller.
let storyboard = UIStoryboard(name: "Main", bundle: nil)
let viewController = storyboard.instantiateViewController(identifier: "NavController")
//Adicionamos um tabBarItem ao view controller.
viewController.tabBarItem = UITabBarItem(tabBarSystemItem: .topRated, tag: 1)
//The default value of this property is nil. When configuring a tab bar controller, you can use this property to specify the content for each tab of the tab bar interface.
tabBarController.viewControllers?.append(viewController)
}
else {
print("Exits with error.")
}
return true
}
Hello everybody,I am new to iOS development and I found an error that I can not get past. I have read a lot online and on Stackoverflow but I don't understand why this error keeps coming up.I have a table view controller and I want to write text to other view controller using:navigationController?.pushViewController(viewController, animated: true)I have this class where I already have an outlet for the label.import UIKit
class PetitionDetailsViewController: UIViewController {
@IBOutlet weak var PetitionDetailsOutlet: UILabel!
override func viewDidLoad() {
super.viewDidLoad()
}
}On the other class I have this code where I try to add text to the label in the PetitionDetailsViewController after tapping one of the rows in the table view.override func tableView(_ tableView: UITableView, didSelectRowAt indexPath: IndexPath) {
let viewController = PetitionDetailsViewController()
print(petitions[indexPath.row].body)
viewController.PetitionDetailsOutlet.text = petitions[indexPath.row].body
navigationController?.pushViewController(viewController, animated: true)
}I don't understand why this error keeps coming up. I have the outlet and initially the label is empty.Why is it nil?
Hello everybody,I am new to iOS development. I am trying to to load a web view into my app. I got it working but there is one line of code that I fundamentally don't understand.override func loadView() {
webView = WKWebView()
webView.navigationDelegate = self
view = webView //Cada ViewController tem uma view associada.
}What am I telling to the view when I write webView.navigationDelegate = self? In my head it makes much more sense if it was:self.view = webView.navigationDelegate Can you help me?Thank you! 😀
Hello everybody ,Two days ago I submitted my first iOS app. I was super excited. Yesterday my app came back rejected with “metadata rejected”. I am very inexperienced and I kind of panic and tried everything to solve it. I replied in the resolution center and then I made the mistake of clicking submit for review. And now I am waiting.Sorry to bother you but I really would like to hear from somebody with experience what is going to happen now...- Since I clicked submit for review will my app be again in the beginning of the queue?- what happens to my response in the resolution center? Will my reviewer still read it?- How long, in the worst case scenario will it take to get some feedback back?Thank you so much.Best regards,