Post not yet marked as solved
On accessing the CoreML Model Deployment dashboard with my developer account,
the page gives a bad request error saying "Your request was invalid".
Also, when I try to create a Model Collection it gives an error saying "One of the fields was invalid".
Post not yet marked as solved
Hi, I have a core ml model that when I try to print the:
modelPrediction?.labelProbability
which is of type String:Double and contains all the features with their corresponding probabilities, the value of type double comes with nan
rest = nan
right = nan
up = nan
Sometimes restarting makes it work again.
Sometimes it can take a lot of restarts to start working again.
Even when deleting the app and installing again the same thing happens.
Also tried changing the deployment version but didn't seem to fix it.
Any help is appreciated.
Post not yet marked as solved
We've 10 CoreML models in our app, each encrypted with a separate key generated in XCode.
After opening and closing the app 6-7 times, the app crashes at model initialization with error:
2021-04-21 13:52:47.711729+0300 MyApp[95443:7341643] Fatal error: 'try!' expression unexpectedly raised an error: Error Domain=com.apple.CoreML Code=9 "Failed to generate key request for 08494FB2-B070-440F-A8A5-CBD0823A258E with error: -42905" UserInfo={NSLocalizedDescription=Failed to generate key request for 08494FB2-B070-440F-A8A5-CBD0823A258E with error: -42905}: file MyApp/Model.swift, line 43
Looks like iPhone is blocking the app for suspicious behavior and the app fails to decrypt the model.
We noticed that after ~10 hours the app is unlocked, it successfully decrypts and initializes the model.
Opening and closing the app many times in a short period of time is indeed unnatural, but the most important question is how to avoid blocking? Would Apple block the app if a user opens and closes it 10 times during a day? How does the number of models in the app affect probability that the app will be blocked?
Thanks!
Post not yet marked as solved
With the release of Xcode 13, a large section of my vision framework processing code became errors and cannot compile. All of these have became deprecated.
This is my original code:
do {
// Perform VNDetectHumanHandPoseRequest
try handler.perform([handPoseRequest])
// Continue only when a hand was detected in the frame.
// Since we set the maximumHandCount property of the request to 1, there will be at most one observation.
guard let observation = handPoseRequest.results?.first else {
self.state = "no hand"
return
}
// Get points for thumb and index finger.
let thumbPoints = try observation.recognizedPoints(forGroupKey: .handLandmarkRegionKeyThumb)
let indexFingerPoints = try observation.recognizedPoints(forGroupKey: .handLandmarkRegionKeyIndexFinger)
let middleFingerPoints = try observation.recognizedPoints(forGroupKey: .handLandmarkRegionKeyMiddleFinger)
let ringFingerPoints = try observation.recognizedPoints(forGroupKey: .handLandmarkRegionKeyRingFinger)
let littleFingerPoints = try observation.recognizedPoints(forGroupKey: .handLandmarkRegionKeyLittleFinger)
let wristPoints = try observation.recognizedPoints(forGroupKey: .all)
// Look for tip points.
guard let thumbTipPoint = thumbPoints[.handLandmarkKeyThumbTIP],
let thumbIpPoint = thumbPoints[.handLandmarkKeyThumbIP],
let thumbMpPoint = thumbPoints[.handLandmarkKeyThumbMP],
let thumbCMCPoint = thumbPoints[.handLandmarkKeyThumbCMC] else {
self.state = "no tip"
return
}
guard let indexTipPoint = indexFingerPoints[.handLandmarkKeyIndexTIP],
let indexDipPoint = indexFingerPoints[.handLandmarkKeyIndexDIP],
let indexPipPoint = indexFingerPoints[.handLandmarkKeyIndexPIP],
let indexMcpPoint = indexFingerPoints[.handLandmarkKeyIndexMCP] else {
self.state = "no index"
return
}
guard let middleTipPoint = middleFingerPoints[.handLandmarkKeyMiddleTIP],
let middleDipPoint = middleFingerPoints[.handLandmarkKeyMiddleDIP],
let middlePipPoint = middleFingerPoints[.handLandmarkKeyMiddlePIP],
let middleMcpPoint = middleFingerPoints[.handLandmarkKeyMiddleMCP] else {
self.state = "no middle"
return
}
guard let ringTipPoint = ringFingerPoints[.handLandmarkKeyRingTIP],
let ringDipPoint = ringFingerPoints[.handLandmarkKeyRingDIP],
let ringPipPoint = ringFingerPoints[.handLandmarkKeyRingPIP],
let ringMcpPoint = ringFingerPoints[.handLandmarkKeyRingMCP] else {
self.state = "no ring"
return
}
guard let littleTipPoint = littleFingerPoints[.handLandmarkKeyLittleTIP],
let littleDipPoint = littleFingerPoints[.handLandmarkKeyLittleDIP],
let littlePipPoint = littleFingerPoints[.handLandmarkKeyLittlePIP],
let littleMcpPoint = littleFingerPoints[.handLandmarkKeyLittleMCP] else {
self.state = "no little"
return
}
guard let wristPoint = wristPoints[.handLandmarkKeyWrist] else {
self.state = "no wrist"
return
}
...
}
Now every line from thumbPoints onwards results in error, I have fixed the first part (not sure if it is correct or not as it cannot compile) to :
let thumbPoints = try observation.recognizedPoints(forGroupKey: VNHumanHandPoseObservation.JointsGroupName.thumb.rawValue)
let indexFingerPoints = try observation.recognizedPoints(forGroupKey: VNHumanHandPoseObservation.JointsGroupName.indexFinger.rawValue)
let middleFingerPoints = try observation.recognizedPoints(forGroupKey: VNHumanHandPoseObservation.JointsGroupName.middleFinger.rawValue)
let ringFingerPoints = try observation.recognizedPoints(forGroupKey: VNHumanHandPoseObservation.JointsGroupName.ringFinger.rawValue)
let littleFingerPoints = try observation.recognizedPoints(forGroupKey: VNHumanHandPoseObservation.JointsGroupName.littleFinger.rawValue)
let wristPoints = try observation.recognizedPoints(forGroupKey: VNHumanHandPoseObservation.JointsGroupName.littleFinger.rawValue)
I tried many different things but just could not get the retrieving individual points to work. Can anyone help on fixing this?
Post not yet marked as solved
I implement a custom pytorch layer on both CPU and GPU following [Hollemans amazing blog] (https://machinethink.net/blog/coreml-custom-layers ). The cpu version works good, but when i implemented this op on GPU it cannot activate "encode" function. Always run on CPU. I have checked the coremltools.convert() options with compute_units=coremltools.ComputeUnit.CPU_AND_GPU, but it still not work. This problem also mentioned in https://stackoverflow.com/questions/51019600/why-i-enabled-metal-api-but-my-coreml-custom-layer-still-run-on-cpu and https://developer.apple.com/forums/thread/695640. Any idea on help this would be grateful.
System Information
mac OS: 11.6.1 Big Sur
xcode: 12.5.1
coremltools: 5.1.0
test device: iphone 11
Post not yet marked as solved
Hello, I'm new using CoreML and I'm trying to do a test app with the models that already exist.
I'm having next error at the moment to classifier the image:
[coreml] Failed to get the home directory when checking model path.
I would like to receive your help to solve this error. Thanks.
Post not yet marked as solved
I have tried many times. When I change the file or re-create it, it shows 404 error
{
"code": 400,
"message": "InvalidArgumentError: Unable to unzip MLArchive",
"reason": "There was a problem with your request.",
"detailedMessage": "InvalidArgumentError: Unable to unzip MLArchive",
"requestUuid": "699afb97-8328-4a83-b186-851f797942aa"
}
Post not yet marked as solved
I am trying to train an image classification network in Keras with tensorflow-metal.
The training freezes after the first 2-3 epochs if image augmentation layers are used (RandomFlip, RandomContrast, RandomBrightness)
The system appears to use both GPU as well as CPU (as indicated by Activity Monitor). Also, warnings appear both in Jupyter and Terminal (see below).
When the image augmentation layers are removed (i.e. we only rebuild the head and feed images from disk), CPU appears to be idle, no warnings appear, and training completes successfully.
Versions: python 3.8, tensorflow-macos 2.11.0, tensorflow-metal 0.7.1
Sample code:
img_augmentation = Sequential(
[
layers.RandomFlip(),
layers.RandomBrightness(factor=0.2),
layers.RandomContrast(factor=0.2)
],
name="img_augmentation",
)
inputs = layers.Input(shape=(384, 384, 3))
x = img_augmentation(inputs)
model = tf.keras.applications.EfficientNetV2S(include_top=False, input_tensor=x, weights='imagenet')
model.trainable = False
x = tf.keras.layers.GlobalAveragePooling2D(name="avg_pool")(model.output)
x = tf.keras.layers.BatchNormalization()(x)
top_dropout_rate = 0.2
x = tf.keras.layers.Dropout(top_dropout_rate, name="top_dropout")(x)
outputs = tf.keras.layers.Dense(179, activation="softmax", name="pred")(x)
newModel = Model(inputs=model.input, outputs=outputs, name="EfficientNet_DF20M_species")
reduce_lr = tf.keras.callbacks.ReduceLROnPlateau(monitor='val_accuracy', factor=0.9, patience=2, verbose=1, min_lr=0.000001)
optimizer = tf.keras.optimizers.legacy.SGD(learning_rate=0.01, momentum=0.9)
newModel.compile(optimizer=optimizer, loss='categorical_crossentropy', metrics=['accuracy'])
history = newModel.fit(x=train_ds, validation_data=val_ds, epochs=30, verbose=2, callbacks=[reduce_lr])
During training with image augmentation, Jupyter prints the following warnings while training the first epoch:
WARNING:tensorflow:Using a while_loop for converting Bitcast cause there is no registered converter for this op.
WARNING:tensorflow:Using a while_loop for converting Bitcast cause there is no registered converter for this op.
WARNING:tensorflow:Using a while_loop for converting StatelessRandomUniformV2 cause there is no registered converter for this op.
WARNING:tensorflow:Using a while_loop for converting RngReadAndSkip cause there is no registered converter for this op.
WARNING:tensorflow:Using a while_loop for converting Bitcast cause there is no registered converter for this op.
WARNING:tensorflow:Using a while_loop for converting Bitcast cause there is no registered converter for this op.
WARNING:tensorflow:Using a while_loop for converting StatelessRandomUniformFullIntV2 cause there is no registered converter for this op.
WARNING:tensorflow:Using a while_loop for converting StatelessRandomGetKeyCounter cause there is no registered converter for this op.
...
During training with image augmentation, Terminal keeps spamming the following warning:
2023-02-21 23:13:38.958633: I metal_plugin/src/kernels/stateless_random_op.cc:282] Note the GPU implementation does not produce the same series as CPU implementation.
2023-02-21 23:13:38.958920: I metal_plugin/src/kernels/stateless_random_op.cc:282] Note the GPU implementation does not produce the same series as CPU implementation.
2023-02-21 23:13:38.959071: I metal_plugin/src/kernels/stateless_random_op.cc:282] Note the GPU implementation does not produce the same series as CPU implementation.
2023-02-21 23:13:38.959115: I metal_plugin/src/kernels/stateless_random_op.cc:282] Note the GPU implementation does not produce the same series as CPU implementation.
2023-02-21 23:13:38.959359: I metal_plugin/src/kernels/stateless_random_op.cc:282] Note the GPU implementation does not produce the same series as CPU implementation.
...
Any suggestions?
Post not yet marked as solved
I am working on the neural network classifier provided on the coremltools.readme.io in the updatable->neural network section(https://coremltools.readme.io/docs/updatable-neural-network-classifier-on-mnist-dataset).
I am using the same code but I get an error saying that the coremltools.converters.keras.convert does not exist. But this I know can be coreml version issue. Right know I am using coremltools version 6.2. I converted this model to mlmodel with .convert only. It got converted successfully.
But I face an error in the make_updatable function saying the loss layer must be softmax output. Even the coremlt package API reference there I found its because the layer name is softmaxND but it should be softmax.
Now the problem is when I convert the model from Keras sequential model to coreml model. the layer name and type change. And the softmax changes to softmaxND.
Does anyone faced this issue?
if I execute this builder.inspect_layers(last=4)
I get this output
[Id: 32], Name: sequential/dense_1/Softmax (Type: softmaxND)
Updatable: False
Input blobs: ['sequential/dense_1/MatMul']
Output blobs: ['Identity']
[Id: 31], Name: sequential/dense_1/MatMul (Type: batchedMatmul)
Updatable: False
Input blobs: ['sequential/dense/Relu']
Output blobs: ['sequential/dense_1/MatMul']
[Id: 30], Name: sequential/dense/Relu (Type: activation)
Updatable: False
Input blobs: ['sequential/dense/MatMul']
Output blobs: ['sequential/dense/Relu']
In the make_updatable function when I execute
builder.set_categorical_cross_entropy_loss(name='lossLayer', input='Identity')
I get this error
ValueError: Categorical Cross Entropy loss layer input (Identity) must be a softmax layer output.
Post not yet marked as solved
In the ml-ane-transformers repo, there is a custom LayerNorm implementation for the Neural Engine-optimized shape of (B,C,1,S).
The coremltools documentation makes it sound like the layer_norm MIL op would support this natively. In fact, the following code works on CPU:
B,C,S = 1,768,512
g,b = 1, 0
@mb.program(input_specs=[mb.TensorSpec(shape=(B,C,1,S)),])
def ln_prog(x):
gamma = (torch.ones((C,), dtype=torch.float32) * g).tolist()
beta = (torch.ones((C), dtype=torch.float32) * b).tolist()
return mb.layer_norm(x=x, axes=[1], gamma=gamma, beta=beta, name="y")
However it fails when run on the Neural Engine, giving results that are scaled by an incorrect value.
Should this work on the Neural Engine?
Post not yet marked as solved
I have a csv file with values ranges between 0-1. When converted this csv values to MLMultiArray and then to UIImage it shows grey image. But the image is actually coloured. So is there any step missing or do I have to perform any more action to get this image to be coloured.
Sample csv values - [0.556862745 0.62745098 0.811764706]
Post not yet marked as solved
I am trying to use this repository https://github.com/Ma-Dan/Yolact-CoreML to test how fast the YOLACT Model would run on a new iPad, but I am getting this error:
Thread 1: EXC_BAD_ACCESS (code=1, address=0x0)
On this line:
#0 0x00000001a2d19830 in Espresso::ANECompilerEngine::transpose_kernel::is_valid_for_engine(std::__1::shared_ptrEspresso::kernels_validation_status_t, Espresso::base_kernel::validate_for_engine_args_t const&) const ()
I modified line 37 on the Yolact.swift file that was throwing this error from:
let model = yolact()
to
let model: yolact = {
do {
let config = MLModelConfiguration()
return try yolact(configuration: config)
} catch{
print(error)
fatalError("Couldn't create model")
}
}()
Removing the only code warning that I got.
I don't know what else I can do.
Post not yet marked as solved
Recently we added model encryption for an application we work on. While doing so, our tests notified us about changes in model inference results. I built a small application to reproduce the phenomena.
While the changes are quite small (about 1e-9 maximum), we do want to know what causes the changes. We assumed encrypting a model does not alter it at all and the result should be bit-exact the same before and after.
Anyone else experienced this? Is there an explanation why this changes happen?
Post not yet marked as solved
I have a csv file with values ranges between 0-1. When converted this csv values to MLMultiArray and then to UIImage it shows grey image. But the image is actually coloured. So is there any step missing or do I have to perform any more action to get this image to be coloured.
Sample csv values - [0.556862745 0.62745098 0.811764706]
Code
func parseCSV(data: String) -> [Float] {
var finalArray = [Float]()
var rows = data.components(separatedBy: "\n")
for row in rows {
let columns = row.components(separatedBy: ",")
if columns.count == 3 {
let r = columns[0]
let g = columns[1]
var b = columns[2]
if b.contains("\r"){
b = b.replacingOccurrences(of: "\r", with: "")
}
finalArray.append(Float(r)!)
finalArray.append(Float(g)!)
finalArray.append(Float(b)!)
}
}
return finalArray
}
let m = try MLMultiArray(shape: [1, 768, 512, 3], dataType: .double)
for (index, element) in data.enumerated() {
m[index] = NSNumber(value: element)
}
let model: ModelInput = {
do {
let config = MLModelConfiguration()
return try ModelInput(configuration: config)
} catch {
fatalError("Couldn't create Prediction model")
}
}()
let op = try model.prediction(input: ModelInput(input1: m))
let opvalue = op.featureValue(for: "Output")
let multiArray = opvalue!.multiArrayValue!
Post not yet marked as solved
Hey guys,
I converted a T5-base (encoder/decoder) model to a CoreML model using https://github.com/huggingface/exporters (which are using coremltools under the hood). When creating a performance report for the decoder model within XCode it shows that all compute units are mapped to the CPU. This is also the experience I have when profiling the model (GPU and ANE are not used).
I was under the impression that CoreML would divide up the layers and run those that can run on the GPU / ANE, but maybe I misunderstood.
Is there anything I can do to get this to not run on the CPU exclusively?
Hello,
I am reaching out for some assistance regarding integrating a CoreML action classifier into a SwiftUI app. Specifically, I am trying to implement this classifier to work with the live camera of the device. I have been doing some research, but unfortunately, I have not been able to find any relevant information on this topic.
I was wondering if you could provide me with any examples, resources, or information that could help me achieve this integration? Any guidance you can offer would be greatly appreciated.
Thank you in advance for your help and support.
Post not yet marked as solved
I use CoreML to infer this model in a loop. And the pridiction will crash randomly with the same problem. Below is the backtrace.
* frame #0: 0x00000001bf659f60 vImage`vConvert_Planar16FtoPlanarF_vec + 172
frame #1: 0x00000001c7f8b304 Espresso`Espresso::ANERuntimeEngine::blob_container::__copy_to_host(std::__1::shared_ptr<Espresso::abstract_batch> const&, std::__1::shared_ptr<Espresso::blob<float, 4> >&) + 2408
frame #2: 0x00000001c7605458 Espresso`EspressoLight::espresso_plan::__copy_outputs(std::__1::shared_ptr<EspressoLight::plan_task_t>, std::__1::shared_ptr<Espresso::abstract_batch> const&, int, std::__1::shared_ptr<Espresso::net>) + 904
frame #3: 0x00000001c75ea338 Espresso`EspressoLight::espresso_plan::dispatch_task_on_compute_batch(std::__1::shared_ptr<Espresso::abstract_batch> const&, std::__1::shared_ptr<EspressoLight::plan_task_t> const&) + 760
frame #4: 0x00000001c75f2168 Espresso`EspressoLight::espresso_plan::execute_sync() + 452
frame #5: 0x00000001c75ea4f0 Espresso`espresso_plan_execute_sync + 132
frame #6: 0x00000001d14a4020 CoreML`-[MLNeuralNetworkEngine executePlan:error:] + 116
frame #7: 0x00000001d14a357c CoreML`-[MLNeuralNetworkEngine evaluateInputs:bufferIndex:options:error:] + 340
frame #8: 0x00000001d1707514 CoreML`__67-[MLNeuralNetworkEngine evaluateInputs:options:verifyInputs:error:]_block_invoke.195 + 36
frame #9: 0x0000000105bca05c libdispatch.dylib`_dispatch_client_callout + 20
frame #10: 0x0000000105bdb884 libdispatch.dylib`_dispatch_lane_barrier_sync_invoke_and_complete + 172
frame #11: 0x00000001d1707380 CoreML`-[MLNeuralNetworkEngine evaluateInputs:options:verifyInputs:error:] + 440
frame #12: 0x00000001d14a90d0 CoreML`-[MLNeuralNetworkEngine evaluateInputs:options:error:] + 156
frame #13: 0x00000001d14aacc8 CoreML`__62-[MLNeuralNetworkEngine predictionFromFeatures:options:error:]_block_invoke + 116
frame #14: 0x0000000105bca05c libdispatch.dylib`_dispatch_client_callout + 20
frame #15: 0x0000000105bdb884 libdispatch.dylib`_dispatch_lane_barrier_sync_invoke_and_complete + 172
frame #16: 0x00000001d14a4764 CoreML`-[MLNeuralNetworkEngine predictionFromFeatures:options:error:] + 500
frame #17: 0x0000000106d11470 AnfCoreMLCodec`AnfCoreMLCodec.encode.prediction(input: AnfCoreMLCodec.encodeInput, options: __C.MLPredictionOptions) throws -> AnfCoreMLCodec.encodeOutput + 56
frame #18: 0x0000000106d11410 AnfCoreMLCodec`AnfCoreMLCodec.encode.prediction(input: AnfCoreMLCodec.encodeInput) throws -> AnfCoreMLCodec.encodeOutput + 68
frame #19: 0x0000000106d08518 AnfCoreMLCodec`AnfCoreMLCodec.EncodeModelPredictor.encodePrediction(input: AnfCoreMLCodec.encodeInput) throws -> __C.MLFeatureProvider + 52
When I run the performance test on a CoreML model, it shows predictions are 834% faster running on the Neural Engine as it is on the GPU.
It also shows, that 100% of the model can run on the Neural Engine:
GPU only:
But when I set the compute units to all:
let config = MLModelConfiguration()
config.computeUnits = .all
and profile, it shows that the neural engine isn’t used at all. Well, other than loading the model which takes 25 seconds when allowed to use the neural engine versus less than a second when not allowing the neural engine:
The difference in speed is the difference between the app being too slow to even release versus quite reasonable performance. I have a lot of work invested in this, so I am really hoping that I can get it to run on the Neural Engine.
Why isn't it actually running on the Neural Engine when it shows that it is supported and I have the compute unit set to run on the Neural Engine?
Post not yet marked as solved
Hello, I am a student and I am doing a search for my thesis on create ML and shape recognition and image processing, so for this subject I want to find the details of the steps used in create ML for this, such as the techniques used for pre-processing, and the methods of extracting characteristics, and the filters applied, ect...
Post not yet marked as solved
We have CoreML models in our app, each encrypted with a separate key generated in XCode. After app update we are receiving following error
`
`[coreml] Could not create persistent key blob for EFD428E8-CDE7-4E0A-B379-FC169E50DE4D : error=Error Domain=com.apple.CoreML Code=8 "Fetching decryption key from server failed." UserInfo={NSLocalizedDescription=Fetching decryption key from server failed., NSUnderlyingError=0x281d80ab0 {Error Domain=CKErrorDomain Code=6 "CKInternalErrorDomain: 2022" UserInfo={NSDebugDescription=CKInternalErrorDomain: 2022, RequestUUID=D5CF13CF-6A10-436B-AB93-4C5C04859FFE, NSLocalizedDescription=Request failed with http status code 503, CKErrorDescription=Request failed with http status code 503, CKRetryAfter=35, NSUnderlyingError=0x281d80000 {Error Domain=CKInternalErrorDomain Code=2022 "Request failed with http status code 503" UserInfo={CKRetryAfter=35, CKHTTPStatus=503, CKErrorDescription=Request failed with http status code 503, RequestUUID=D5CF13CF-6A10-436B-AB93-4C5C04859FFE, NSLocalizedDescription=Request failed with http status code 503}}, CKHTTPStatus=503}}}`
Tried deleting app, restarting device but nothing works. This was released on Appstore earlier and was working fine. It stopped working after update.
Any help is appreciated.
