CoreML performance issue on M1Max about Neural Engine

Question

Tinyfool OP

Created Dec ’21

Replies 4

Boosts 0

Views 2.3k

Participants 4

I am using a CoreML model from https://github.com/PeterL1n/RobustVideoMatting.

I have an M1Macbook13 16G and an M1Max Macbook 16 64G.

When "computeUnits" using .all or default, M1Max 16 is much slower than M1 13, finish one prediction time is 0.202 and 0.155.

Using .cpuOnly, M1Max 16 is fast a little, time is 0.129 and 0.146.

Using .cpuAndGPU, M1Max 16 is much fast than M1 13, time is 0.057 and 0.086.

And when I use .all or default, M1Max will appear error messages like this:

H11ANEDevice::H11ANEDeviceOpen IOServiceOpen failed result= 0xe00002e2
H11ANEDevice::H11ANEDeviceOpen kH11ANEUserClientCommand_DeviceOpen call failed result=0xe00002bc
Error opening LB - status=0xe00002bc.. Skipping LB and retrying

But M1 13 doesn't have any errors.

So I want to know is this a bug of CoreML or M1Max?

My Codes is like this:

let config = MLModelConfiguration()
config.computeUnits = .all

let model = try rvm_mobilenetv3_1920x1080_s0_25_int8_ANE(configuration: config)

let image1 = NSImage(named: "test1")?.cgImage(forProposedRect: nil, context: nil, hints: nil)
let input = try? rvm_mobilenetv3_1920x1080_s0_25_int8_ANEInput(srcWith:image1!, r1i: MLMultiArray(), r2i: MLMultiArray(), r3i: MLMultiArray(), r4i: MLMultiArray())

 _ = try? model.prediction(input: input!)

Boost

Answer 1

TaigaTakano OP

Jan ’22

I ran into this problem too. I used CoreML to do a simple image classification. I tested it on the following hardware.

M1 Pro MacBook Pro 16 16G
M1 iMac 24 16G
intel MacBook Pro 13 16G
iPhone 13 mini

As a result of the verification, the following problems occurred only with M1Pro.

H11ANEDevice::H11ANEDeviceOpen kH11ANEUserClientCommand_DeviceOpen call failed result=0xe00002bc
Error opening LB - status=0xe00002bc.. Skipping LB and retrying

So I also compared the execution speed and got the following results.

M1 Pro(74s)
M1 (52s)

As shown above, the M1 was 42% faster than the M1Pro.

My codes is like this:

import Foundation
import Vision
import CoreML
import CoreImage

class MacInference{

    private let model = try? testml(configuration: MLModelConfiguration()).model //CoreML model   

    var TopResultConfidence : Float = 0.0
    var TopResultId = ""
    var debug = false

    private func checkFile(fileURL : String) -> Bool{
        let filePath = fileURL.replacingOccurrences(of: "file://", with: "")
        if FileManager.default.fileExists(atPath: filePath) {
            return true
        }else{
            return false
        }
    }

    func startML(fileURL : String){
        let coreMLModel = try? VNCoreMLModel(for: self.model!)
        let request = VNCoreMLRequest(model: coreMLModel!){ request, error in
            if let results = request.results as? [VNClassificationObservation]{
                self.TopResultConfidence = results[0].confidence
                self.TopResultId = results[0].identifier
                
                if self.debug {
                    for result in results {
                        print(result.confidence * 100, result.identifier)
                    }
                }
            }
        }

        if !self.checkFile(fileURL: fileURL){
            print("URL error")
            return
        }

        let ciimage = CIImage(contentsOf: URL(string: fileURL)!)
        let handler = VNImageRequestHandler(ciImage: ciimage!, options: [:])
        do {
            try handler.perform([request])
        } catch {
            print(error)
        }
    }
}

let macML = MacInference()

macML.debug = true

macML.startML(fileURL: "FILE URL")

2

Answer 2

OP

Apple

Mar ’22

Hello, could one of you please file a bug report on http://feedbackassistant.apple.com/ along with a sysdiagnoses from the device?

0

Answer 3

TaigaTakano OP

Mar ’22

For some reason feedbackassistant didn't work for me, so I'm uploading it here.

https://1drv.ms/u/s!AmGvVvDPI0CMwTgzVGFESvcaAC2E?e=FcufOp

This OneDrive folder contains the following contents

・Project Files

・sysdiagnoses

・Output of Xcode

Here is what I wanted to write to FeedbackAssistant.

Title : ANE in M1 max does not seem to be working.

Xcode Version : Version 13.2.1 (13C100)

Steps to reproduce : Run the project that exists at the URL presented earlier. That is all there is to the procedure. Note, the problem seems to occur only with M1 Pro , M1 Max.

0

Answer 4

vade OP

Mar ’22

I haven't had a chance to test my vanilla m1 for performance numbers, but it seems like macOS 12.3 has fixed the

H11ANEDevice::H11ANEDeviceOpen kH11ANEUserClientCommand_DeviceOpen call failed result=0xe00002bc

error I have also been getting on M1 Pro / M1 Max systems.

0