Live Text API fails: "failed because the buffer is nil"

Hello,

I am trying to play around with the Live Text API according to this docs - https://developer.apple.com/documentation/visionkit/enabling_live_text_interactions_with_images?changes=latest_minor

But it always fails with [api] -[CIImage initWithCVPixelBuffer:options:] failed because the buffer is nil.

I am running this on a UIImage instance that I got from VNDocumentCameraViewController.

This is my current implementation that I run after the scanned image is displayed:

private func setupLiveText() {
        guard let image = imageView.image else {
            return
        }

        let interaction = ImageAnalysisInteraction()
        imageView.addInteraction(interaction)

        Task {
            let configuration = ImageAnalyzer.Configuration([.text])

            let analyzer = ImageAnalyzer()

            do {
                let analysis = try await analyzer.analyze(image, configuration: configuration)

                DispatchQueue.main.async {
                    interaction.analysis = analysis
                }

            } catch {
                print(error.localizedDescription)
            }
        }
    }

It does not fail, it returns non-nil analysis object, but setting it to the interaction does nothing.

I am testing this on iPhone SE 2020 which has the A13 chip. This feature requires A12 and up.

Answered by DTS Engineer in 716206022

Hello,

You need to set the preferredInteractionTypes on your ImageAnalysisInteraction.

As noted in the docs:

"The default value for this property is an empty array that disables any interactions."

Accepted Answer

Hello,

You need to set the preferredInteractionTypes on your ImageAnalysisInteraction.

As noted in the docs:

"The default value for this property is an empty array that disables any interactions."

Hi! As far as the error is concerned, the code looks correct, I would suggest filing a bug report so we can take a further look.

Aside from that issue, you will need to set the preferredInteractionTypes in order to activate the interaction. Try setting them to .automatic just after you set the analysis. Also, It's best to keep the analyzer around, and only have one per app, if you are preforming more than just the one.

Thanks both! That was it! Once I saw the error it did not occur to me to check how the interaction is set up.

Also note for others trying this, you need to enable user interaction on the UIImageView otherwise the Live Text button does not work.

WARNING: Bug present on iOS 16 RC (possibly on earlier versions of iOS 16 beta): if interaction.preferredInteractionTypes = [.automatic], attempting to translate the text using the ad-hoc button will fail. Nothing happens.

As a workaround, I used interaction.preferredInteractionTypes = [.textSelection, .dataDetectors] which makes the translation work.

Live Text API fails: "failed because the buffer is nil"
 
 
Q