Banded result when converting HEIF CGImage to vImage

Question

Created Dec ’19

Replies 5

Boosts 0

Views 1.4k

Participants 3

I've ran into a problem when reading a high-efficiency image file and converting it into 24-bit (packed without alpha channel) vImage for further processing. The resulting image has vertical black bands in some parts and other vertical bands in which some channels are missing (i.e has a magenta or green tone to it). The problem appears ony when the source image is 3000x2000 pixels or greater.

Here is a Playground code to reproduce the issue – unfortunately I can't find a way to attach files here.

import Cocoa
import Accelerate

// width: 2000 pixels results in no banding
// but width 3000 pixels results in banded image
let imageFileURL = Bundle.main.url(forResource: "sample_3000", withExtension: "heic")!
let imageSource  = CGImageSourceCreateWithURL(imageFileURL as CFURL, nil)!
let originalImage = CGImageSourceCreateImageAtIndex(imageSource, 0, nil)!

let colorSpace = CGColorSpace(name: CGColorSpace.sRGB)!

var expectedImageFormat = vImage_CGImageFormat(
    bitsPerComponent: UInt32(8),
    bitsPerPixel: UInt32(8 * 3),
    colorSpace: Unmanaged.passUnretained(colorSpace),
    bitmapInfo: CGBitmapInfo(rawValue: CGBitmapInfo.byteOrder32Big.rawValue | CGImageAlphaInfo.none.rawValue),
    version: 0,
    decode: nil,
    renderingIntent: originalImage.renderingIntent
)


var buff = vImage_Buffer()
buff.height = vImagePixelCount(originalImage.height)
buff.width = vImagePixelCount(originalImage.width)
buff.rowBytes = Int(buff.width) * (Int(expectedImageFormat.bitsPerPixel) + 7) / Int(expectedImageFormat.bitsPerComponent)

let bufferData = NSMutableData(length: Int(buff.height) * buff.rowBytes)!
buff.data = bufferData.mutableBytes

let initErr = vImageBuffer_InitWithCGImage(&buff, &expectedImageFormat, nil, originalImage, numericCast(kvImagePrintDiagnosticsToConsole | kvImageNoAllocate))
guard initErr == kvImageNoError else {
    print("error creating image: \(initErr)")
    abort()
}

var convertError = kvImageNoError
let convertedImageUnmanaged = vImageCreateCGImageFromBuffer(
    &buff,
    &expectedImageFormat,
    nil, // callback
    nil, // user data
    numericCast(kvImagePrintDiagnosticsToConsole | kvImageNoAllocate),
    &convertError
)

guard convertError == kvImageNoError else {
    print("error converting image: \(convertError)")
    abort()
}

let convertedImage = convertedImageUnmanaged!.takeRetainedValue()
// show the image – has vertical banding

// write it to a temp file
let tempBase = URL(fileURLWithPath:NSTemporaryDirectory())
let tempDir = tempBase.appendingPathComponent(UUID().uuidString, isDirectory: true)
try FileManager.default.createDirectory(at: tempDir, withIntermediateDirectories: true, attributes: nil)

let targetURL = tempDir.appendingPathComponent("result.tiff")
try NSImage(cgImage: convertedImage, size: .zero).tiffRepresentation?.write(to: targetURL)
print("Result image written to directory \(tempDir)")

Environment:

Xcode 11.3 (11C29)
macOS 10.15.2 (19C57)

In case anybody from Apple is watching, this is filed as FB7497362 and also contains the Playground bundle along with sample input and output images.

Boost

Answer 1

DTS Engineer OP

Apple

Dec ’19

I haven't analyzed your conversion code to find the issue, but I advise you to use vImageConvert_RGBA8888toRGB888 to remove the alpha channel:

var sourceBuffer = try! vImage_Buffer(cgImage: originalImage)
var destinationBuffer = try! vImage_Buffer(width: originalImage.width, height: originalImage.height, bitsPerPixel: UInt32(24))
vImageConvert_RGBA8888toRGB888(&sourceBuffer, &destinationBuffer, vImage_Flags())

let colorSpace = originalImage.colorSpace!

var destinationFormat = vImage_CGImageFormat(
    bitsPerComponent: UInt32(8),
    bitsPerPixel: UInt32(8 * 3),
    colorSpace: Unmanaged.passUnretained(colorSpace),
    bitmapInfo: CGBitmapInfo(rawValue: CGBitmapInfo.byteOrder32Big.rawValue | CGImageAlphaInfo.none.rawValue),
    version: 0,
    decode: nil,
    renderingIntent: originalImage.renderingIntent
)
var convertError = kvImageNoError
let convertedImageUnmanaged = vImageCreateCGImageFromBuffer(&destinationBuffer, &destinationFormat, nil, nil, numericCast(kvImagePrintDiagnosticsToConsole | kvImageNoAllocate), &convertError)
let convertedImage = convertedI

0

Answer 2

adib OP

Dec ’19

The desired intent is to normalize any CGImage into an RGB24 vImage buffer. I have no control on the source image format, and hence couldn't use the vImageConvert_xxxx without going through hoops.

0

Answer 3

DTS Engineer OP

Apple

Dec ’19

In that case, have a look at this article (https://developer.apple.com/documentation/accelerate/building_a_basic_conversion_workflow), which explains how to use the any-to-any converter.

0

Answer 4

adib OP

Jan ’20

Yes I saw that tutorial before.

The point I was trying to make is that the code snippet works fine for JPEG, PNG, and some other image files. However when it came to HEIF it failed.

0

Answer 5

OP

Apple

Jun ’20

content removed-

0