Image modification running in the background

Hello experts,


I want to create a small image modification app. Purpose is to replace a certain pixel color with another one by including a specific tolcerance value.


The concrete 'Replace color' functionality looks like following:

  func replaceColor(color:SKColor, withColor:SKColor, image:UIImage, tolerance:CGFloat, alpha:CGFloat) -> UIImage{
        
        var result = UIImage()
        
            // This function expects to get source color(color which is supposed to be replaced)
            // and target color in RGBA color space, hence we expect to get 4 color components: r, g, b, a
            assert(color.cgColor.numberOfComponents == 4 && withColor.cgColor.numberOfComponents == 4,
                   "Must be RGBA colorspace")
            
            // Allocate bitmap in memory with the same width and size as source image
            let imageRef = image.cgImage
            let width = imageRef?.width
            let height = imageRef?.height
            
            let colorSpace = CGColorSpace(name: CGColorSpace.sRGB)!
            
            let bytesPerPixel = 4
            let bytesPerRow = bytesPerPixel * width!;
            let bitsPerComponent = 8
            let bitmapByteCount = bytesPerRow * height!
            
            let rawData = UnsafeMutablePointer.allocate(capacity: bitmapByteCount)
            
            let context = CGContext(data: rawData, width: width!, height: height!, bitsPerComponent: bitsPerComponent, bytesPerRow: bytesPerRow, space: colorSpace,
                                    bitmapInfo: CGImageAlphaInfo.premultipliedLast.rawValue | CGBitmapInfo.byteOrder32Big.rawValue)
            
            
            let rc = CGRect(x: 0, y: 0, width: width!, height: height!)
            
            // Draw source image on created context
            context?.draw(imageRef!, in: rc)
            
            
            // Get color components from replacement color
            let withColorComponents = withColor.cgColor.components
            let r2 = UInt8(withColorComponents![0] * 255)
            let g2 = UInt8(withColorComponents![1] * 255)
            let b2 = UInt8(withColorComponents![2] * 255)
            let a2 = UInt8(withColorComponents![3] * 255)
            
            // Prepare to iterate over image pixels
            var byteIndex = 0
            
            while byteIndex < bitmapByteCount {
                
                // Get color of current pixel
                let red:CGFloat = CGFloat(rawData[byteIndex + 0])/255
                let green:CGFloat = CGFloat(rawData[byteIndex + 1])/255
                let blue:CGFloat = CGFloat(rawData[byteIndex + 2])/255
                let alpha:CGFloat = CGFloat(rawData[byteIndex + 3])/255
                
                let currentColor = SKColor(red: red, green: green, blue: blue, alpha: alpha);
                
                // Compare two colors using given tolerance value
                if self.compareColor(color: color, withColor: currentColor , withTolerance: tolerance) {
                    
                    // If the're 'similar', then replace pixel color with given target color
                    rawData[byteIndex + 0] = r2
                    rawData[byteIndex + 1] = g2
                    rawData[byteIndex + 2] = b2
                    rawData[byteIndex + 3] = a2
                }
                
                byteIndex = byteIndex + 4;
            }
            
            // Retrieve image from memory context
            let imgref = context!.makeImage()
            result = UIImage(cgImage: imgref!)
            
            // Clean up a bit
            rawData.deinitialize()
        
        return result
    }
    
    func compareColor(color:SKColor, withColor:SKColor, withTolerance:CGFloat) -> Bool {
        
        var r1: CGFloat = 0.0, g1: CGFloat = 0.0, b1: CGFloat = 0.0, a1: CGFloat = 0.0;
        var r2: CGFloat = 0.0, g2: CGFloat = 0.0, b2: CGFloat = 0.0, a2: CGFloat = 0.0;
        
        color.getRed(&r1, green: &g1, blue: &b1, alpha: &a1);
        withColor.getRed(&r2, green: &g2, blue: &b2, alpha: &a2);
        
        return fabs(r1 - r2) <= withTolerance &&
            fabs(g1 - g2) <= withTolerance &&
            fabs(b1 - b2) <= withTolerance &&
            fabs(a1 - a2) <= withTolerance;
    }
   


Everything works so far. But depening on the image, I have a big problem - the time for processing the image. As you can see, the functionality is running within the main thread. I think, it definately makes sense to have a seperate thread for that. But even with threads, it took too much time. Is there any better way to run this function "pretty fast"?


'replaceColor' is everytime called, when there is a value change on a UI slider object. For user experience, it is really not good.

    @objc func toleranceSlider_changed() {
        let deadlineTime = DispatchTime.now() + .seconds(3)
        DispatchQueue.main.asyncAfter(deadline: deadlineTime) {
            //Perform code here
            self.imageView.image = self.colorUtil.replaceColor(color: (self.COLOR_TO_BE_RELACED), withColor: (self.SELECTED_COLOR), image: (self.imageView.image!), tolerance: CGFloat((self.toleranceSlider.value)), alpha: CGFloat((self.alphaSlider.value)))
        }
    }


Thanks in advance!

Accepted Reply

Purpose is to replace a certain pixel color with another one by including a specific tolcerance value.

Doing this byte-by-byte in your own code is possible but it’s never going to be fast. I recommend that you look at one of two frameworks:

  • Accelerate > vImage — This supports a vast array of image manipulation operations, all done on the CPU.

  • Core Image — This supports a vast array of image manipulation operations, all of which are typically done on the GPU.

I think the specific operation you’re trying to do can be done using built-in vImage or Core Image primitives (although I’m a Networking Guy™, so I’m not to be trusted when it comes to graphics :-) but, if not, Core Image supports a custom kernel mechanism where you can write your own image manipulation code and have Core Image run it over each pixel automatically.

These techniques will be much faster than code you write yourself, especially if you can shift the work to the GPU. And by much I mean several orders of magnitude.

Share and Enjoy

Quinn “The Eskimo!”
Apple Developer Relations, Developer Technical Support, Core OS/Hardware

let myEmail = "eskimo" + "1" + "@apple.com"

Replies

Not sure if that's where it takes time, and if the change would be enough to improve, but line 52:


why do you create a SKColor,

                let currentColor = SKColor(red: red, green: green, blue: blue, alpha: alpha);

and not change your compareColor

to work directly with color components in order to avoid multiple conversions ?


            while byteIndex < bitmapByteCount {
               
                // Get color of current pixel
                let red:CGFloat = CGFloat(rawData[byteIndex + 0])/255
                let green:CGFloat = CGFloat(rawData[byteIndex + 1])/255
                let blue:CGFloat = CGFloat(rawData[byteIndex + 2])/255
                let alpha:CGFloat = CGFloat(rawData[byteIndex + 3])/255
               
                // NO MORE let currentColor = SKColor(red: red, green: green, blue: blue, alpha: alpha);
               
                // Compare two colors using given tolerance value
                // USE a different compare
                //     if self.compareColor(color: color, withColor: currentColor , withTolerance: tolerance) {
                if self.compareColorComponents(red: red, green: green; blue: blue, alpha: alpha, withRed: r2, withGreen: g2, withBlue: b2, withAlpha: a2 , withTolerance: tolerance) {

                    // If the're 'similar', then replace pixel color with given target color
                    rawData[byteIndex + 0] = r2
                    rawData[byteIndex + 1] = g2
                    rawData[byteIndex + 2] = b2
                    rawData[byteIndex + 3] = a2
                }
               
                byteIndex = byteIndex + 4;
            }


    func compareColorComponents(red: CGFloat, green: CGFloat; blue: CGFloat, alpha: CGFloat, withRed: CGFloat, withGreen: CGFloat, withBlue: CGFloat, withAlpha: CGFloat , withTolerance: CGFloat) -> Bool {

        return fabs(red - withRed) <= withTolerance &&
            fabs(green - withGreen) <= withTolerance &&
            fabs(blue - withBlue) <= withTolerance &&
            fabs(alpha - withAlpha) <= withTolerance;
    }

I think, it definately makes sense to have a seperate thread for that.

If you want the main thread not to be blocked, using another thread makes sense. But if you want to accelerate the actual calculation time, it has no meaning...


Is there any better way to run this function "pretty fast"?

If you mean this function literally and strictly, the answer is no. You need to update your code to make it run faster.


Your code has some room to be tuned up (for example, calling non-private instance method is not a good way), but that would affect a little.


Please explain how much do you expect with saying pretty fast. How long does it take your function to finish for an image? (Please do not forget to show the size of the image.)


2x is faster enough? How about 4x or you want more?

Purpose is to replace a certain pixel color with another one by including a specific tolcerance value.

Doing this byte-by-byte in your own code is possible but it’s never going to be fast. I recommend that you look at one of two frameworks:

  • Accelerate > vImage — This supports a vast array of image manipulation operations, all done on the CPU.

  • Core Image — This supports a vast array of image manipulation operations, all of which are typically done on the GPU.

I think the specific operation you’re trying to do can be done using built-in vImage or Core Image primitives (although I’m a Networking Guy™, so I’m not to be trusted when it comes to graphics :-) but, if not, Core Image supports a custom kernel mechanism where you can write your own image manipulation code and have Core Image run it over each pixel automatically.

These techniques will be much faster than code you write yourself, especially if you can shift the work to the GPU. And by much I mean several orders of magnitude.

Share and Enjoy

Quinn “The Eskimo!”
Apple Developer Relations, Developer Technical Support, Core OS/Hardware

let myEmail = "eskimo" + "1" + "@apple.com"

Thanks a lot eskimo. I will check the sources out and see how I can proceed 🙂 I really like this forum. Always getting some very good ideas and hints.

Hi Claude31,


that is probably a very good hint. I would like to checkout the sources provided by eskimo and see how I can proceed. 🙂

Hi OOPer,


you were right. I haven't provided more detailed information about the "pretty fast" topic 🙂 Currently, it is the processesing speed is acceptable if you upload a small/medium sized image like a gif etc. But as soons as I upload taken photos from the album then it takes about 15 sec. to finish the processing which is too long.

I guess, I follow the instructions provided by eskimo. Thanks 🙂