Image Caching and Actors

Question

Created Jun ’21

Replies 1

Boosts 0

Views 1.5k

Participants 2

I was fortunate enough to talk to two very knowledgeable Apple Engineers in a lab tonight and they helped me through my refactor for image caching using actors. In this case I want to be able to access the images directly on the main thread for use in the UI but have all image downloading and resizing happen on background threads. Briefly, it looked something like this:

@MainActor
class OsuImageCache {
	init() { ... }	

	/// Grab an image on the main thread for use in collection view cells
	func image(for url: URL) -> UIImage? { }

	/// Cache an image on a background thread
	func cacheImage(for url: URL) async throws -> (image: UIImage?, thumbnail: UIImage?) {
		let (data, _) = try await session.data(from: url)

        async let processedImages = await processImageData(data, for: url)
        
        try saveInFileSystem(imageData: processedImages.imageData, thumbnailImageData: processedImages.thumbnailData, for url: URL)
        
        return (UIImage(data: processedImages.imageData), UIImage(data: processedImages.thumbnailData))
	}
	
	/// Resize images and create thumbnails
	func processImageData(_ data: Data, url: URL) async -> (UIImage, UIImage) {
	
		// Code that resizes the image and the thumbnail goes here. It's all inline, no completion handlers nor async calls...
		...
		// Finally, return
		
		return (finalProcessedImageData, finalProcessedThumbnailData)
	}
}

When we were talking, what I expected to have happen was the line with async let images = await processImageData(data, for: url) was supposed to kick off to a background thread and do the processing of the image for me. I happily disconnected with them thinking it was solved, but when I ran the app afterwards I noticed to my dismay that the image processing was still happening on the main thread.

At this point I assume that I need to try something different, and I refactor by adding in a separate actor for image processing like so:

@MainActor
class OsuImageCache {

	/// Actor just for image processing and thumbnail generation
	let imageProcessing = ImageProcessing()

	init() { ... }
	
	func image(for url: URL) -> UIImage? { }
	
	func cacheImage(for url: URL) async throws -> (image: UIImage?, thumbnail: UIImage?) {
		let (data, _) = try await session.data(from: url)

        async let processedImages = await imageProcessing.processImageData(data, for: url)
        
        try saveInFileSystem(imageData: processedImages.imageData, thumbnailImageData: processedImages.thumbnailData, for url: URL)
        
        return (UIImage(data: processedImages.imageData), UIImage(data: processedImages.thumbnailData))
	}
}

actor ImageProcessing {
    func processImageData(_ data: Data, url: URL) async -> (UIImage, UIImage) {
	
		// Code that resizes the image and the thumbnail goes here. It's all inline, no completion handlers nor async calls...
		...
		// Finally, return
		
		return (finalProcessedImageData, finalProcessedThumbnailData)
	}
}

When I run it this time, I still see that the image processing within the ImageProcessing actor is still on the main thread. This is not what I expected. I flipped between different session videos and then changed the ImageProcessingActor like so:

actor ImageProcessing {
    func processImageData(_ data: Data, url: URL) async -> (UIImage, UIImage) {
	
		typealias ImageContinuation = CheckedContinuation<(Data, Data), Never>
		
		return await withCheckedContinuation { (continuation: ImageContinuation) in 
			async {
				// Code that resizes the image and the thumbnail goes here. 
				...
				// Finally, return
		
				continuation.resume(returning: (finalProcessedImageData, finalProcessedThumbnailData))
			}
		}
		
	}
}

At this point it works! Image resizing happens on background threads and I still have the ability to use the images on the main thread! What I'm curious to know is if this is considered an OK pattern, or if I just stumbled by chance into something that worked but isn't actually the intended path. The checked continuation + async combo gets me the behavior I want, but I'm not sure if it's clean.

Is there a way to simplify this further and remove any of this boilerplate?

Thanks for any input you can give me!

Boost

Answer 1

DrewFitz OP

Jun ’21

Is there any state that's being read/written to in the ImageProcessing actor? If not then I don't know if you should use an actor at all unless whatever processImageData does requires exclusive execution.

As for calling that code on the background from the main actor, you need to use asyncDetached because regular async will inherit the actor from the current execution context (the MainActor in this case).

I think this is the code you're aiming for:

@MainActor
class OsuImageCache {
	init() { ... }	

	/// Grab an image on the main thread for use in collection view cells
	func image(for url: URL) -> UIImage? { }

	/// Cache an image on a background thread
	func cacheImage(for url: URL) async throws -> (image: UIImage?, thumbnail: UIImage?) {
		let (data, _) = try await session.data(from: url)

    let processTask = asyncDetached(priority: /* use whatever you want like .userInitiated or .utility */) {
       return processImageData(data, for url)
    }

    let processedImages = await processTask.get()
        
    try saveInFileSystem(imageData: processedImages.imageData, thumbnailImageData: processedImages.thumbnailData, for url: URL)
        
    return (UIImage(data: processedImages.imageData), UIImage(data: processedImages.thumbnailData))
	}
	
	/// Resize images and create thumbnails
  /// NOT ASYNC
	func processImageData(_ data: Data, url: URL) -> (UIImage, UIImage) {
	
		// Code that resizes the image and the thumbnail goes here. It's all inline, no completion handlers nor async calls...
		...
		// Finally, return
		
		return (finalProcessedImageData, finalProcessedThumbnailData)
	}
}

0