I was trying to migrate Core Image based code that's rotating an image in a CVPixelBuffer to the newer VTPixelRotationSession from Video Toolbox. Hoping to increase performance.
The original code does:
let rotatedImage = CIImage(cvPixelBuffer: origPixelBuffer).oriented(.left)
context.render(rotatedImage, to: newPixelBuffer)
The new code uses a session:
_ = VTPixelRotationSessionRotateImage(rotationSession, origPixelBuffer, newPixelBuffer)
However I immediately ran into memory limitations, since my code has to be able to run in an iOS extension. It seems VTPixelRotationSessionRotateImage easily lets memory usage spike over the 50MB of allowed memory. While the CIImage based implementation has no such high memory usage at all.
Is this expected? Does the VTPixelRotationSession implementation gain more performance by sacrificing memory? Or is there something I'm overlooking?
I was expecting the VTPixelRotationSession at worst to be on par in terms of memory usage and processing speed compared to CIImage. At this moment it seems VTPixelRotationSession is unusable in extensions.
See also Feedback: FB14977240
Post
Replies
Boosts
Views
Activity
We have an app with a broadcast extension with a RPBroadcastSampleHandler. The implementation is working fine, however for quite some users the extension suddenly crashes during the broadcast.
The stacktrace stacktrace of the crashing thread always looks like the shortened sample below. (Full crash reports and stack traces are attached to the submitted Feedbacks.) Looking at the stacktrace none of our code is running, just ReplayKit code handling XPC messages at that moment:
Thread:
#0 0x00000001e2cf342c in __pthread_kill ()
#1 0x00000001f6a51c0c in pthread_kill ()
#2 0x00000001a1bfaba0 in abort ()
#3 0x00000001a9e38588 in malloc_vreport ()
#4 0x00000001a9e35430 in malloc_zone_error ()
[...]
#18 0x0000000218ac91bc in -[RPBroadcastSampleHandler processPayload:completion:] ()
#19 0x0000000198b81360 in __NSXPCCONNECTION_IS_CALLING_OUT_TO_EXPORTED_OBJECT_S2__ ()
Is anyone aware of there issues with ReplayKit? Are there known workarounds? Could anything we're doing affect crashes like this?
Would greatly appreciate it if anyone from Apple DTS could look into this and flag the below Feedbacks to the relevant teams!
Feedback IDs: FB13949098, FB13949188