How to convert local video into CMSampleBuffer for unit tests

Hi,

I'm working on an app that uses the live camera output to perform text recognition.

In order to improve the performance , I want to write some unit tests to check how well the text recognition works over time. I want to load a video from my app bundle, and use that as the 'camera output' so that I can write tests around what should and what should not get recognized.

I'm trying to find out how to generate a CMSampleBuffer from a video so I can feed that into my Vision request. Any ideas? Thanks!
Up vote post of jordi Down vote post of jordi
1.7k views
Posted by
jordi
Reply
Add a Comment

Accepted Reply

Another option, if you don't need the frames in real-time, is to use AVAssetReader.
Posted by
Media Engineer
Up vote reply of Media Engineer Down vote reply of Media Engineer
Post marked as solved
Add a Comment

Replies

Take a look at Action & Vision sample code. The CameraViewController class supports two modes: live camera capture and reading form the video file.
The idea is to use AVPlayerItemVideoOutput together with CADisplayLink - this will closely simulate the live camera capture scenario.
The actual conversion of CVPixelBuffer to CMSampleBuffer happens inside CADisplayLink callback.
See handleDisplayLink(_ displayLink: CADisplayLink) function.


Posted by
Frameworks Engineer
Post not yet marked as solved Up vote reply of Frameworks Engineer Down vote reply of Frameworks Engineer
Add a Comment
Another option, if you don't need the frames in real-time, is to use AVAssetReader.
Posted by
Media Engineer
Up vote reply of Media Engineer Down vote reply of Media Engineer
Post marked as solved
Add a Comment
Thanks to both of you for the suggestions! Got a first version up and running!
Posted by
jordi
Up vote reply of jordi Down vote reply of jordi
Add a Comment
Oops, I wanted to choose the other answer as 'the answer'. Can't undo it now, sorry!
Posted by
jordi
Up vote reply of jordi Down vote reply of jordi
Add a Comment
Could you, please, share how you finally mocked buffers in tests? Have you used handleDisplayLink or AVAssetReader?
Posted by
alex-any
Up vote reply of alex-any Down vote reply of alex-any
Add a Comment