Hi everyone! I'm implementing a multithreaded approach on text recognition through Vision's VNSequenceRequestHandler and VNRecognizeTextRequest.
I've created multiple threads (let's use 3 for example) and created 3 instances of VNSequenceRequestHandler for each thread.
My AVSession sends me sample buffers (60 per second) and I'm trying to handle them one by one in 3 different threads. These threads constantly trying to consume sample buffers from my temporary sample buffer queue (1 to 3 sample buffers are in queue, they got deleted after handling). Sample buffers are not shared between these threads - one sample buffer is only for one thread's VNSequenceRequestHandler. For each performRequests operation I create a new VNRecognizeTextRequest.
By this I was trying to increase count of sample buffers handled per second.
But what I found out is no matter how many threads I've created (1 or 3), the speed is always about 10 fps (iPhone 13 Pro).
When I use 1 thread, only one instance of VNSequenceRequestHandler is created and used. In this case the [requestHandler performRequests:@[request] onCMSampleBuffer:sampleBuffer error:&error] takes about 100-150ms.
When I use 3 threads, each instance of VNSequenceRequestHandler takes up to 600ms to handle the request with [requestHandler performRequests:@[request] onCMSampleBuffer:sampleBuffer error:&error].
When I have 2 threads, the average time is about 300-400ms.
Does it mean that the VNSequenceRequestHandler inside of a Vision framework share some buffer's or request's queue so they're not able to work separately? Or maybe some single core of a GPU is used for detection?
I saw in the debug session window that the VNSequenceRequestHandler creates separate concurrent dispatch queues for handling the requests (for 2 instances 2 queues created), which in my opinion should not block the resources that much causing requests execution time grow 2 times.
Any ideas what causing the problem?