I am writing a RTSP client with Flutter SDK that is also using the Apple VideoToolKit API for the HW decoder. I will refrain from posting code at the moment because I think it would just be easier to explain. I'm using wireshark to inspect contents and it seems parsing is working correctly. My problem is I can't get the data in the right format for the decoder, hence getting OSStatus (Swift) error code -8969. If someone could please clarify the AVCC format along with the inner payload format expected by the decoder, that would be great. My decoding is done for one frame at a time. So for each frame I create a new decoding session. The SPS and PPS value is set to a static value on program startup, and then updated once the server begins sending RTP. I don't parse the sprop-parameter-sets value at the moment, I will add this at a later time. The below buffer will be the resulting AVCC format for a FU-A RTP payload with 3 slices. Please let me know if there is anything I get wrong here. The 4 byte length is big endian representation. When I create the decoding session in Swift, I consider all of this to be 1 sample. [4 byte length][FU identifier slice 1][FU header slice 1][NAL Unit payload slice 1][4 byte length][FU identifier slice 2][FU header slice 2][NAL Unit payload slice 2][4 byte length][FU identifier slice 3][FU header slice 3][NAL Unit payload slice 3] The length is: length = RTSP length field - RTP_HEADER_LEN Where the RTP_HEADER_LEN is equal to 12 bytes. Any guidance appreciated, thank you. Update: I removed the FU header/FU identifier from the buffer, and am now getting 2 successful decodes out of hundreds of frames, but am still getting OSStatus codes: -666, -8969. and -12348. If anyone could explain any of these to me that would be helpful. Thanks.

Hello, I have been writing an RTSP client for an app and I'm at the point where I need to play a video of the images output from the HW decoder. What would be the best method to play a series of images, say an array of 30-60 images displayed for the duration of 1 second? I looked at the AVPlayer class and it doesn't provide the ability I need. Ideally I would like to provide some buffer that the class could reference and it would just read the buffer and program the display. The issue is I am decoding the video on the client system, and the APIs do not have the necessary parameters. It seems like AVPlayer does all the work for a given URL, but I don't think it supports RTSP/RTP. One possible solution is to use UIImageView to display an array of images. But I'm not sure if this is the correct usage of the class. I am still new to image formats and so on, so I'm not sure if there would be unnecessary overhead for converting CVImageBuffer (output of HW decoder) to UIImage. If this sounds like an idea that would work, please let me know. If anyone knows of any helpful classes that could solve this use case, please let me know. Any help is appreciated, thank you.

Hello, I am new to Swift and have a specific requirement needing to be implemented through the VideoToolbox API. I am using MethodChannel() from Flutter SDK to invoke Swift functions. I need access to the HW video decoder and the plan is to receive encoded frames from flutter application, decode with VideoToolbox API, then return the decoded image back through the MethodChannel(). I am having trouble checking if the session succeeds in decoding a frame. The callback given to VTDecompressionSessionDecodeFrame() via the VTDecompressionOutputCallbackRecord class may only return a void. However, this callback recieves as argument the decoded image buffer and status, which I have no way of accessing at the moment; this is essentially my problem. I tried some method such as creating a variable at a higher level scope but I get compiler error A C function pointer cannot be formed from a closure that captures context. It was also quite frustrating that I simply cannot get the base address of a variable like in C/C++ so I can perform memcpy(), but that is another topic. So my question is, how may I return/access data within a closure that may only return void? Are there any helpful classes to facilitate communication between the closure and function calling VTDecompressionSessionDecodeFrame(). My idea would be to have the calling function wait until the value of the image buffer or status changes. This doesn't sound like a good implementation but it's only what I can think of at the moment. Any help would be appreciated. Thank you.