Post

Replies

Boosts

Views

Activity

H264 videotoolbox wrong decoding
I am trying to decode my h264 stream by using only c++ (not obj-c and swift). I wrote decoding part by following this stackoverflow post. According to status the decoding is successfully completed. But I saw that most of decoded data are zero when check the data. This shoow that decoding is wrong or not completed. I think, I made a mistake at decoding initializing stage. I have added my code to below. VTDecompressionOutputCallbackRecord cb{}; cb.decompressionOutputCallback = &VisionModuleHandler::decode_cb; cb.decompressionOutputRefCon = NULL; CMFormatDescriptionRef format; CMBlockBufferRef block = NULL; CMSampleBufferRef buffer = NULL; VTDecodeInfoFlags flags; OSStatus status; /* Initialize the decoder. */ std::vector<uint8_t> sps(encoded_frame.sps_pps_frame.begin()+4, encoded_frame.sps_pps_frame.begin()+20 ); std::vector<uint8_t> pps(encoded_frame.sps_pps_frame.begin()+24, encoded_frame.sps_pps_frame.end() ); std::size_t sps_pps_size_array [] = {16,5}; const size_t sampleSizeArray {encoded_frame.encoded_frame.size()}; const uint8_t* const parameterSetPointers[2] = {sps.data(), pps.data()}; status = CMVideoFormatDescriptionCreateFromH264ParameterSets(kCFAllocatorDefault, 2, parameterSetPointers, sps_pps_size_array, 4, &format); if (status != noErr) spdlog::info("Error while decomprasion - 1"); status = VTDecompressionSessionCreate(kCFAllocatorDefault, format, NULL, NULL, &cb, &session); if (status != noErr) spdlog::info("Error while decomprasion - 2"); status = CMBlockBufferCreateWithMemoryBlock(kCFAllocatorDefault, (void*)(encoded_frame.encoded_frame.data()), encoded_frame.encoded_frame.size(), kCFAllocatorNull, NULL, 0, encoded_frame.encoded_frame.size(), 0, &block); if (status != noErr) spdlog::info("Error while decomprasion - 3"); status = CMSampleBufferCreate(kCFAllocatorDefault, block, true, NULL, NULL, format, 1, 0, NULL, 1, &sampleSizeArray, &buffer); if (status != noErr) spdlog::info("Error while decomprasion - 4"); /* Loop over compressed data; our callback will be called with * each decoded frame buffer. Passed flags make this asynchronous. */ status = VTDecompressionSessionDecodeFrame(session, buffer, 0, NULL, 0 ); if (status != noErr) spdlog::info("Error while decomprasion - 5"); /* Flush in-process frames. */ status = VTDecompressionSessionFinishDelayedFrames(session); if (status != noErr) spdlog::info("Error while decomprasion - 6"); /* Block until our callback has been called with the last frame. */ status = VTDecompressionSessionWaitForAsynchronousFrames(session); if (status != noErr) spdlog::info("Error while decomprasion - 7"); I also added Callback part of decoding. if (imageType == CVPixelBufferGetTypeID()) { auto a = 1; } CVPixelBufferLockBaseAddress(imageBuffer, 0); OSType format = CVPixelBufferGetPixelFormatType(imageBuffer); void *baseaddress = CVPixelBufferGetBaseAddressOfPlane(imageBuffer,0); size_t width = CVPixelBufferGetWidth(imageBuffer); size_t height = CVPixelBufferGetHeight(imageBuffer); uint8_t *yBuffer = (uint8_t*)CVPixelBufferGetBaseAddressOfPlane(imageBuffer, 0); size_t yPitch = CVPixelBufferGetBytesPerRowOfPlane(imageBuffer, 0); uint8_t *cbCrBuffer = (uint8_t*)CVPixelBufferGetBaseAddressOfPlane(imageBuffer, 1); size_t cbCrPitch = CVPixelBufferGetBytesPerRowOfPlane(imageBuffer, 1); void *srcYpData = malloc(height *yPitch); memcpy(srcYpData, yBuffer, height *yPitch); void *srcCbCrData = malloc(height *cbCrPitch); memcpy(srcCbCrData, cbCrBuffer, height *cbCrPitch); size_t aRgbPitch = width * 4; uint8_t *aRgbBuffer = (uint8_t*)malloc(height *aRgbPitch); memset(aRgbBuffer, 0, height *aRgbPitch); vImage_Buffer srcYp = {srcYpData,height,width,yPitch}; vImage_Buffer srcCbCr = {srcCbCrData,height,width,cbCrPitch}; vImage_Buffer dest = {aRgbBuffer,height,width,aRgbPitch}; vImage_YpCbCrPixelRange pixelRange = {16,128,265,240,235,16,240,16}; vImage_YpCbCrToARGB infoYpCbCrToARGB = {}; vImage_Error error = vImageConvert_YpCbCrToARGB_GenerateConversion(kvImage_YpCbCrToARGBMatrix_ITU_R_601_4, &pixelRange, &infoYpCbCrToARGB, kvImage420Yp8_CbCr8, kvImageARGB8888, kvImageNoFlags); uint8_t permuteMap[4] = {0, 1, 2, 3}; error = vImageConvert_420Yp8_CbCr8ToARGB8888(&srcYp, &srcCbCr, &dest, &infoYpCbCrToARGB, permuteMap, 255, kvImageNoFlags); How can I fix this?
2
0
1.9k
Jan ’22