Hello everyone,
I 'm doing a AR Medical Solution project. and have some questions.
We can get depth frame, color frame, camera transform matrix and camera projection matrix per every frame.
For my comfort, I render on scnview using camera matrix and projection matrix.
At a point I want to render simutaneously refresh color frame and camera matrices belonging to one frame, but it wasn't.
maybe drawcalls be conducted different times
It's my code and can you advice for me?
scnview.delegate = self
...
@MainActor func renderer(_: SCNSceneRenderer,
didRenderScene _: SCNScene,
atTime _: TimeInterval)
{
if let camTransform = lastFrame.camTransform,
let camProjection = lastFrame.camProjection,
let colorFrame = lastFrame.colorFrame
{
mars.updateCamera(cameraPose: camTransform,
cameraGLProjection: camProjection)
// update camera transforms at scenekit
render(colorFrame: colorFrame)
}
}
func render(colorFrame: STColorFrame) {
let buffer = CMSampleBufferGetImageBuffer(colorFrame.sampleBuffer)!
convertVideoFrameToImage(buffer)
// ycbcr buffer to rgba image texture by metal computing shader
scnview.scene?.background.contents = outTexture
}