Crash when enabling jittering on SCNRenderer

It's telling you that your texture descriptor settings are incorrect relative to your other settings.

Notice how the number is exactly 1/2. Based on the limited information you wrote, it is expected because you went from BGRA8 to RGBA16, which is twice the size.

Update: Thanks for the note, @MoreLightning. If you're curious, there was a link to my repository reproducing this behavior in the post. Here it is.

I've been at this all day. No solution. Here's what I tried

Attempt: Change the texture descriptor

Since I'm creating my Metal texture from a CVPixelbuffer with CVMetalTextureCacheCreateTextureFromImage, I can't figure out how to set its attributes - including the pixel format.

Attempt: Change the render pipeline's pixel format

I created a MTLRenderPipelineState and set the pixel format on that to match, but it was ignored. I think it's because of CVMetalTextureCacheCreateTextureFromImage

Attempt: Try H264

Didn't change anything. Also tried changing just the alpha quality, with HEVC with alpha, but no change.

Backing up a bit, I thought maybe isJitteringEnabled is not the way to get rid of jagged edges.

Attempt: Enable multi sampling

I was able to get my pipeline to pick up that I wanted multi sampling, but it crashes due to the texture not being set up for multisampling (more precisely a MTLTexture of type [.2DMultisample] (docs)

And then we're back at the problem of not being able to set the attributes for the MTLTexture.

Attempt: Copy the MTLTexture created by Core Video

I tried to use a MTLBlitCommandEncoder to copy the texture I was given by Core Video into a texture I had set up with the right attributes. But it crashes telling me that the attributes don't match.

I'm starting to think there's no solution to this?

I missed your link earlier...

I'll take a look tonight, did you first double check that isJitteringEnabled is in fact working as of today in a scenekit project without the rest of that sample project?

SceneKit is notorious for having bugs. So you should confirm that first. Then you should try changing Scenekit to rgba16 without the rest of that sample project. If you run into the same error, you'll know that you still need to change the texture settings.

I just checked the project files...

It is as I stated in my first reply, you need to ensure the texture settings match, and then you will not get the error regarding the textures.

After you change to rgba16 in both places (such as kCVPixelFormatType_64RGBAHalf), you will then be confronted by another texture setting match problem for the texture in the second attachment. Since you are overriding the render pass descriptor for SceneKit, it would be logically assumed that you then have to create and setup that texture on your own. If you spend more time with Metal, these things will be more obvious as they are very normal things.

I will leave that to you to complete, but I will say that the entire thing is clunky, extremely inefficient, and I would not personally recommended following this sample code's strategy or even using SceneKit or Swift. The use of AVAssetWriter is also not recommended. These, in their current form today and in the past, are Apple's half-baked attempts at providing convenience with a heavy price. There is little expectation based on Apple's prior results, that the standards of quality will change anytime soon. (Even though I hope for that every day)

I would instead recommend avoiding as many Apple frameworks as you can, and becoming more familiar with Metal and C, and get to your goals that way.

For people wondering how to actually set up the multipass texture
https://stackoverflow.com/questions/64958392/antialiasing-a-scenekit-rendering-with-metal