I am using SCNTechnique in combination with ARSCNView. The technique is doing so minor post-processing. I have written several filter variant for this post-processing, but I'm facing an issue when with one of the filters/fragment shaders, SCNTechnique discards my output and just presents the plain camera feed on screen instead. This is clearly visible in the Metal pipeline, using the GPU frame debugger. Let me stress that my setup works for 90% of my filters, but not this one and I want to know why. iOS 18.1, iPhone 13 Mini. Xcode 16.1. Encoder 0 & 1 are injected by the system. Render encoder 2 & 3 correspond to my SCNTechnique's render passes: one to manipulate pixel data (darken it in this case) and another to BLIT it back to the main texture. I know the separate buffer is not strictly for this particular operation, but it shouldn't matter. Note that the issue occurs in encoder 4 (not mine but ARKit's). In Render Encoder 4, scn_postprocess_AR_fragment handle my texture (#0, ending in f980) and another from the camera feed (Texture 2). I know this pass is typically used for grain because that's what it used to do before I disabled grain on ARSCNView (+ the buffer still contains grain paramaters). I have other post-processing filters that work just fine. By what magic is ARKit determining to use Texture 2 instead of my Texture 0? Sure, I could keep digging into the minute differences between my shaders to find out which LoC affects how some ARKit shader down the line operates, but it's awfully opaque so far.

I want to apply a SCNTechnique pipeline to the camera feed. To achieve this, I want to bring the camera input into the SceneKit world. The perfects API seems to be: let captureDevice = … scnScene.background.contents = captureDevice This is demonstrated in "SceneKit: What's New" (WWDC17) (at 44m19s) and is mentioned in the documentation of SCNMaterialProperty's contents. Instead of showing camera feed, it crashes with these messages: *** Terminating app due to uncaught exception 'NSInvalidArgumentException', reason: '*** -[AVCaptureVideoDataOutput setVideoSettings:] Unsupported pixel format type - use -availableVideoCVPixelFormatTypes' *** First throw call stack: (0x18993c7cc <REDACTED> 0x211e18488) libc++abi: terminating due to uncaught exception of type NSException Please advise. STEPS TO REPRODUCE Create a new Xcode project, starting from the SceneKit game template. Add Info.plist entry for NSCameraUsageDescription. Add a capture device property to GameViewController: class GameViewController: UIViewController { let captureDevice = AVCaptureDevice.default(for: .video) Set the background contents: scene.background.contents = captureDevice Run the app on device. PLATFORM AND VERSION iOS Development environment: Xcode 16.1, macOS 15.0.1. Run-time configuration: iOS 18.1

For the past two days I've been trying to use stencil buffers with SCNTechnique. I feel like I've tried everything and I still have a lot of open questions. Usually I gather information from other people's questions, but posts on this topic appear few and far between. I hope someone here can point me in the right direction.Running on macOS 10.15.3, Xcode 11.3.1, using Metal. Using basic SceneKit template, modified slightly.I'm trying to achieve a portal effect with a couple of passes:Render a single plane that acts as the portal, into a stencil buffer.Render the inside of the portal, clipped by the stencil buffer.Render the outside of the portal.Mix the two color + depth buffers together (using a simple custom Metal shader).For steps 1-3 I don't need a custom program/shaders, for step 4 I've written a custom shader that mixes two inputs.There's three objects in my scene: ship (bitmask 8), plane (bitmask 4), sphere (bitmask 2). The sphere shold only be visible through the stencil of the plane, like a portal.From what I've learned from SCNTechnique documentation and OpenGL paradigms, the stencil buffer offers the functionality I'm looking for.The technique as attached below does not pass Metal API validation, complaining that "validateDepthStencilState:3857: failed assertion `MTLDepthStencilDescriptor uses frontFaceStencil but MTLRenderPassDescriptor has a nil stencilAttachment texture'".I can change "stencilStates.enable" to `false` in the "inner" pass to at least make it run, but it appears the the stencil buffer is not fed correctly into the "inner" pass and cropping/stencilling does not occur. When I look at the GPU frame capture, the "portal_stencil" pass output is marked with a purple exclamation mark.Since the "inner" pass does not need to write to the stencil buffer (only test against it), I think "stencilStates.enable" should be `false`, right?(Q) How do I get the "inner" pass to respect/use the stencil buffer? Do I even need separate stencil buffer, or should I be able to use the default stencil buffer?This technique does use a stencil buffer but does not specify its own inside `targets` like I'm attempting. It uses custom shaders for every pass, which I'd like to avoid unless it's necessary. I haven't found yet how to write a passthrough shader in Metal (a shader that follows default behavior).Conceptually, I think should also be able to render the "inner" pass immediately into the "outer" color buffer while leveraging depth testing (against outer buffer) and stencil testing (against portal buffer). So [outer -&gt; portal_stencil -&gt; inner]. That would reduce the draw passes to three. (Q) Am I thinking the right way?Thanks a lot in advance, I'm really struggling with this.{ "passes": { "outer": { "draw": "DRAW_SCENE", "outputs": { "color": "color_outer" }, "excludeCategoryMask": 6 }, "portal_stencil": { "draw": "DRAW_SCENE", "outputs": { "stencil": "stencil_scene" }, "stencilStates": { "clear": true, "enable": true, "behavior": { "depthFail": "zero", "fail": "zero", "pass": "replace", "function": "always", "referenceValue": 255, "writeMask": 1 } }, "includeCategoryMask": 4, "excludeCategoryMask": 10 }, "inner": { "draw": "DRAW_SCENE", "inputs": { "stencil": "stencil_scene" }, "outputs": { "color": "color_inner" }, "colorStates": { "clear": true }, "stencilStates": { "enable": true, "behavior": { "depthFail": "keep", "fail": "keep", "pass": "keep", "function": "equal", "referenceValue": 255, "readMask": 1, "writeMask": 0 } }, "includeCategoryMask": 2, "excludeCategoryMask": 12 }, "mix": { "draw": "DRAW_QUAD", "metalVertexShader": "mixVertex", "metalFragmentShader": "mixFragment", "inputs": { "colorInner": "color_inner", "colorOuter": "color_outer" }, "outputs": { "color": "COLOR" } } }, "sequence": [ "portal_stencil", "inner", "outer", "mix" ], "targets": { "color_outer": { "type": "color" }, "color_inner": { "type": "color" }, "stencil_scene": { "type": "stencil" } }, "symbols": { } }