How to pass right model view projection matrix from scenekit to metal shader?

Hello, Everyone.

I try to use a metal kit with a scene kit. Because, the scene kits scene graph is great, I want to implement a low-level metal shader.

I want to use SCNNodeRenderDelegate, without SCNProgram. Because I want low-level implement for example passing custom extra MTLBuffer, or multi-pass-rendering.

So I pass model view projection matrix like that,

in metal shader

struct NodeBuffer {
  float4x4 modelTransform;
  float4x4 modelViewProjectionTransform;
  float4x4 modelViewTransform;
  float4x4 normalTransform;
  float2x3 boundingBox;
};

in Swift code

    struct NodeMatrix: sizeable {
    var modelTransform = float4x4()
    var modelViewProjectionTransform = float4x4()
    var modelViewTransform = float4x4()
    var normalTransform = float4x4()
    var boundingBox = float2x3()
  }

   ...
   private func updateNodeMatrix(_ camNode: SCNNode) {
    guard let camera = camNode.camera else {
      return
    }
     
    let modelMatrix = transform
    let viewMatrix = camNode.transform
    let projectionMatrix = camera.projectionTransform
     
    let viewProjection = SCNMatrix4Mult(viewMatrix, projectionMatrix)
    let modelViewProjection = SCNMatrix4Mult(modelMatrix, viewProjection)
    nodeMatrix.modelViewProjectionTransform = float4x4(modelViewProjectionMatrix)
  }
...

public func renderNode(_ node: SCNNode,
              renderer: SCNRenderer,
              arguments: [String: Any])
  {
    guard let renderTexturePipelineState = renderTexturePipelineState,
       let renderCommandEncoder = renderer.currentRenderCommandEncoder,
       let camNode = renderer.pointOfView,
       let texture = texture
    else { return }
     
    updateNodeMatrix(camNode)
    guard let nodeBuffer
      = renderer.device?.makeBuffer(bytes: &nodeMatrix,
                     length: NodeMatrix.stride,
                     options: [])
    else { return }
     
    renderCommandEncoder.setDepthStencilState(depthState)
    renderCommandEncoder.setRenderPipelineState(renderTexturePipelineState)
    renderCommandEncoder.setFragmentTexture(texture, index: 0)
    renderCommandEncoder.setVertexBuffer(vertexBuffer, offset: 0, index: 0)
    renderCommandEncoder.setVertexBuffer(nodeBuffer, offset: 0, index: 1)
    renderCommandEncoder.drawIndexedPrimitives(type: .triangle,
                          indexCount: indexCount,
                          indexType: .uint16,
                          indexBuffer: indexBuffer,
                          indexBufferOffset: 0)
  }

But I got the wrong model view projection matrix in the shader. I think scene kit has modify intermediate transform hiding. I can't know, help me...

Answered by wonkieun in 702104022

Sorry, I did it. problem is in this code (https://stackoverflow.com/questions/33730468/how-to-convert-camera-extrinsic-matrix-to-scncamera-position-and-rotation)

   private func updateNodeMatrix(_ camNode: SCNNode) {
    guard let camera = camNode.camera else {
      return
    }
     
    let modelMatrix = transform
    let viewMatrix = SCNMatrix4Invert(camNode.transform) // invert is right
    let projectionMatrix = camera.projectionTransform
     
    let viewProjection = SCNMatrix4Mult(viewMatrix, projectionMatrix)
    let modelViewProjection = SCNMatrix4Mult(modelMatrix, viewProjection)
    nodeMatrix.modelViewProjectionTransform = float4x4(modelViewProjection)
  }
Accepted Answer

Sorry, I did it. problem is in this code (https://stackoverflow.com/questions/33730468/how-to-convert-camera-extrinsic-matrix-to-scncamera-position-and-rotation)

   private func updateNodeMatrix(_ camNode: SCNNode) {
    guard let camera = camNode.camera else {
      return
    }
     
    let modelMatrix = transform
    let viewMatrix = SCNMatrix4Invert(camNode.transform) // invert is right
    let projectionMatrix = camera.projectionTransform
     
    let viewProjection = SCNMatrix4Mult(viewMatrix, projectionMatrix)
    let modelViewProjection = SCNMatrix4Mult(modelMatrix, viewProjection)
    nodeMatrix.modelViewProjectionTransform = float4x4(modelViewProjection)
  }
How to pass right model view projection matrix from scenekit to metal shader?
 
 
Q