User Profile

I would like to extract depth data for a given point in ARSession.currentFrame.smoothedSceneDepth. Optimally this would end up looking something like: ARView.depth(at point: CGPoint) With the point being in UIKit coordinates just like the points passed to the raycasting methods. I ultimately to use this depth data to convert a 2D normalized landmark from a Vision image request into a 3D world space coordinate in the 3D scene - I only lack the accurate depth data for a given 2D point. What I have available is: The normalized landmark from the Vision request. Ability to convert this^ to AVFoundation coordinates. Ability to convert this^ to screen-space/display coordinates. When the depth data is provided correctly I can combine the 2D position in UIKit/screen-space coordinates with the depth (in meters) to produce an accurate 3D world position with the use of ARView.ray(through:) What I have not been able to figure out is how to get this depth value for this coordinate on screen. I can index the pixel buffer like this: extension CVPixelBuffer {     func value(for point: CGPoint) -> Float32 {         CVPixelBufferLockBaseAddress(self, .readOnly)         let width = CVPixelBufferGetWidth(self)         let height = CVPixelBufferGetHeight(self)          //Something potentially going wrong here.         let pixelX: Int = width * point.x         let pixelY: Int = height * point.y            let bytesPerRow = CVPixelBufferGetBytesPerRow(self)         let baseAddress = CVPixelBufferGetBaseAddress(self)! assert(kCVPixelFormatType_DepthFloat32 == CVPixelBufferGetPixelFormatType(depthDataMap))         let rowData = baseAddress + pixelY * bytesPerRow         let distanceAtXYPoint = rowData.assumingMemoryBound(to: Float32.self)[pixelX]                CVPixelBufferUnlockBaseAddress(self, .readOnly)         return distanceAtXYPoint     } } And then try to use this method like so:         guard let depthMap = (currentFrame.smoothedSceneDepth ?? currentFrame.sceneDepth)?.depthMap else { return nil } //The depth at this coordinate, in meters. let depthValue = depthMap.value(for: myGivenPoint) The frame semantics [.smoothedSceneDepth, .sceneDepth] have been set properly on my ARConfiguration. The depth data is available. If I hard-code the width and height values like so:         let pixelX: Int = width / 2         let pixelY: Int = height / 2 I get the correct depth value for the center of the screen. I have only been testing in portrait mode. But I do not know how to index the depth data for any given point.