Post

Replies

Boosts

Views

Activity

Unprojecting color image into MeshAnchors
Like answered on previous question. [If you want to color the mesh based on the camera feed, you could do so manually, for example by unprojecting the pixels of the camera image into 3D space and color the according mesh face with the pixel's color.] Could you be able to give some hint how to solve this? I'm not really familiar with this concept.
1
0
929
Jun ’20
Arkit & hand pose detection
On Arkit project, in funtion - func session(_ session: ARSession, didUpdate frame: ARFrame) I tried to get - guard let observation = handPoseRequest.results?.first as? VNRecognizedPointsObservation else { return } and get - let thumb = try! observation.recognizedPoints(forGroupKey: .handLandmarkRegionKeyThumb) segmentation fault 11 pop up. Is this a bug ? or did I made any mistake?
0
0
1.2k
Jun ’20
ARKIT & HAND POSE RECOGNITION
Please give me some answer... On arkit, I put these codes... func session(_ session: ARSession, didUpdate frame: ARFrame) { let handler = VNImageRequestHandler(cvPixelBuffer: frame.capturedImage, orientation: .up, options: [:]) do {             try? handler.perform([handPoseRequest])             guard let observation = handPoseRequest.results?.first as?VNRecognizedPointsObservation else {return}             let thumb = try! observation.recognizedPoints(forGroupKey: .handLandmarkRegionKeyThumb)         } catch {} } At last part with let thumb.. segmentation fault 11 comes up. I saw a twit of video running this, but I tried this for week in any possible ways, I couldn't figure this out. Please, please give me some answer 😭
2
0
1.4k
Jul ’20
Not with Object Capture Session, I tried with RealityKit
With code below, I added color and depth image from RealityKit ARView, and ran Photogrammetry on iOS device, the mesh looks fine, but the scale of the mesh is quit different with real world scale. let color = arView.session.currentFrame!.capturedImage let depth = arView.session.currentFrame!.sceneDepth!.depthMap //😀 Color let colorCIImage = CIImage(cvPixelBuffer: color) let colorUIImage = UIImage(ciImage: colorCIImage) let depthCIImage = CIImage(cvPixelBuffer: depth) let heicData = colorUIImage.heicData()! let fileURL = imageDirectory!.appendingPathComponent("\(scanCount).heic") do { try heicData.write(to: fileURL) print("Successfully wrote image to \(fileURL)") } catch { print("Failed to write image to \(fileURL): \(error)") } //😀 Depth let context = CIContext() let colorSpace = CGColorSpace(name: CGColorSpace.linearGray)! let depthData = context.tiffRepresentation(of: depthCIImage, format: .Lf, colorSpace: colorSpace, options: [.disparityImage: depthCIImage]) let depth_dir = imageDirectory!.appendingPathComponent("IMG_\(scanCount)_depth.TIF") try! depthData!.write(to: depth_dir, options: [.atomic]) print("depth saved") And also tried this. let colorSpace = CGColorSpace(name: CGColorSpace.linearGray) let depthCIImage = CIImage(cvImageBuffer: depth, options: [.auxiliaryDepth : true]) let context = CIContext() let linearColorSpace = CGColorSpace(name: CGColorSpace.linearSRGB) guard let heicData = context.heifRepresentation(of: colorCIImage, format: .RGBA16, colorSpace: linearColorSpace!, options: [.depthImage : depthCIImage]) else { print("Failed to convert combined image into HEIC format") return } Does Anyone know why and how to fix this?
1
0
1k
Jun ’23
AVFoundation with lidar and this year's RealityKit Object Capture.
With AVFoundation's builtInLiDARDepthCamera, if I save photo.fileDataRepresentation to heic, it only has Exif and TIFF metadata. But, RealityKit's object capture's heic image has not only Exif and TIFF, but also has HEIC metadata including camera calibration data. What should I do for AVFoundation's exported image has same meta data?
2
0
1.3k
Jun ’23
PhotogrammetrySession and PhotogrammetrySample
In WWDC 2021, It saids 'we also offer an interface for advanced workflows to provide a sequence of custom samples. A PhotogrammetrySample includes the image plus other optional data such as a depth map, gravity vector, or custom segmentation mask.' But in code, PhotogrammetrySession initialize with data saved directory. How can I give input of PhotogrammetrySamples to PhotogrammetrySession?
0
0
598
Jul ’23