Hey @brandonK212 Did you figure out a solution to this? Im trying to do something similar and running into similar problem. As a debugging step, I tried to draw detected rectangle on RealityKit's ARView. When drawing the detected rectangle as a sublayer of ARView, it seems to move away from the actual face as I slightly turn the device or my head(almost as if, responding to pitch, roll, yaw of the device/face) But the overall dimensions of the rectangle(especially the height of the rectangle) seems to cover the entire face as expected Im using displayTransform from ARView to adjust the boundingbox here's some code outlining the steps I'm doing: let viewPortRect = self.arView.frame let toArViewScaleTx = CGAffineTransform(scaleX: viewPortRect.size.width, y: viewPortRect.size.height) self.displayTransform = frame.displayTransform(for: interfaceOrientation, viewportSize: self.viewPortRect.size).concatenating(toArViewScaleTx) //... //drawObservations guard let faceRes = self.facePoseRequest.results?.first as? VNFaceObservation else { return } let arDisplayRect = faceRes.boundingBox.applying(self.displayTransform) //Draw rect let path = UIBezierPath(rect: arDisplayRect) self.bbLayer.path = path.cgPath // self.bbLayer is created and added as a sublayer of arView.layer in viewDidLoad self.bbLayer.strokeColor = color.cgColor

Re-posting above reply with better formatting: AugmentedReality app built with ARKit + RealityKit template is resulting in large memory footprint With no code changes to the template app created by Xcode, Im noticing about 450MB of memory usage on iPhone 13 Pro Max When I profiled and looked at live-objects while ARExperience is running, here are the top allocations: Here are the heavy memory regions: VIRTUAL RESIDENT DIRTY SWAPPED VOLATILE NONVOL EMPTY REGION REGION TYPE SIZE SIZE SIZE SIZE SIZE SIZE SIZE COUNT (non-coalesced) =========== ======= ======== ===== ======= ======== ====== ===== ======= IOSurface 146.2M 129.2M 129.2M 0K 0K 44.0M 0K 100 MALLOC_LARGE 85.4M 58.5M 58.5M 0K 0K 0K 0K 372 see MALLOC ZONE table below IOAccelerator 178.4M 34.3M 34.3M 0K 4480K 29.9M 320K 152 `` VM:IOSurface: 159MB non-Object: 56MB Here's the breakdown of IOSurface: 2 * 13.38MB (QuartzCore) CAMetalLayer Display Drawable shared with backboardd; // CoreRE on callstack 32 * 3.03MB (CMCapture) 3 * 784KB (CoreVideo) non-objects: 16MB (AppleCV3D) 2.94MB (CoreRE) 2 * 2.36MB (AppleCV3D) 1.02MB (CoreRE) Many more AppleCV3D non-objects To check if this memory goes away when I transition out of the viewcontroller, I added an initial-view controller with a button to transition into the VC hosting ARView. When I transition out, memory remains close to the peak memory and does NOT go back to what it was before launching into AR-experience. Here are the heavy memory regions: VIRTUAL RESIDENT DIRTY SWAPPED VOLATILE NONVOL EMPTY REGION REGION TYPE SIZE SIZE SIZE SIZE SIZE SIZE SIZE COUNT (non-coalesced) =========== ======= ======== ===== ======= ======== ====== ===== ======= IOSurface 173.7M 156.6M 156.6M 0K 0K 44.0M 0K 110 MALLOC_LARGE 84.2M 73.6M 71.3M 2048K 0K 0K 0K 317 IOAccelerator 182.5M 38.4M 38.4M 0K 8832K 29.7M 320K 154 This time even though I have transitioned out of ViewController hosting ARView, Im still seeing 3 * 13.38MB (QuartzCore) CAMetalLayer Display Drawable shared with backboardd; // CoreRE on callstack Im still seeing a similar number of IOSurface and non-objects How can I properly cleanup after user is done using AR experience i.e transitioned out of ViewController hosting ARView

Another problem I'm noticing is that this large spike in memory doesn't get released when I transition out of the ViewController hosting ARView Following is a summary of profiling a skeleton AugmentedReality app AugmentedReality app built with ARKit + RealityKit template is resulting in large memory footprint With no code changes to the template app created by Xcode, Im noticing about 450MB of memory usage on iPhone 13 Pro Max When I profiled and looked at live-objects while ARExperience is running, here are the top allocations: Here are the heavy memory regions: VIRTUAL RESIDENT DIRTY SWAPPED VOLATILE NONVOL EMPTY REGION REGION TYPE SIZE SIZE SIZE SIZE SIZE SIZE SIZE COUNT (non-coalesced) =========== ======= ======== ===== ======= ======== ====== ===== ======= IOSurface 146.2M 129.2M 129.2M 0K 0K 44.0M 0K 100 MALLOC_LARGE 85.4M 58.5M 58.5M 0K 0K 0K 0K 372 see MALLOC ZONE table below IOAccelerator 178.4M 34.3M 34.3M 0K 4480K 29.9M 320K 152 VM:IOSurface: 159MB non-Object: 56MB Here's the breakdown of IOSurface: 2 * 13.38MB (QuartzCore) CAMetalLayer Display Drawable shared with backboardd; // CoreRE on callstack 32 * 3.03MB (CMCapture) 3 * 784KB (CoreVideo) non-objects: 16MB (AppleCV3D) 2.94MB (CoreRE) 2 * 2.36MB (AppleCV3D) 1.02MB (CoreRE) Many more AppleCV3D non-objects To check if this memory goes away when I transition out of the viewcontroller, I added an initial-view controller with a button to transition into the VC hosting ARView. When I transition out, memory remains close to the peak memory and does NOT go back to what it was before launching into AR-experience. Here are the heavy memory regions: VIRTUAL RESIDENT DIRTY SWAPPED VOLATILE NONVOL EMPTY REGION REGION TYPE SIZE SIZE SIZE SIZE SIZE SIZE SIZE COUNT (non-coalesced) =========== ======= ======== ===== ======= ======== ====== ===== ======= IOSurface 173.7M 156.6M 156.6M 0K 0K 44.0M 0K 110 MALLOC_LARGE 84.2M 73.6M 71.3M 2048K 0K 0K 0K 317 IOAccelerator 182.5M 38.4M 38.4M 0K 8832K 29.7M 320K 154 This time even though I have transitioned out of ViewController hosting ARView, Im still seeing 3 * 13.38MB (QuartzCore) CAMetalLayer Display Drawable shared with backboardd; // CoreRE on callstack Im still seeing a similar number of IOSurface and non-objects How can I properly cleanup after user is done using AR experience i.e transitioned out of ViewController hosting ARView