Reduce Memory footprint of RealityKit App

Spatial Computing ARKit
srnlwm OP
Created May ’22
Replies 6
Boosts 2
Views 2.1k
Participants 5

Im noticing about 450MB of memory footprint when loading a simple 2MB USDZ model.

To eliminate any mis-use of the frameworks on my part, I built a basic RealityKit app using Xcode's Augmented Reality App, with no code changes at all. Im still seeing 450MB in Xcode gauges(so in debug mode)

When looking at memgraph, Im seeing IOAccelerator and IOSurface regions have 194MB and 131MB of dirty memory respectively. Is this all camera-related memory?

In the hopes of reducing compute & memory, I tried disabling various rendering options on ARView as follows:

arView.renderOptions = [
            .disableHDR,
            .disableDepthOfField,
            .disableMotionBlur,
            .disableFaceMesh,
            .disablePersonOcclusion,
            .disableCameraGrain,
            .disableAREnvironmentLighting
        ]

This brought it down to 300MB which is still quite a lot.

When I configure ARView.cameraMode to be nonAR its still 113MB

Im running this on iPhone 13 Pro Max which could explain some of the large allocations, but would still like to see opportunities to reduce the foot print.

When I use QLPreviewController same model (~2MB) takes only 27MB in Xcode-gauges.

Any ideas on reducing this memory footprint while using ARKit

Replies  6
Boosts  2
Views  2.1k
Participants  5
srnlwm OP
Jun ’22

Another problem I'm noticing is that this large spike in memory doesn't get released when I transition out of the ViewController hosting ARView

Following is a summary of profiling a skeleton AugmentedReality app

AugmentedReality app built with ARKit + RealityKit template is resulting in large memory footprint With no code changes to the template app created by Xcode, Im noticing about 450MB of memory usage on iPhone 13 Pro Max

When I profiled and looked at live-objects while ARExperience is running, here are the top allocations:

Here are the heavy memory regions: VIRTUAL RESIDENT DIRTY SWAPPED VOLATILE NONVOL EMPTY REGION REGION TYPE SIZE SIZE SIZE SIZE SIZE SIZE SIZE COUNT (non-coalesced) =========== ======= ======== ===== ======= ======== ====== ===== ======= IOSurface 146.2M 129.2M 129.2M 0K 0K 44.0M 0K 100 MALLOC_LARGE 85.4M 58.5M 58.5M 0K 0K 0K 0K 372 see MALLOC ZONE table below IOAccelerator 178.4M 34.3M 34.3M 0K 4480K 29.9M 320K 152

VM:IOSurface: 159MB non-Object: 56MB

Here's the breakdown of IOSurface: 2 * 13.38MB (QuartzCore) CAMetalLayer Display Drawable shared with backboardd; // CoreRE on callstack 32 * 3.03MB (CMCapture) 3 * 784KB (CoreVideo)

non-objects: 16MB (AppleCV3D) 2.94MB (CoreRE) 2 * 2.36MB (AppleCV3D) 1.02MB (CoreRE) Many more AppleCV3D non-objects

To check if this memory goes away when I transition out of the viewcontroller, I added an initial-view controller with a button to transition into the VC hosting ARView. When I transition out, memory remains close to the peak memory and does NOT go back to what it was before launching into AR-experience.

Here are the heavy memory regions: VIRTUAL RESIDENT DIRTY SWAPPED VOLATILE NONVOL EMPTY REGION REGION TYPE SIZE SIZE SIZE SIZE SIZE SIZE SIZE COUNT (non-coalesced) =========== ======= ======== ===== ======= ======== ====== ===== ======= IOSurface 173.7M 156.6M 156.6M 0K 0K 44.0M 0K 110 MALLOC_LARGE 84.2M 73.6M 71.3M 2048K 0K 0K 0K 317 IOAccelerator 182.5M 38.4M 38.4M 0K 8832K 29.7M 320K 154

This time even though I have transitioned out of ViewController hosting ARView, Im still seeing 3 * 13.38MB (QuartzCore) CAMetalLayer Display Drawable shared with backboardd; // CoreRE on callstack Im still seeing a similar number of IOSurface and non-objects

How can I properly cleanup after user is done using AR experience i.e transitioned out of ViewController hosting ARView

0
srnlwm OP
Jun ’22

Re-posting above reply with better formatting:

AugmentedReality app built with ARKit + RealityKit template is resulting in large memory footprint With no code changes to the template app created by Xcode, Im noticing about 450MB of memory usage on iPhone 13 Pro Max

When I profiled and looked at live-objects while ARExperience is running, here are the top allocations:

Here are the heavy memory regions:

                                  VIRTUAL RESIDENT    DIRTY  SWAPPED VOLATILE   NONVOL    EMPTY   REGION 
REGION TYPE                          SIZE     SIZE     SIZE     SIZE     SIZE     SIZE     SIZE    COUNT (non-coalesced) 
===========                     ======= ========    =====  ======= ========   ======    =====  ======= 
IOSurface                        146.2M   129.2M   129.2M       0K       0K    44.0M       0K      100 
MALLOC_LARGE                      85.4M    58.5M    58.5M       0K       0K       0K       0K      372         see MALLOC ZONE table below
IOAccelerator                    178.4M    34.3M    34.3M       0K    4480K    29.9M     320K      152 ``

VM:IOSurface: 159MB

non-Object: 56MB

Here's the breakdown of IOSurface:

2 * 13.38MB (QuartzCore) CAMetalLayer Display Drawable shared with backboardd; // CoreRE on callstack
32 * 3.03MB (CMCapture) 
3 * 784KB (CoreVideo)

non-objects:

16MB (AppleCV3D)
2.94MB (CoreRE)
2 * 2.36MB (AppleCV3D)
1.02MB (CoreRE)
Many more AppleCV3D non-objects

To check if this memory goes away when I transition out of the viewcontroller, I added an initial-view controller with a button to transition into the VC hosting ARView. When I transition out, memory remains close to the peak memory and does NOT go back to what it was before launching into AR-experience.

Here are the heavy memory regions:

                                  VIRTUAL RESIDENT    DIRTY  SWAPPED VOLATILE   NONVOL    EMPTY   REGION 
REGION TYPE                          SIZE     SIZE     SIZE     SIZE     SIZE     SIZE     SIZE    COUNT (non-coalesced) 
===========                     ======= ========    =====  ======= ========   ======    =====  ======= 
IOSurface                          173.7M   156.6M   156.6M       0K       0K    44.0M       0K      110 
MALLOC_LARGE                        84.2M    73.6M    71.3M    2048K       0K       0K       0K      317 
IOAccelerator                      182.5M    38.4M    38.4M       0K    8832K    29.7M     320K      154

This time even though I have transitioned out of ViewController hosting ARView, Im still seeing

3 * 13.38MB (QuartzCore) CAMetalLayer Display Drawable shared with backboardd; // CoreRE on callstack

Im still seeing a similar number of IOSurface and non-objects

How can I properly cleanup after user is done using AR experience i.e transitioned out of ViewController hosting ARView

0
bhavin84 OP
Jun ’22

Hey, Any update. looks like I'm facing the same issue

0
mohen OP
Sep ’22

Yep, I've done the same tests as you and came to the same conclusion. Both Xcode templates (UIKit, SwiftUI) seem to be leaving memory footprints. Hopefully, some Apple engineer can share some info regards that.

0
masaldana2 OP
Sep ’22

reality kit still a toy

0
TESTSUP OP
Oct ’22

same problem😞

0
Reduce Memory footprint of RealityKit App
First post date Last post date
 
 
Q