JSVirtualMachine memory limitations?

How does it crash? If you run it outside of Xcode, does the crash generate a crash report?

The reason this matters is that there’s a variety of limits that could be in play here:

• A limit to JavaScriptCore itself (I’m not aware of such a limit, but it’s definitely possible)

• A limit to the amount of physical memory you can use

• A limit to the amount of virtual memory (that is, address space) you can use

The crash report should give us some idea as to which limit you’re running into.

Share and Enjoy
—
Quinn “The Eskimo!” @ DTS @ Apple
let myEmail = "eskimo" + "1" + "@apple.com"

WWDC runs Mon, 22 Jun through to Fri, 26 Jun. During that time all of DTS will be busy with conference duties.

Yes, sorry, I should also have mentioned that I don't know how it crashes as the crash is so spectacular that Xcode reports nothing.
I set break points to make sure it was the execution that it crashed in because there is no stack trace reported.
In the function below the crash occurs at line 12 so I know the VM loads the JS but then dies on execution.

Running on my iPhone X I selected the debug navigator top left of Xcode and watched memory usage shoot up to 1.74G before it ***** out. That was loading the 357Mb JS file.
The device is cleared from Xcode and is no longer selectable until I recompile and reload. (Xcode 11.5, iOS12 for iPad Mini and iOS13.5 on all other devices)

We've run additional tests using models of varying sizes on all the platforms that we have so we can figure out the limits.
Devices with 2Gb of memory can run JS files up to 150Mb.
It maybe higher but that was as high as we went other than the 357Mb file that only my iPad Pro will run.
We plan to split the processing over multiple files passing the result from each execution between them for now.

Code Block 
- (NSDictionary *) processBatchFile: (NSString *) filePath {
    NSError *error;
    NSString *jsonData = [NSString stringWithContentsOfFile:filePath
                                                   encoding:NSUTF8StringEncoding
                                                      error:&error];
    if (error) {
        NSLog(@"processBatchFile: batch load failed. %@",error);
        return Nil;
    }
    NSLog(@"processBatchFile: %@",[filePath lastPathComponent]);
    JSValue *jsProcessor = self.jsContext[@"processBatch"];
    JSValue *result = [jsProcessor callWithArguments:@[jsonData, self.configs]];
    NSLog(@"processBatchFile: result = %@",[result toDictionary]);
    return [result toDictionary];
}

If you run it outside of Xcode, do you get a crash report?

Share and Enjoy
—
Quinn “The Eskimo!” @ DTS @ Apple
let myEmail = "eskimo" + "1" + "@apple.com"

WWDC runs Mon, 22 Jun through to Fri, 26 Jun. During that time all of DTS will be busy with conference duties.

TestFlight reports one user as having seen the crashes and includes their feedback but no stack trace.
I also got Xcode's Organiser to check for Crashes that could be downloaded but nothing was found.

My hunch is that some process required for debugging is getting jetsammed, which would explain why there's no crash log. iOS doesn't have a memory paging system, so if free RAM gets low the system starts killing processes. This might also explain why your issue doesn't happen on devices that have more RAM. Unfortunately, if this is the case, the only solution I know of is to reduce peak memory usage.

One tool that may be helpful in your investigation is Safari Web Inspector's heap snapshotting feature, which will tell you where memory is being used in your JSContext. Another possibility (hard to tell from the snippet of code above), is to pass your a string of your JSON payload to JSON.parse rather than using an object literal is source code. It's currently more memory efficient to pass a string to JSON.parse because JSC doing so won't allocate bytecode to initialize each part of your JSON.

For what it's worth, I've filed a bug on our public bug tracker to add a JSON.parse bytecode for JSON literals: https://bugs.webkit.org/show_bug.cgi?id=213498

I've done some more testing and it looks like I was wrong about the crash point.
It isn't during execution that it crashes but at line 15 below at evaluateScript when the code is loaded.

Code Block 
		self.jsContext = [[JSContext alloc] initWithVirtualMachine:[JSVirtualMachine new]];
    self.jsContext[@"console"][@"log"] = ^(NSString *message){
        NSLog(@"JS Console: %@", message);
    };
    self.jsContext.exceptionHandler = ^(JSContext *context, JSValue *exception) {
        NSLog(@"JS Error: %@", exception);
    };
    NSString *model = [[NSString alloc] initWithContentsOfFile:pathToModel
                                                      encoding:NSUTF8StringEncoding
                                                         error:&error];
    if (error) {
        NSLog(@"setUpJSContextWithModel: model load failed: %@",error);
        return;
    }
    JSValue *result = [self.jsContext evaluateScript:model withSourceURL:[NSURL fileURLWithPath:pathToModel]];

Sorry for mis-leading in my initial post.

Also Xcode's debug memory profiler seems to suggest that the upper limit is about 60% of the device memory.
For example an iPhone X with 3Gb of memory crashes after the memory profile reaches 59.4% of memory.

The data scientist who is building these models sent this to me today:

I profiled the memory usage when running the models on my Mac, in JavaScriptCore and in Node.js. The full-357MB used 2.42GB in JavaScriptCore vs. 799MB in Node.js.I think some of that excess usage might come from the way the model-parameters are being defined in the JS file (we start with them in a separate JSON and use a tool to automatically pack everything into one file) - basically, we might start with them in string-form and then also have to JSON.parse that string, almost doubling the footprint. There’s still an overhead from ~700 to 799MB unaccounted for, but I don’t think this huge memory usage is (solely) down to poor coding by me!

Is there anything we can do to improve memory efficiency in JavaScriptCore?
Thanks!

We'd love to look into why we're using so much memory here. Two things:

1. Can you create a sample project that contains the large memory usage? That way we can look at the project locally and help diagnose the issue.

2. One thing that jumps out at me: is the string you're passing to "evaluateScript:withSourceURL:fileURLWithPath" ASCII or does it contain non ASCII characters? For example, unicode characters. If so, you may want to consider this to "\uABCD" form instead of inline unicode characters. For source strings in JavaScriptCore, when a string contains only ASCII characters, each character takes 1 byte of memory. If there are unicode characters, we use two bytes of memory for each character. In the latter case, we will use 2x memory. For example, say we have a string with length 100, which has a single unicode character, we'll use 200 bytes to represent this string.

To the "Frameworks Engineer", sorry I missed your first reply. For some reason I only saw Eskimo's replies to me.

I think your initial hunch is probably correct so I'm going to ask the data scientist to build the JS code without data and pass it as a parameter to be parsed by JSON.parse at run time. I'm assuming that's what you meant?

1. I'll need to seek permission to share any data as each AI model is proprietary IP of the company's clients. The App is a B2B enterprise app and not a consumer app. One of the other things we are looking into is how to secure these models within iOS as they do represent significant IP but that's a separate issue.

2. I'll find out tomorrow from the data scientist if there's anything non-ASCII in the string.

Thanks!

The data scientist has confirmed that the data in the JS is just plain ASCII text with very long lines.
From smaller, readable, models I've been able to look at I can add that they appear to be dictionaries of real numbers.
e.g.

Code Block 
var coefs_ = {"1":-3.601963153,"2":-0.817495724,"3":-0.0812173088, ......

If the data is just a list of numeric keys to floating point numbers you might be able to decode the data into a Float32Array. That's assuming you don't need the full 64 bits of precision so you should ask your data scientist. Unlike the backing buffer for JS Objects/Arrays, which use 64-bits for each property, Float32Arrays use 32-bits per index, so they take up roughly half the memory.

I don't think there's a builtin API for this deserialization but it's probably not hard to code one up. You could also save some memory by omitting the quotes around each index. e.g.

Code Block js
var coefs_ = {1:-3.601963153,2:-0.817495724,3:-0.0812173088, ......

__
Keith