mmap() fails on Big Sur beta10 Apple Silicon - TEST PROGRAM INCLUDED

Porting our application to beta10 Apple Silicon, we have a small test program that works on Mojave/Catalina (Intel). We haven't been able to install beta10 on Intel because VMware Fusion Pro 12.0.0 doesn't work with beta10.

Here's a run on beta10/ARM:

Code Block
@macarm[git:master]$ cc -m64 -g -Wall -Werror -o mapfail mapfail.c
@macarm[git:master]$ codesign -vvvv mapfail
mapfail: valid on disk
mapfail: satisfies its Designated Requirement
@macarm[git:master]$ ./mapfail 10000000 10000
maping from 0x10000000 to 0x10010000 is OK
Unable to reserve at 65536 (0x10000) bytes of memory for the test1 heap
test1 not mapped
@macarm[git:master]$


Here's a run on 10.15.7:
Code Block
@macdev[git:master]$ cc -m64 -g -Wall -Werror -o mapfail mapfail.c
@macdev[git:master]$ codesign -vvv mapfail
mapfail: code object is not signed at all
In architecture: x86_64
@macdev[git:master]$ ./mapfail 10000000 10000
maping from 0x10000000 to 0x10010000 is OK
Unable to reserve 0x10000000 for the test1 heap,
using 0x109c14000 instead
test1 heap mapped from 0x109c14000 to 0x109c25000
@macdev[git:master]$


This has stopped our app porting effort, since this is a basic error that doesn't even allow the app to start up.



I was all set to use one of my developer technical support tickets, but as I was filling it out it clearly says they don't do pre-release software.

I hope this gets to an Apple developer...
Should I file a DTS ticket for this?
Can someone that has 11.0.0 (at Apple) or someone that installed the 11.0.1 beta confirm that the bug is fixed or not?

Thank you.
On 11.0.1 Beta (20B5022a)

Code Block language
bryan@nvbigsur mapfail % cc -m64  -g -Wall -Werror -o mapfail mapfail.c
bryan@nvbigsur mapfail % codesign -vvvv mapfail
mapfail: code object is not signed at all
In architecture: x86_64
bryan@nvbigsur mapfail % ./mapfail 10000000 10000
maping from 0x10000000 to 0x10010000 is OK
Unable to reserve 0x10000000 for the test1 heap,
 using 0x10ad37000 instead
test1 heap mapped from 0x10ad37000 to 0x10ad48000
bryan@nvbigsur mapfail % 


I found this when looking for additional information on mmap on arm64, as I have an application where mmap is failing as well. I ran your test application on Big Sur 11.3 and it currently fails completely.

Code Block
cc -m64 -g -Wall -Werror -o mapfail main.c
codesign -vvv mapfail
mapfail: valid on disk
mapfail: satisfies its Designated Requirement
./mapfail 10000000 10000
maping from 0x10000000 to 0x10010000 is OK
Unable to reserve at 65536 (0x10000) bytes of memory for the test1 heap
test1 not mapped


I suspected this probably has to do with PROT_EXEC, so I tried mmap with different or'd bits.

Code Block if (base == 0 || ok_to_map(base, base+size)) {
res = (char *) mmap((void *) base, size, PROT_READ | PROT_WRITE,
MAP_PRIVATE | FIMAP_ANON | (base ? MAP_FIXED : 0),
bucket_o_zeros, 0);
}

Removing the PROT_EXEC bit produces:

Code Block
./mapfail 10000000 10000
maping from 0x10000000 to 0x10010000 is OK
Unable to reserve 0x10000000 for the test1 heap,
using 0x1040ec000 instead
test1 heap mapped from 0x1040ec000 to 0x104100000


Unfortunately for me, I need the exec bit. I'm assuming this is an attempt at further sandboxing the ecosystem, and if there's something I'm missing I'd be happy to be enlightened.

mmap() fails on Big Sur beta10 Apple Silicon - TEST PROGRAM INCLUDED
 
 
Q