3 Replies
      Latest reply on Mar 29, 2019 2:49 AM by eskimo
      libe Level 1 Level 1 (0 points)

        Hello,

         

        I am having an hard time figuring out how memory mapped files works under iOS. Suppose that I want to read a big file, let's say 4GB. If I use memory mapped files in READ mode I should be able to get a valid pointer to a file and benefit from the page fault mechanism of virtual memory but apparently iOS can map only up to 2.5GB of data. I run the following test on an iPad Pro A1673 that has 2GB of ram and got as result:

         

        2018-04-06 17:20:12.660888+0200 TestMemory[414:316466] Data address: 0x102224000

        2018-04-06 17:20:12.662355+0200 TestMemory[414:316466] Data address: 0x112224000

        2018-04-06 17:20:12.663801+0200 TestMemory[414:316466] Data address: 0x12b900000

        2018-04-06 17:20:12.665235+0200 TestMemory[414:316466] Data address: 0x13b900000

        2018-04-06 17:20:12.666756+0200 TestMemory[414:316466] Data address: 0x14b900000

        2018-04-06 17:20:12.668202+0200 TestMemory[414:316466] Data address: 0x15b900000

        2018-04-06 17:20:12.669597+0200 TestMemory[414:316466] Data address: 0x16ff24000

        2018-04-06 17:20:12.739549+0200 TestMemory[414:316466] Data address: 0x1e0000000

        2018-04-06 17:20:12.747604+0200 TestMemory[414:316466] Data address: 0x1f0000000

        2018-04-06 17:20:12.749130+0200 TestMemory[414:316466] Data address: 0x200000000

        2018-04-06 17:20:12.764753+0200 TestMemory[414:316466] NSData failed: Error Domain=NSCocoaErrorDomain Code=256 "The file “02808580-5D42-43BC-B5AA-628E3682A546” couldn’t be opened." UserInfo={NSFilePath=/var/mobile/Containers/Data/Application/5207275C-5525-47CA-AC4B-2AA07E05C1E9/Documents/02808580-5D42-43BC-B5AA-628E3682A546, NSUnderlyingError=0x1c0455270 {Error Domain=NSPOSIXErrorDomain Code=12 "Cannot allocate memory"}}

        2018-04-06 17:20:12.764801+0200 TestMemory[414:316466] Mapped 2684354560

         

         

        - (NSString*)createFileOfSize:(unsigned long long)size {
            NSString* uuid = [[NSUUID UUID] UUIDString];
            NSString* documentFolderPath = [[[NSFileManager defaultManager] URLsForDirectory:NSDocumentDirectory inDomains:NSUserDomainMask] lastObject].path;
            NSString* filePath = [documentFolderPath stringByAppendingPathComponent:uuid];
            NSFileHandle *fileHandle = [NSFileHandle fileHandleForWritingAtPath:filePath];
            if (fileHandle == nil) {
                [[NSFileManager defaultManager] createFileAtPath:filePath contents:nil attributes:nil];
                fileHandle = [NSFileHandle fileHandleForWritingAtPath:filePath];
            }
            [fileHandle truncateFileAtOffset:size];
            [fileHandle closeFile];
            return filePath;
        }
        - (void)test {
            const unsigned long long MB = 1 << 20;
            const unsigned long long GB = 1 << 30;
            unsigned long long space = 4 * GB;
            unsigned long long size = 64 * MB;
            unsigned long long fileCount = space / size;
            NSMutableArray* mapped = [NSMutableArray array];
            int i = 0;
            for (i = 0; i < fileCount; i++) {
                NSString* filePath = [self createFileOfSize:size];
                NSError* error = nil;
                NSData* data =  [NSData dataWithContentsOfFile:filePath options:NSDataReadingMappedAlways error:&error];
                if (error) {
                    NSLog(@"NSData failed: %@", error);
                    break;
                }
                else {
                    const void* bytes = [data bytes];
                    NSLog(@"Data address: %p", bytes);
                    [mapped addObject:data];
                }
            }
        
            NSLog(@"Mapped %llu", i * size);
        }
        

         

        Any idea of why we have this limitation?

         

        Thanks!

        Libe

        • Re: Memory mapped file: "Cannot allocate memory"
          eskimo Apple Staff Apple Staff (11,795 points)

          This is not about physical memory but address space.  iOS puts limits on the address space available to app processes, meaning your can’t map files beyond a certain size.

          Share and Enjoy

          Quinn “The Eskimo!”
          Apple Developer Relations, Developer Technical Support, Core OS/Hardware
          let myEmail = "eskimo" + "1" + "@apple.com"

            • Re: Memory mapped file: "Cannot allocate memory"
              Sumbooo Level 1 Level 1 (0 points)

              Thank you for the infomation you provide, that realy make sense to me.

              But how can i release the address space?

              I use "UIGraphicsBeginImageContextWithOptions", and I got this "CGBitmapContextInfoCreate: unable to allocate 6512640 bytes for bitmap data",so I can't get the image I want, but the memory is just 170M~180M, and App doesn't crash, I just want the image.

              Could you give me some advise?

                • Re: Memory mapped file: "Cannot allocate memory"
                  eskimo Apple Staff Apple Staff (11,795 points)

                  But how can i release the address space?

                  By freeing the address space using the deallocator associated with the allocator you used.  For example:

                  • If you mapped a file with mmap, you’d free the address space with free.

                  • If you mapped a file with NSData, you’d free the address space by releasing all of the references to the resulting data object.

                  Share and Enjoy

                  Quinn “The Eskimo!”
                  Apple Developer Relations, Developer Technical Support, Core OS/Hardware
                  let myEmail = "eskimo" + "1" + "@apple.com"