Post

Replies

Boosts

Views

Activity

Reply to iOS writeback behavior for mmap(MAP_SHARED) dirty pages
Okay. Among msync/madvise options only MADV_WILLNEED seems to be somewhat close to my case: MADV_WILLNEED Indicates that the application expects to access this address range soon. Though it is vague as user can not tell the kernel what is the nature of planned access - read, write or read-write. It is easy to imagine that this knowledge might allow some shortcuts in the UBC, including deferring writebacks. Actually it looks like a deficiency of madvise interface overall, which only operates by abstract "accesses", never clarifying about their direction. I'd ask you if it is possible for Apple to clarify MADV_WILLNEED effects to see if it can minimize writebacks but my gut feeling is that documentation here sticks exactly to how POSIX_MADV_WILLNEED is defined and no deviations from POSIX are allowed. If talking about my second question - is there anything else that could be used by iOS app to convince kernel to retain content of some app pages after app termination and then have access to this content upon next app start? I guess it still would boild down to mmap(MAP_SHARED) but for an object such that uncontrolled writebacks are not an issue, like ramfs file or some shmem. Thanks.
Sep ’24
Reply to iOS writeback behavior for mmap(MAP_SHARED) dirty pages
I don't think POSIX actually requires Right, that was my far-fetched speculation based on the phrase "If MAP_SHARED is specified, write references shall change the underlying object.". For some reason I implied that "underlying object" is not just a set of memory pages but something that is responsible for all system-wide accesses to the mapped file. Nevertheless it turns out that my speculation is correct for at least some popular platforms. I don't think the result you found there is accurate, at least not in the broad/general case. What were the specifics of what you were actually testing? I have to admit that testing was not really meticulous. For the project planned the size of needed mappings will be about few megabytes. No more that 20-30M under any circumstances. It can be a single mapping or a chain of smaller ones that yield the same total size. Each page is to be modified at least once in a few seconds (typically more frequently). So I did testing with these numbers in mind. Tested on M1 Max 64G (because I haven't figured out how to see low-level disk IO on iOS, excuse me my ignorance) apparently it is not the best playground for drawing conclusions about how iOS device would behave. There was no memory pressure to the system and I haven't seen any flushing for a couple minutes for sure until the referencing process was killed. At the same time I saw a clear effect of memory pressure and mapping size on Android which made me think that Apple platforms could exhibit the same behavior which I just did not manage to achieve. So I dropped experimenting in favor of searching for documented operation description. What are you actually looking for/expecting here? How "long" do you want the system to defer your writes? Ideally - to not flush at all until process terminates or does munmap(). This mapping is intended for using as a storage for crash-persistent critical app logs. The only purpose of MAP_SHARED here is to achieve survival across process termination and avoid doing syscall during the act of logging. It is acceptable to lose this log in case of kernel panic assuming that typically OS kernels are more stable than applications are and thus kernel panic is a rare event if compared with number of app crashes. In the cases where excessive writeback is a serious concern, the typical solution is to add an intermediate layer which collects data first before committing it to the I/O layer. Certainly, that would be a natural move. Though in my case an intermediate layer (that is inevitably implemented in the userspace as a part of the app) in principle can not offer content persistence across process termination, hence the desire to pass data immediately to the side where app crash is no more a problem. My primary concern was that uncontrolled excessive flushing from high traffic logger mapping may impact performance of other disk operations of the app and speed-up wear of non-volatile storage. Though maybe I'm overly pessimistic and wear problem is not relevant these days, not sure. ... the kernel already has a strong incentive to delay writes. That sounds promising, at least to know that what I want is aligned with the principles of the system. Thanks.
Sep ’24