[NSDate date] memory leak!

I once asked a question about how to detect memory leaks when app is running. Thanks for someone who points out that Instruments is the right tool for this purpose.


Unfortunately by using Instruments, no explicit memory leaks are detected. But I found a very bewildering problem. It seems [NSDate date] leaks 16 bytes memory on each call. I soon constructed a very simple app to verify this:

- (void)viewDidLoad {
    [super viewDidLoad];
    static NSOperationQueue* q;
    q = [[NSOperationQueue alloc] init];
    [q addOperationWithBlock:^{
        while (YES) {
            NSDate* date = [NSDate date];
        }
    }];
}

When the simple app runs, in just about 10 seconds, it's memory usage rockets high to more than 1GB. That explains why one of my App Store app eats up memory gradually on daily usage.


I think it is a glaring bug because the code is so simple that it leaves no room for coding mistakes. But, I post it here so that this can be confirmed so that I can submit a formal bug report to Apple.


EDIT: it seems the bug occurs after I attached to the test app using Instruments and then stops recoding.


EDIT: It seems the above simple code does not always reproduce the bug. Please try the follow:

// Replace the line NSDate* date = [NSDate date] with following line
NSDateComponents* parts = [BingImageDate getDateParts:[NSDate date] ofCulture:@"en-US"];


//BingImageDate.h
#import <Foundation/Foundation.h>
@interface BingImageDate : NSObject
+ (NSDateComponents *)getDateParts:(NSDate *)date ofCulture:(NSString*)culture;
@end


//BingImageDate.m
#import "BingImageDate.h"
static NSCalendar* _gregorian;
static NSTimeZone* _TZ_PST;
static NSDictionary* _cultureHourOffset;
@implementation BingImageDate
+ (void)initialize
{
    _gregorian = [[NSCalendar alloc] initWithCalendarIdentifier:NSCalendarIdentifierGregorian];
    _gregorian.timeZone = [NSTimeZone timeZoneForSecondsFromGMT:0];
    _TZ_PST = [NSTimeZone timeZoneWithAbbreviation:@"PST"];
    _cultureHourOffset = @{
                           @"de-DE": @1500,
                           @"en-AU": @700,
                           @"en-CA": @2100,
                           @"en-GB": @1600,
                           @"en-NZ": @0,
                           @"en-US": @0,
                           @"fr-FR": @1500,
                           @"hi-IN": @1130,
                           @"ja-JP": @800,
                           @"pt-BR": @2000,
                           @"zh-CN": @900,
                           };
}
+ (NSDateComponents *)getDateParts:(NSDate *)date ofCulture:(NSString*)culture
{
    NSNumber* cultureHourOffset = _cultureHourOffset[culture];
    NSDateComponents* parts = [_gregorian componentsInTimeZone:_TZ_PST fromDate:date];
    NSInteger hourOffset = parts.hour * 100 + parts.minute;
    if (hourOffset < cultureHourOffset.integerValue)
    {
        NSDate* pivotDate = [_gregorian dateByAddingUnit:NSCalendarUnitDay
                                                   value:-1
                                                  toDate:parts.date
                                                 options:0];
        parts = [_gregorian componentsInTimeZone:_TZ_PST fromDate:pivotDate];
    }
    return parts;
}
@end

Replies

Doesn't seem there is a memory leak in your first code.


But you do :

while (YES) {

NSDate* date = [NSDate date];

}


So, you create an instance at each loop, hence the memory use.


If you replace NSDate by any allocator, you should get the same behavior.

Try the new code in my latest EDIT. On my MacBook with Sierra 10.12.6, it can eats up 10+GB memory if I let it run for minutes.

Cannot you simplify a bit ? Or is the _cultureHourOffset essential to see the problem ?


Anyway, if you keep looping

while (YES) { }

then you keep creating new instances for parts. When you reach millions of them, then you crash.


Could see it by testing (I write the swift equivalement):

       var counter = 0
        while (YES) { 
          counter += 1
         if counter % 100_000 == 0 { print(counter) }
}

>> the code is so simple that it leaves no room for coding mistakes


No. In fact, the code is so simple that it is a coding mistake.


You are allocatiing memory in a tight loop, and the memory being allocated is known to be autoreleased. (Factory methods like +[NSDate date] have "+0" return semantics. The autorelease is the mechanism that keeps the result alive until your code can retain it.) The memory is not reclaimed until your app drains the relevant autorelease pool, and your "simple" code prevents this from happening.


This outcome is a well-known consequence of Cocoa's reference-counted memory management, and avoiding it is a core competency for macOS and iOS programming.

I'm not sure if I am competent for writing apps for macOS, but I did publish two apps.


About the ARC thing - the compiler should detect that there is no code referencing the returned NSDate instance and should immediately destroys it when it falls out of scope. Or my understanding is completely wrong?


EDIT: BTW, it seems you're quite compenent at this domain, do you have any suggestions on how to refactor my code to eliminate the memory 'leaks'?

Please read my second EDIT (the first code part).


Do you have any idea on how to refactor my code the avoid memory leaks? My code has to repeated call methods in BingImageDate.

If I use NSString, then there is no memory leak. Maybe I misunderstood your reasoning, but here is the revised code:

//BingImageDate.m
+ (NSString*)testString:(NSString *)s
{
    return s.description;//[s copy];
}
+ (NSString *)testDate:(NSDate *)date
{
    return date.description;
}


//viewController.m ViewDidLoad
#if 1
            //NSString* s = [NSString string];
            NSString* s = [NSString stringWithUTF8String:"replace this with a very long text"];
            NSString* text = [BingImageDate testString:s];
#else
            NSDate* now = [NSDate date];
            NSString* text = [BingImageDate testDate:now];
#endif

In theory, NSString factory method should behave identically with NSDate, but why does NSString have no memory leaks?

It's not so easy to predict what the memory behavior of Cocoa classes will be. NSString has multiple internal optimizations, whose purpose is to try to make strings perform well in a variety of situations. In particular, what you see as a NSString object may not be an object all, or may not be a reference counted object. (The result of +[NSString string], for example, is almost certainly a tagged pointer, which has no memory management at all, or a "constant" object, for which retain and release are a no-op.)


>> I'm not sure if I am competent for writing apps for macOS, but I did publish two apps.


Ii certainly wasn't doubting your competence. Rather, I was pointing out that an apparently very simple test can produce misleading results because it is too simple, and it's not necessarily obvious what is valid reasoning and invalid reasoning.


>> the compiler should detect that there is no code referencing the returned NSDate instance


The compiler can't reason about this very far, because of the dynamic nature of the Obj-C language. It doesn't know what the implementation of (say) +[NSDate date] is — it can even be replaced dynamically during execution of your app. There are cases where the called method can optimize away the autorelease (when both caller and callee have code that's compile in ARC mode), but it's likely that NSDate is an older implementation that doesn't support this.


>> how to refactor my code


It's hard to answer without knowing what you're really trying to do. In principle, something like this:


     [q addOperationWithBlock:^{
          while (YES) {
               @autoreleasepool {
                    NSDate* date = [NSDate date];
               }
          }
     }];


would discard the unused object in every iteration, but it has a performance penalty. (However, using a local autorelease pool is a commonly-used technique for scenarios like this, where each iteration does enough work to make the penalty insignificant. You just have to be careful about the scope's effect on lifetimes, which can be a bit subtle.) If you're doing something with a high iteration count and stricter performance requirements, you can do something like this:


     [q addOperationWithBlock:^{
          while (YES) {
               @autoreleasepool {
                    for (i = 0; i < 1000; i++) {
                         NSDate* date = [NSDate date];
                    }
               }
          }
     }];


In more normal circumstances, the best solution is to use an operation of this form:


     [q addOperationWithBlock:^{
          NSDate* date = [NSDate date];
     }];


and then queue multiple operations instead of just one. That has the advantage of being able to take advantage of multiple CPUs at once. Of course, that's only feasible if the individual iterations don't depend on each other. If they do, you need communication and synchronization techinques of greater or lesser complexity.


That's why there's no general strategy here. Getting this sort of thing right can be an incredible difficult problem, or it can be straightforward, depending on what you're doing.

Yeah! Explicit @autoreleasepool block solves the problem.


I never give a deep dive into ARC. I vaguely remember 3 years ago I read some articles on this when I first started Objective-C programming, as a hobby. What I understand is that ARC is basically compillation time work in that it is not GC as found in some other languages (eg, C#, Java etc); the compiler generates a stub class that automatically addref/decref to objects and when the counter reaches 0, the object is 'deallocated'.


But I am still not sure why the loop cannot be an autoreleasepool as seen by compiler. And the NSDate instance apparently has no other code referencing it, whhich can be determined at compilation time.

ARC is a mixture of compile-time and run-time features. There's no stub class. Instead (almost) all Obj-C objects have a reference count that is increment by a retain and decremented by a release. You can still write manual memory management that does this explicitly if you want, but ARC simply does it for you, provided you abide some usage rules that everyone has followed since macOS 10.0 anyway.


The problem with your loop is that the compiler doesn't know that the object returned from "+[NSDate date]" has been autoreleased. (We know, because they accumulate at run time. If that factory method were re-written to use ARC itself, it wouldn't do the autorelease and you wouldn't see this issue.) All it knows is that it must retain the return value immediately, and release it at the end of your scope. That doesn't affect the outcome if the autorelease has already been done.


You might wonder why Apple doesn't re-compile all of the original Cocoa classes converted to ARC. The problem there is that the source code would have to be converted, and that could well be a source of bugs, and new bugs in basic classes like NSDate and NSString are likely to be very bad news for all developers. IAC, autorelease pool issues are a well-understood problem — it's just a shock when they hit you personally for the first time.

Sorry, your explanation is too advanced for me to understand; and it puzzles me more. My standpoint is that the code piece is so simple and it is totally valid and correct in plain C/C++, yet it is produces unexpected results in Objective-C.


I have another question. Now I know that in the loop the instance never gets released without explicit autoreleasepool. What happens after the loop breaks out and the operation block finishes?


I did some tests. I run the app for several minutes until its memory reaches 10GB (as viewed in Activity Monitor). Then I stop the loop and eventually the operation block finishes. After that, the compressed memory keeps growing for several minutes and then starts declining with the (real) memory number. That takes another several minutes and then suddenly it drops from 7GB to 4GB. And the number stops there, forever.


I guess it should eventually free all instances, because loops and if/else blocks are ubiquitous in Objectiv-C coding. But the test results seem don't agree with my guess.

Is it possible you have reached a system limit of how many objects it can keep track ? (10 GB means a huge number of objects to track for future release).

No, no, no. It's not what I meant. 10GB is not the absolute upper limit because last night I tried 16GB. What I tried to emphasize is that the memory was released very slowly and it halted on a point at a little more than 4GB.

The short answer is: It's complicated. Generally, the way to avoid the complication is to stick to established coding patterns.


On top of everything else, interpreting the memory usage numbers reported by Activity Monitor is also complicated. There are different kinds of memory, handled in different kinds of ways, and different ways to add up the usage. Of course, if your app starts creating millions of objects, you will see numbers in Activity Monitor go up dramatically, but it's not at all obvious how or when or how far they will go down again.