CoreData stack init is freeze

The CoreData stack initialization is freeze at the function call

-[NSPersistentStoreCoordinator addPersistentStoreWithType:configuration:URL:options:error:].

It doesn't return any errors. The CoreData debug logs console shows:

CoreData: annotation: Core Data multi-threading assertions enabled.
CoreData: annotation: Connecting to sqlite database file at "/var/mobile/Containers/Data/Application/1B4CD169-5CCB-4949-B8B2-943A8EDC7478/Documents/myDB.sqlite"
CoreData: sql: SELECT TBL_NAME FROM SQLITE_MASTER WHERE TBL_NAME = 'Z_METADATA'
2023-02-28 16:29:37.357278-0800 AppName[1016:183178] [BackgroundTask] Background Task 8 ("CoreData: Adding persistent store"), was created over 30 seconds ago. In applications running in the background, this creates a risk of termination. Remember to call UIApplication.endBackgroundTask(_:) for your task in a timely manner to avoid this.
CoreData: error: Unexpected background task assertion cancellation.
2023-02-28 16:30:35.668186-0800 AppName[1016:183178] [BackgroundTask] Background task still not ended after expiration handlers were called: <_UIBackgroundTaskInfo: 0x28258fec0>: taskID = 8, taskName = CoreData: Adding persistent store, creationTime = 16118 (elapsed = 753). This app will likely be terminated by the system. Call UIApplication.endBackgroundTask(_:) to avoid this.

It repeats multiple times the same log:

2023-02-28 16:30:01.186448-0800 AppName[1016:183461] [GEOXPC] Throttled "PlaceRequest.REQUEST_TYPE_REVERSE_GEOCODING" request: Tried to make more than 50 requests in 60 seconds, will reset in 1 seconds - Error Domain=GEOErrorDomain Code=-3 "(null)" UserInfo={details=(
        {
        intervalType = short;
        maxRequests = 50;
        "throttler.keyPath" = "app:com.company.myApp/0x20304/short(default/any)";
        timeUntilReset = 1;
        windowSize = 60;
    }
), requestKindString=PlaceRequest.REQUEST_TYPE_REVERSE_GEOCODING, timeUntilReset=1, requestKind=772}

The DB is populated with 11 MB size data, is created by iOS app, is opened after app version upgraded. The new app version has new DB module version with basic data migration (removed 3 tables, changed data type with entity mapping).

It happens rare with a released app. I cannot reproduce it in the Xcode debug or released app. No crash reports. The exported DB sqlite file is opened successfully with the SQLite editors and all module updates are applied successful. If it's imported to another iOS device then the app works well and can read data. Something happens on this iPad and after app reinstall it works.

Why CoreData init freeze on Z_METADATA query? Possible the table is corrupted. How I can verify it for any issues?

Why Background Task 8 is created? The func is called on the background thread. Possible migration process creates a Background Task.

What GEOXPC log means?

  • I found that the sqlite-wal file exists so far the DB session is opened but it's still on the iPad disk storage even the iOS app is force closed. This file keeps transactions that were previously committed to the DB. Is it possible that the file keeps bad DB status and next app execution freezes at the first `SELECT' query?

    I tried to export the DB sqlite, sqlite-shm, sqlite-wal files from the iPad but only sqlite, sqlite-shm are copied to my mapbook. How I can copy WAL file?

  • I found that the sqlite-wal file exists so far the DB session is opened but it's still on the iPad disk storage even the iOS app is force closed. This file keeps transactions that were previously committed to the DB.

    Is it possible that the file keeps bad DB status and next app execution freezes at the first `SELECT' query?

    Should iOS close DB session when app is force closed?

Add a Comment

Replies

I found that the sqlite-wal file exists so far the DB session is opened but it's still on the iPad disk storage even the iOS app is force closed. This file keeps transactions that were previously committed to the DB. The WAL file is not removed because SQLite journal mode prevents database corruption. If the application is crashed in the middle of a checkpointing operation. Then the database file .sqlite is in an invalid state unless joined with a new data contained in the WAL. The file can be removed by SQLite when it finished checkpoint.

I cannot copy this file from the iPad on my mac book via the iExplorer app or Xcode -> Download Container, it's immediately disappear (or never copied). That why original iPad can reproduce this issue but other iPad with copied files cannot.

  • Is it possible that the WAL file keeps bad DB status and next app execution freezes at the first `SELECT' query? How to resolve it?
  • Can I fix the CoreData freeze if to open the database with the DELETE journal mode pragma? This will checkpoint and then delete the WAL file for me.
@{ NSSQLitePragmasOption: @{ @"journal_mode": @"DELETE"}};

Thanks

I tried to export the WAL file after coping the original file to a new one with a different name. Yes, I can export myDB2.sqlite-wal file and it's identical.

        NSString *pathOld = @".../myDB.sqlite-wal";
        NSString *pathNew = @".../myDB2.sqlite-wal";
        [[NSFileManager defaultManager] copyItemAtPath:pathOld toPath:pathNew error:nil];

If all files are imported to the different iPad then the CoreData freeze is NOT experienced. If to initialize CoreData stack with the copied files on the same iPad (where the issue is reproducible) then the CoreData freeze is NOT experienced! The original WAL file still have the issue.

The access permission can be changed on the WAL file but it's similar what sqlite has:

    NSFileExtensionHidden = 0;
    NSFileGroupOwnerAccountID = 501;
    NSFileGroupOwnerAccountName = mobile;
    NSFileModificationDate = "2022-12-03 01:39:41 +0000";
    NSFileOwnerAccountID = 501;
    NSFileOwnerAccountName = mobile;
    NSFilePosixPermissions = 420;
    NSFileProtectionKey = NSFileProtectionCompleteUntilFirstUserAuthentication;
    NSFileReferenceCount = 1;
    NSFileSize = 282108792;
    NSFileSystemFileNumber = 285206;
    NSFileSystemNumber = 16777220;
    NSFileType = NSFileTypeRegular;

I noticed a CoreData error when keeps app working with the copied sqlite files. The console shows:

CoreData: error: (13) database or disk is full
CoreData: annotation: Disconnecting from sqlite database due to an error.
CoreData: error: SQLCore dispatchRequest: exception handling request: <NSSQLSaveChangesRequestContext: 0x2838cf480> , error during SQL execution : database or disk is full with userInfo of {
    NSFilePath = "/var/mobile/Containers/Data/Application/77B8EAB6-120F-4E4D-AE96-EDC81258A250/Documents/myDB2.sqlite";
    NSSQLiteErrorDomain = 13;
}

It happens randomly and multiple times after different CoreData activities:

  • SELECT TBL_NAME FROM SQLITE_MASTER WHERE TBL_NAME = 'ACHANGE'
  • SELECT all columns from the regular table.
  • SQLite bind[2] = ""
  • SQLite: EXPLAIN QUERY PLAN SELECT
  • sql: COMMIT
  • Fetching 13543 rows

Is it mean that the iPad doesn't have enough disk or RAM space? The iPad storage shows 26 GB of 128 GB used and application alocated only 48 MB RAM. I don't see the same errors on a different iPad with the same copied files.

Verified any differences between sqlite files before/after using PRAGMA options:

  1. journal_mode=DELETE
  2. wal_checkpoint=FULL

These options force checkpointing from WAL file to sqlite file. I want to know what kind transactions was applied, if it's migration issue. In result, the sqldiff myDB.sqlite myDB2.sqlite doesn't show any difference and 1 - will remove the WAL file, 2 - keeps WAL file without modifications in size or date.

I decided to apply my previous attempts on the original WAL file.

First, the checkpoint forcing via journal_mode=DELETE doesn't work for the original WAL file. The CoreData is freeze and file is not removed at the end of session.

Second, I moved (didn't copy) original file under different name myDB2.sqlite and the CoreData stack init is freeze again. Interesting in my previous tests that the copied file with the same name was opened but with periodic errors.

Third, If the original file is moved and copied to the path with initial name then the CoreData stack initialized successfully and opened the DB. The addPersistentStoreWithType:configuration:URL:options:error: func started to return the error:

...
CoreData: error: 	sourceURL : file:&#x2F;&#x2F;&#x2F;var&#x2F;mobile&#x2F;Containers&#x2F;Data&#x2F;Application&#x2F;08D158CE-7216-4271-A3F2-AF3E4E5DA772&#x2F;Documents&#x2F;myDB.sqlite
CoreData: error: 	reason : Cannot migrate store in-place: error during SQL execution : database or disk is full
CoreData: error: 	destinationURL : file:&#x2F;&#x2F;&#x2F;var&#x2F;mobile&#x2F;Containers&#x2F;Data&#x2F;Application&#x2F;08D158CE-7216-4271-A3F2-AF3E4E5DA772&#x2F;Documents&#x2F;myDB.sqlite
CoreData: error: 	NSUnderlyingError : Error Domain=NSCocoaErrorDomain Code=134110 "An error occurred during persistent store migration." UserInfo={NSSQLiteErrorDomain=13, NSFilePath=&#x2F;var&#x2F;mobile&#x2F;Containers&#x2F;Data&#x2F;Application&#x2F;08D158CE-7216-4271-A3F2-AF3E4E5DA772&#x2F;Documents&#x2F;myDB.sqlite, NSUnderlyingException=error during SQL execution : database or disk is full, reason=error during SQL execution : database or disk is full}
CoreData: error: storeType: SQLite
CoreData: error: configuration: (null)
CoreData: error: URL: file:&#x2F;&#x2F;&#x2F;var&#x2F;mobile&#x2F;Containers&#x2F;Data&#x2F;Application&#x2F;08D158CE-7216-4271-A3F2-AF3E4E5DA772&#x2F;Documents&#x2F;myDB.sqlite
CoreData: annotation: options:
CoreData: annotation: 	NSInferMappingModelAutomaticallyOption : 1
CoreData: annotation: 	NSMigratePersistentStoresAutomaticallyOption : 1

It happens when the CoreData attempts to migrate a store and SQL disk space estimation throws error.

I am not sure if it happens because if the migration or the last SQL commit request. The last SQL commit request has a lot of CREATE INDEX for every table, DELETE and INSERT new BLOB data to the Z_METADATA and Z_MODELCACHE tables:

CoreData: sql: CREATE INDEX IF NOT EXISTS Z_Table1_Table2_Attribute1 ON ZTABLE1 (ZTABLE2 COLLATE BINARY ASC, ZATTRIBUTE1 COLLATE BINARY ASC)
CoreData: sql: SELECT TBL_NAME FROM SQLITE_MASTER WHERE TBL_NAME = &#039;ANSSQLITEINDEXSTATISTICS&#039;
CoreData: sql: SELECT TBL_NAME FROM SQLITE_MASTER WHERE TBL_NAME = &#039;ACHANGE&#039;
CoreData: sql: SELECT TBL_NAME FROM SQLITE_MASTER WHERE TBL_NAME = &#039;Z_METADATA&#039;
CoreData: sql: DELETE FROM Z_METADATA WHERE Z_VERSION = ?
CoreData: details: SQLite bind[0] = 1
CoreData: sql: INSERT INTO Z_METADATA (Z_VERSION, Z_UUID, Z_PLIST) VALUES (?, ?, ?)
CoreData: details: SQLite bind[0] = 1
CoreData: details: SQLite bind[1] = "51083DD1-5CBD-4F94-AF82-74684B6C5A07"
CoreData: details: SQLite bind[2] = &lt;NSData len=3259&gt;
CoreData: annotation: Saving new meta data; version = 1 ; UUID = 51083DD1-5CBD-4F94-AF82-74684B6C5A07
CoreData: sql: SELECT TBL_NAME FROM SQLITE_MASTER WHERE TBL_NAME = &#039;Z_MODELCACHE&#039;
CoreData: sql: DELETE FROM Z_MODELCACHE
CoreData: sql: INSERT INTO Z_MODELCACHE (Z_CONTENT) VALUES (?)
CoreData: details: SQLite bind[0] = &lt;NSData len=41191&gt;
CoreData: annotation: Saving new model cache
CoreData: sql: COMMIT
CoreData: error: (13) database or disk is full
CoreData: annotation: Disconnecting from sqlite database due to an error.
CoreData: annotation: Disconnecting from sqlite database.
CoreData: annotation: Disconnecting from sqlite database.
CoreData: error: addPersistentStoreWithType:configuration:URL:options:error: returned error NSCocoaErrorDomain (134110)
...

The 13 error from SQLITE is : #define SQLITE_FULL 13 Insertion failed because database is full

Other iPad devices with copied files finish the migration successfully with the same logs (without error) and WAL has zero size:

...
CoreData: annotation: Saving new model cache
CoreData: sql: COMMIT
CoreData: sql: pragma wal_checkpoint(truncate)
CoreData: annotation: sql execution time: 0.0028s
CoreData: annotation: Disconnecting from sqlite database.

The error can happen when:

  1. An attempt to insert new data that would cause the database file to grow larger than Maximum Number Of Pages In A Database File. In my DB now 2856 number of pages.
  2. The inserted page size larger than the default value - 4096 bytes.

How to found the page size that the migration tries to insert?

Why one iPad can migrate but another not? The App container is the same.

The issue with the full database or disk happens because of a free space on the physical disk or inserted a large DB page size. I verified on the original iPad if a disk has a free space limit or app container has a large size for the SQLite migration process.

The first test. The app container size is 9.3 GB. I removed data for 1.3 GB and total left 8 GB. I launched application with the copied and original WAL file. As a result, respectively, the logs shows the same error and CoreData stack is freeze.

The second test, I removed the moved myDB2.sqlite-wal file (original file with the CoreData freeze) and left just a copied myDB.sqlite-wal. The myDB2.sqlite-wal file has 282.1 MB. The CoreData stack successfully initialized and no errors in the CoreData/SQLite.

Looks like a problem was around the original WAL file that was not accessible by the CoreData and by the export file option.

Currently, the possible solution for the CoreData freze is:

  1. Move WAL file under new name.
  2. Copy moved WAL file under the original name.
  3. Remove moved WAL file
  4. Init CoreData stack