Database growing in size. Recommendations for compaction?

Since adding frozen objects, I have gotten a few out of memory errors in production. I have read the issues on GitHub and tried to follow the suggestions there https://github.com/realm/realm-cocoa/issues/6469. I was definitely keeping frozen references too long while doing write transactions, something I have now adressed.

But, it still makes me wonder. I currently don’t use “compactOnLaunch” for my local realm. Should this always be used? What is the default setting, does it never compact if this setting isn’t specified?

The docs here: https://docs.mongodb.com/realm/sdk/ios/advanced-guides/compacting/ says that I should experiment with compaction to see what setting to use. I don’t mind adding a setting, but how do I know what is a good setting? I assume that compaction is done on first access in the same thread. Can I detect that this is needed before executing the compaction? Does compaction need to be run on a background thread?

@Simon_Persson Yes in production I would always have a compactOnLaunch callback setup. Compact is never run if this callback is not setup. A good setting is a tradeoff between how often you want to do it, since it will block the app startUp because you cannot open the Realm while it is being compacted, and how large you are okay with the file growing, but I think the example given in the docs is a good starting point of 50% free space. I would set the max file size something comparable to your average realm state size - so if you have an average realm state size of 10MB then set the max file size to 20MB or 40MB.

You can see in the below code snippet that the callback only returns true if the file size is greater than 100MB and there is 50% free space. If those conditions are not satisfied then compaction is not run.

let config = Realm.Configuration(shouldCompactOnLaunch: { totalBytes, usedBytes in
    // totalBytes refers to the size of the file on disk in bytes (data + free space)
    // usedBytes refers to the number of bytes used by data in the file
    // Compact if the file is over 100MB in size and less than 50% 'used'
    let oneHundredMB = 100 * 1024 * 1024
    return (totalBytes > oneHundredMB) && (Double(usedBytes) / Double(totalBytes)) < 0.5
})
do {
    // Realm is compacted on the first open if the configuration block conditions were met.
    let realm = try Realm(configuration: config)
} catch {
    // handle error compacting or opening Realm
}
1 Like

Thanks! I have been running the app for years without compacting😮 I think maybe this could be a bit more clear in the docs. It is listed in the advanced section.

Is there a way to detect if the Realm should be compacted? Should this be done on a background thread?

using asyncOpen() will automatically compact on a background thread and of course you could do it manually too. If you take a look at the code snippet you can see that totalBytes and usedBytes are passed into the compact callback as arguments - you can use this to see how fragmented your realm file is and then make a determination on whether to compact.

Thanks, but I am not sure AsyncOpen alone will solve this.

If I always use asyncOpen on the first open, then I will always compact the realm and increase startup time, if it is an expensive operation. I only want to do this when needed right?

Is it safe to do compaction on the main thread? Or is it better to always spin off a separate thread and then using the normal try Realm(configuration:config) and let this complete before my normal realm usage?

Hmm… I just changed the configuration so that the local configuration has a compaction callback. I noticed that the callback is called multiple times for the same configuration. I assume it will do this whenever there is no cached version? In practice I guess that the first open would take care of the compaction and that the following callbacks will return false, but it means that I don’t have full control over when compaction is done.

Then I guess I can’t guarantee that the compaction is done on a background thread. But I assume that it safe to do on the main thread as well? If not, should I use a separate configuration for the first open to guarantee that compaction only happens on first open?

This is confusing:

Will asyncOpen() automatically compact on a background thread

OR

Will asyncOpen() automatically compact on a background thread ONLY IF compactOnLaunch = YES

Sorry I should have been more clear - if you use asyncOpen AND compactOnLaunch is set then it is automatically done in the background. We never compact unless compactOnLaunch is explicitly set by the developer

FIY, I simply added the shouldCompactOnLaunch and changed the thresholds based on the recommendations here. I haven’t had any complaints so far from users, so I assume it is working. The crashes related to the database growing in size is gone now that I am using compaction and I am being more careful with frozen realms and write transactions.

Can you confirm compactOnLaunch works for synced and non-synced Realms ?

This topic was automatically closed 5 days after the last reply. New replies are no longer allowed.