Why close_idle_time defaults to 100000s?

Other MongoDB Topics MongoDB Server Development
1

Hi, all.

I find close_idle_time defaults to 30s in wiredtiger(config_def.c)，why does mongo 4.4 change it to 100000(~28h) ? (related to https://jira.mongodb.org/browse/SERVER-41492, I know 4.0 also uses 100000.)

If I create many collections and don’t access them for a long period of time, is it a good idea to sweep them from memory quickly and make room for other active collections ?

And can’t we change it at runtime ?

2

We actually changed the default for this recently:
https://jira.mongodb.org/browse/SERVER-24949

The context for the original setting of 28 hours is here:
https://jira.mongodb.org/browse/SERVER-17907

Note that idle collection pages don’t need to be removed from memory to “make room for other active collections”, as the eviction policy will do that automatically anyway. The real reason to close idle file handles is to remove the overhead of the bookkeeping and resources necessary to keep each file open. The removal of pages in memory associated with a file handle being closed is a necessary (and somewhat undesirable) side-effect.

2 Likes
3

Thanks for replying.

Eviction does work when there’s cache pressure. But if eviction cannot keep up with the rate of collection being created and inserted (We once saw performance problem of eviction efficiency in mongo 4.0 production, that is, cache_used is hard to decrease even if we increase number of eviction worker), reducing close_idle_time will help sweep inactive pages, right ?

I suppose changing to 10min also applies to mongo 4.0 ? (For some reason we’re still using mongo 4.0.)

4

I think you are talking about eviction of dirty cache pages. Clean cache pages are simple to evict, and idle file handles only have clean pages in the cache (unless your checkpoints are taking longer than the idle sweep time; in such a case, lowering the idle sweep time will do nothing to improve this).
If you are having cache pressure, that typically means the number of dirty cache pages has hit the dirty cache limit, and no amount of freeing clean cache pages will do anything to alleviate that.

5

The default value for close_idle_time in many software systems is 100,000 seconds, or approximately 27.8 hours, for a few reasons:

  1. Relevance: This time frame allows for a balance between keeping connections open for short-lived transactions and closing idle connections that are no longer needed.
  2. Efficiency: Closing idle connections frees up system resources, such as memory and file descriptors, that can be used by other active connections.
  3. Security: Long-lived idle connections can be vulnerable to attack and therefore it is good practice to close them after a certain amount of time has passed.

This default value is often considered a reasonable compromise between the need for efficient resource usage and the risk of breaking active connections. It can be adjusted based on the specific needs and requirements of the system.