Fix db.mycollection.count() after mongo crash recovery

Ops and Admin
1

I have a mongo setup with 2 Billion entries in a collection with size of around 500GB.

There was a hard crash because of memory issue. And when mongod restarted after a hard crash (and a successful recovery) the values returned by ‘db.mycollection.count()’ is different than that of db.mycollection.find().hint(‘ix_retention_date’).count() (full index hint). I believe it is because of already known issue https://jira.mongodb.org/browse/SERVER-19472.

now I want to fix this, and the ticket suggests that we use
db.mycollection.validate({ full: true}) to fix this. But when I run that command it’s very slow and looks like it will take more than few day for this command to complete.

Even I tried db.mycollection.validate({ full: false}) and it is still running for 12 hours now and slow but quite fast compared to running with true flag. But I am not sure if this will fix the count issue .

So what is the best way to fix the count thing. Is there a easy way out, surely it should have –
I don’t believe every time mongo crashes and we will need to run db.mycollection.validate({ full: true}) that takes ages to fix count ?

Mongo version is 4.4

2

@Sameer_Kattel,

Use the db.collection.countDocuments() instead of count. From the documentation:

Unlike db.collection.count() , db.collection.countDocuments() does not use the metadata to return the count. Instead, it performs an aggregation of the document to return an accurate count, even after an unclean shutdown.

From the db.collection.count():

IMPORTANT

  • Avoid using the db.collection.count() method without a query predicate; since without the query predicate, the method returns results based on the collection’s metadata, which may result in an approximate count. In particular,
4

@Prasad_Saya,

So it is ok to let db.collection.count() to remain incorrect?
And the only wat to fix it is running validate command ? Should run full validation db.mycollection.validate({ full: true}) or db.mycollection.validate({ full: false}) is good enough?

If there is no other way, then wondering why there is no way to only update meta data count results with db.collection.countDocuments() count which would be much much faster?

Thanks

5

It is up to you to determine that. Why did you need to count the documents (please ask the question to yourself)?

The db.collection.validate() method’s { full: <boolean> } is optional.

The default value is false. It is a flag that determines whether the command performs a slower but more thorough check or a faster but less thorough check. If false, omits some checks for a faster but less thorough check. Also, note:

  • The db.collection.validate() method is potentially resource intensive and may impact the performance of your MongoDB instance
  • The db.collection.validate() method obtains an exclusive lock on the collection. This will block all reads and writes on the collection until the operation finishes.

If you want an accurate (and exact) count of documents in the collection, use the countDocuments method.

6

@Prasad_Saya

https://jira.mongodb.org/browse/SERVER-19472 suggest that it should be run with true flag. So I was asking if both version would fix metadata, as I was not sure.

db.collection.validate() is the only way to fix the metadata count? It is painfully slow either running with full option set to true or false. It took around 17 hours for db.collection.validate() on system with 16vcpus and 64 GB ram
so wondering if there is more quicker way to fix that.

Also countDocuments is pathetically slow – it has been running for more than 12 hours and it still is not completed.

Thanks