Is there something similar to journal for read operations in mongodb?

Working with Data
1

I am currelying devoloping a program that read documents from mongo and write them in a file… something like this:

 for doc in db.col.find(field=="bla"):
   file.write(doc)

my problem is that something can happen while doing this process (it gonna take a week to do all the writes), for example, a shutdown or network problem. my question is… is there something similar to journal for write operations to recover from a checkpoint? So i dont need to do all the write to file all over again.

2

Hi Jonathan,

Could you elaborate on the purpose of the code? It looks like you’re trying to dump the whole collection. Are you trying to backup up the whole collection?

If backup for restoring into another MongoDB server is the goal, you might want to examine mongodump and mongorestore, where they have the --oplog option that caters for exactly this purpose (performing a dump and taking note of writes while the dump operation is ongoing).

Having said that, this feature would require you to have a replica set deployment, as the oplog is not available on a standalone mongod deployment.

If you need further help with this, could you post additional details:

  • What is your goal
  • What is your deployment topology (standalone/replica set/sharded cluster)
  • What is your MongoDB version
  • Anything you have tried that failed to work

Best regards,
Kevin

1 Like