My Node app uses Mongo change streams, and the app runs 3+ instances in production (more eventually, so this will become more of an issue as it grows). So, when a change comes in the change stream functionality runs as many times as there are processes.
How to set things up so that the change stream only runs once?
Feed the change-stream into a message queue and have all your processes take changes from the task queue. the task queue will ensure that only one process gets each task. The input to the message queue can be the change-stream.
Yes that is one solution that I am working using aws sqs , I think mongodb should also provide this option of exactly once read , like if I open change stream from 5 places then only one process will get it , but for now seems it is not possible , Thanks for clarification btw .
mongodb should also provide this option of exactly once read , like if I open change stream from 5 places then only one process will get it
That’s not really possible without MongoDB implementing a queue/message broker that’s built on top of change streams… Seems like there are already quite a few software packages that do exactly this…
Hey, I have the same question. So basically, we must only deploy app with change stream (that feed into message-queue) into a single server / single pod?