Change events stream stops working when a node fails in ReplicaSet

[Giovanni_Desiderio]

Hello all and thanks admins for accepting me.

I am having a problem with mongodb (version 4.2), specifically with the Change Stream functionality.

I have a ReplicaSet cluster consisting of 1 primary, 1 secondary and 1 Arbiter and, in the code (Spring boot), I have a .watch () process on a collection of interest.

Basically, the stream works fine. When an insert / update operation occurs, the event is recognized and the document is streamed correctly.

However, when one of the two nodes (either the primary or the secondary) goes down, the watcher stops streaming anything.

Update/insert operations continue fine. Therefore, the program keeps interfacing correctly with the database, even after re-election of the primary.

However, the stream is blocked. As soon as I restart one of the two nodes, the stream immediately resumes correctly and also shows me the events not streamed previously.

Can anyone help me to solve this problem?

Thanks in advance.

[Pavel_Duchovny]

Hi @Giovanni_Desiderio,

Welcome to MongoDB community!

Starting MongoDB 3.6 change streams are using read concern “majority” by default which means that only majority node commited data will be streamed.

When your PSA replica has only one data node there is no majority commited data therefore it won’t stream

The good news for you is 4.2+ changestream has disabled the required need for read concern majority but you have to disable this default behaviour:

https://docs.mongodb.com/manual/reference/read-concern-majority/#disable-read-concern-majority

Btw its a recommended change for better stability for PSA replica sets anyway.we recommend moving to PSS and replacing arbiter with a secondary for overal rs HA…

Thanks
Pavel

[Giovanni_Desiderio]

First of all, thanks for your answer.
Are you suggesting that if I set “replication.enableMajorityReadConcern” equals to false within the .conf file of each of my replicaSet’s member, will my collection.watch() be able to stream insert/update anyway?
Honestly, I already tried it but my .watch() still does not work when one of the two data-bearing node is down… any other tip?

[Katya (Katya Kamenieva)]

Change streams do not report events that haven’t been majority-committed. Change stream will wait until the event has propagated to a majority of nodes. See this section in docs.
This is because an event which has not yet been majority-committed may be rolled back , and this also can lead to change stream becoming non-resumable.