If I understand this correctly, while the data copy is happening from the source to the target, any data changes picked up on the source are written to the local database on the target. Once the data copy is complete, the changes in the target local are then executed to ensure the source and target are in sync? This is the behaviour from 3.4 onwards.
The Oplog is 150GB. My thinking is that if we generated 300GB of data changes, they would be synced to the local on target and then replayed once the data copy stage has completed?
I was always under the impression that the data copy phase had to complete before the Oplog records were overwritten. This may have been the behaviour in 3.2 and earlier. Or it could be that I never properly understood this area!
The reason for asking is that I am syncing a very large database. The replication oplog window for this replicaset can vary between 1 and 4 hours - it is a very volatile database in terms of collection’s being created and dropped very frequently.
An increase in the Oplog size is something being considered while as a short term solution until we can look to add sharding into the mix - an 8.5TB shard isn’t particularly fun.
Thanks for the link - I didn’t think to read up on 3.2 and confirm what I thought may be the case.
I’d like to directly answer some of your questions:
If I understand this correctly, while the data copy is happening from the source to the target, any data changes picked up on the source are written to the local database on the target. Once the data copy is complete, the changes in the target local are then executed to ensure the source and target are in sync? This is the behaviour from 3.4 onwards.
Correct.
The Oplog is 150GB. My thinking is that if we generated 300GB of data changes, they would be synced to the local on target and then replayed once the data copy stage has completed?
Correct.
Previously in MongoDB pre-3.4, initial sync happens in two distinct phases: data copy → oplog pull & apply. The primary would need to have enough oplog to cover the whole initial sync time. In some cases, this could be hard to approximate.
Starting from 3.4, the oplog was tailed during the data copy phase, eliminating the need for the primary to have a large oplog to cover the data copy phase. However, the primary would still need to have oplog coverage during the oplog apply phase. This is typically a much smaller window than the data copy phase, though.
Thanks for this - I’d pretty much confirmed my thoughts whilst running through the logs / monitoring the process but appreciate your confirming this.
I’ve read up on the 4.4 features and these are of value to us. We have some software on 4.2 and other software on 3.6 but will be looking at 4.4 at some point next year!