Hello,
I want to create an architecture with hot data (0 - 3 month period) and cold data (0 - unlimited).
Actually I already have the hot database, a mongo cluster sharded on transaction key.
Our transactions are document with many upsert during the first 10 minutes so I want a 1 hour delay between the data creation in the hot and the copy in the cold. So I can have a really small architecture for the cold with 30x less writing per second.
My question is how can I do that ? I don’t see any tool with mongo to real time + delay copy database and I see with kafka connect but it does not work as I expected.
Thank you.