We need to store in mongo around 20B documents, where each document is tied into a specific id.
Also, each document has an expiration and needs to be deleted after a configurable amount of time.
We came up with this model for storing the documents:
The documents are small, and from our testing we can fit around 190,000 documents per _id, but we only expect around 5000.
The document are queried by the _id and a date for the nested documents. We use a simple $unwind aggregation to get the results.
My only fear is the deletion time. Each document has a date, but from what I understand one can’t create TTL index on nested document, so we needed to create a cron for it.
Can you suggest a better solution?