Keeping Extended Reference Pattern up to date

Hello everyone, a newbie in mongodb and document schema modeling. I have a question about techniques to keep extended references up to date. Let’s say I have two collections: an aliment collection and a meal collection. Each aliment is created and update by the user. Each aliment has a name and a list of nutritional values. Each meal is created and updated by the user. Each meal has a name, a list of extended reference to Aliments and a computed nutritional values based on the nutritional values of each aliment of the meal.

Assume now that the user changes the name from PASTA to PASTA Barilla. What are the best techinques to propagate that change to all meals that reference the old PASTA aliment?

For the moment I can think of the following:

When the user updates the aliment, I check each Meal and update the extended reference.In this case, I will have one transaction updating the Aliment document and a second bulk operation to update all the meals.

But what happen if for some reason one meal update fails because the user updated the meal in the meantime?

Thank you

Hi @Green,

Thats an interesting design considerations.

For some design the extended reference could be very minimal due to update consideration.

The idea is to keep user specific data only on the users documents and updates that might be cross user/meals changes to another reference collection. Keeping data in both places makes sense only if it benefits the queries and not changed often.

Now I have a few questions to better help you.

  1. How often would names of products might be changed?
  2. What other fields are expected to change over meals? Do you expect ingridants to change nutrition values?

In general I think that to guarantee consistency across collection you will need to:

  1. Either use transactions to update the meals and aliment and finally commit or rollback.
  2. Have in the extended reference only data that needs to be shown using the query to meals and does not need to be updated over time. If all fields could be updated maybe it makes sense to store the id’s of those ingridants and make 2 queries to show them.

Thanks,
Pavel

Hi @Pavel_Duchovny, thank you again for your answer! To be honest, I suppose that Aliment’s name and nutritional values are not going to change often. Probably, it would be better to keep the created aliment read only and let the user create a new aliment out of it. Thinking about it, since the Aliment is not only the name, but also its nutritional values, if you update it, you are probably creating a new aliment. Also, wouldn’t it be weird that if you have 10 meals with the same aliment and when you change it all the meals gets updated without your knowledge?

For your first general consisentcy concern, does that mean that I need to implement compensating transactions in case subsequent ones fail?

Thank you

Hi @Green,

The transactional behaviour make sense if you want users to see or update only commited data and if one changes something outside of a transaction it won’t succed until transaction is committed or rollback.

If your flows only update one independent document I would use retrayable writes as a retry mechanism rather than transactions.

Regarding the data model, it sounds like you can potentially create draft meals for approval to all the users ones there is a change.

Potentially, you can run the update by creating new versions of the user meals:

db.meals.findAndUpdate({userid : ..., Aliments:  previousAliment},{$set: {status : "inDraft}})
...
db.meals.update({mealid : ...},{$set : { Aliments : [ {...}] , status : "approved"}

So application only count on approved meals and mark those in change as inDraft. This way even if you fail the version will still be in draft and not yet approved.

Let me know if that makes sense.

Thanks
Pavel

Hi Pavel, I did not think about drafts, thank you!
For transactions, some of the flows will touch in fact multiple documents, mostly to notify that the data of a related document changed, but in general these changes are not going to trigger business logic on other document rather than simple updates of the view data.
Since Im new to mongo and I dont really like retriable stuff, I would like to try follow the DDD style and use a single transaction per document. If an action that touch a document trigger something on another document, I will trigger a transaction on that document and compensate on the first transaction if necessary (most of the failures will be due to versioning and Im expecting much more reads than writes) (looks like a saga). Everything is for now synchronous, if it works well I will move to async communication between documents.

Thank you

Green

@Pavel_Duchovny I had to change my model. I realized, that the users may have personal aliments, and putting all inside the same collection may lead to performance problem when querying that data. I was thinking to create a AlimentCollection document and have inside a nested array of aliments. Correct me if Im wrong, this way I can:

  1. Regulate the number of aliments created
  2. Reduce query overhead since I will first ask for the collection using the userId index and then select on the nested array and project for example only the name of the aliment.

Now, I have some doubts:

  1. Is it possible to implement paging and text search on the nested Aliment array?
  2. I saw that is possible to update a single element in the array without having to extract the whole document. Does this apply for insertion and deletion of a new Aliment?
  3. To implement versioning (but Im not sure if I need it since only one user is updating the Aliment Collection), can I add a condition to the update statement to check the given version and the actual version in the db?

Thank you very much!

Green