Change stream does not contain removed fields when nested array object is deleted

Team,

When end user deletes a nested array object, respsective change stream document doesnt show deleted entity in updatedescription.removedfields also fulldocument is null, is there a way to get this. We do not need the fullDocuement we are only looking for the deleted object under removed fields ?

Example sample doc structure:

When a user deletes array.objX we need that obj in removed fields.

_id:<>
field1:<>
field2<>
Arraysample:Array
-0:obj1
-1:obj2
-2:obj3

Hi @vinay_murarishetty and thanks for your question :smiley: !

It’s currently impossible to retrieve the “old” version of the document in a change stream. The only thing you can do is retrieve the entire document “post” update by using the updateLookup option:

#!/usr/bin/env bash
echo '
cursor = db.users.watch([],{"fullDocument":"updateLookup"});

while (!cursor.isExhausted()) {
  if (cursor.hasNext()) {
    print(tojson(cursor.next()));
  }
}' | mongo

The fullDocument field in your change event is always populated for an insert or a replace. It can be populated if you provide the updateLookup option for an update and is never provided for a delete operation as there is no document anymore… But it’s always the most recent majority committed version of the document that you get.

Stay tune for MongoDB 5.0 though because it could potentially change fairly soon :nerd_face: !

More info at https://www.mongodb.com/live! No spoilers (almost)!

1 Like

Hi @MaBeuLux88,

Thanks for the update, but here the operation type is update where user deletes a nested array object(not the complete document) with in the main document as mentioned in above structure. In this case why the updatedescription.removefields doesnt contain the removed object ? We need this to be populated for our use case.

What you described is an update of the array, not a deletion of field. That’s why it appears in the updates and not in the deletes.

Let me explain with an example. First I insert a document:

> db.coll.insertOne({firstname:"Maxime",surname:"Beugnet", pets: [{name: "Bob"},{name: "Raoul"}]})
{
	"acknowledged" : true,
	"insertedId" : ObjectId("60a52dc14328bfa256cd18ba")
}

Then I remove one of my pet from the pets array by updating it with a $pop operation:

> db.coll.updateOne({}, {$pop: {pets: 1}})

Which results in this change event:

{
	"_id" : {
		"_data" : "8260A52E05000000012B022C0100296E5A10041EA1C19171774980BB0020FFA9BD24F746645F6964006460A52DC14328BFA256CD18BA0004"
	},
	"operationType" : "update",
	"clusterTime" : Timestamp(1621437957, 1),
	"ns" : {
		"db" : "test",
		"coll" : "coll"
	},
	"documentKey" : {
		"_id" : ObjectId("60a52dc14328bfa256cd18ba")
	},
	"updateDescription" : {
		"updatedFields" : {
			"pets" : [
				{
					"name" : "Bob"
				}
			]
		},
		"removedFields" : [ ]
	}
}

As expected, the pets field is the one that has been updated. One of its value (here a subdocument) has been removed.

If I $unset one of my fields though:

> db.coll.updateOne({}, {$unset: {surname: 1}})

I do get this change event that contains, this time, a field deletion:

{
	"_id" : {
		"_data" : "8260A52E3A000000012B022C0100296E5A10041EA1C19171774980BB0020FFA9BD24F746645F6964006460A52DC14328BFA256CD18BA0004"
	},
	"operationType" : "update",
	"clusterTime" : Timestamp(1621438010, 1),
	"ns" : {
		"db" : "test",
		"coll" : "coll"
	},
	"documentKey" : {
		"_id" : ObjectId("60a52dc14328bfa256cd18ba")
	},
	"updateDescription" : {
		"updatedFields" : {
			
		},
		"removedFields" : [
			"surname"
		]
	}
}

This works as designed.

I hope this helps. Sorry if it’s not what you expected. There is currently no work around to “fix” this to my knowledge, but stay tuned for MongoDB 5.0…

Cheers,
Maxime.