Set a nonce field to a new ObjectId when inserting/updating

Working with Data
,
1

I’m trying to figure out a way to update a non-id ObjectId field to a new ObjectId() unique value when calling updateMulti.

db.document.update({}, 
 {
	$set: { updateId: new ObjectId() } <- I want each updated record to have a unique new objectId
 },
{ multi: true });

Is this possible?

2

Hello,

In my test case, it does not find working. The issue is it duplicates.

test> db.test.insertMany([{'non_id' : null}, {'non_id' : null}]);
test> db.test.find();
[
  { _id: ObjectId('65e7dc6c9845eedc848ed269'), non_id: null },
  { _id: ObjectId('65e7dc6c9845eedc848ed26a'), non_id: null }
]
test> db.test.updateMany({}, { $set : { non_id : new ObjectId() } });
test> db.test.find();
[
  {
    _id: ObjectId('65e7dc6c9845eedc848ed269'),
    non_id: ObjectId('65e7dcb99845eedc848ed26b')
  },
  {
    _id: ObjectId('65e7dc6c9845eedc848ed26a'),
    non_id: ObjectId('65e7dcb99845eedc848ed26b')
  }
]

test> db.test.createIndex({non_id:1},{unique:true});
MongoServerError: Index build failed: d7f38555-b3d9-4797-9246-b8428906a507: Collection test.test ( 12c6cf49-1207-4ee7-80f0-f236f65d54fa ) :: caused by :: E11000 duplicate key error collection: test.test index: non_id_1 dup key: { non_id: ObjectId('65e7dcb99845eedc848ed26b') }
test>

Thanks
WeDoTheBestFor4

3

Hello,

Please see the post if helpful. MongoDB update field value as ObjectId - Questions - n8n

Will this help?

db.test.insertMany([{},{}]);
db.test.find();
[
  { _id: ObjectId('65e831268b05ee991300eea2') },
  { _id: ObjectId('65e831268b05ee991300eea3') }
]
db.test.aggregate([ { $set: {"zObjectId": "$_id" }}, {"$out": "test" } ]);
db.test.find();
[
  {
    _id: ObjectId('65e831268b05ee991300eea2'),
    zObjectId: ObjectId('65e831268b05ee991300eea2')
  },
  {
    _id: ObjectId('65e831268b05ee991300eea3'),
    zObjectId: ObjectId('65e831268b05ee991300eea3')
  }
]

Thanks
WeDoTheBest4You

4

This is expected since

new ObjectId is called once by the shell/driver before the updateMany is sent to the server.

That is probably useless at is simply duplicate the _id under a new name, so what ever you could do with zObjectId could be done directly with _id without the duplication. This also assume that _id is already an ObjectId, which is not clear from the OP. May be the _id are UUIDs rather than ObjectId.

I could not find an API that could do that directly.

My approach to generate a new ObjectId would be to aggregate the document collection to project _id as owner_id into a temporary collection.

db.document.aggregate( [
    { $match : { updateId : { "$exists" : false } } } ,
    { $project : { _id : 0 , owner_id : "$_id"} } ,
    { $out : "temp" }
] )

The next step will aggregate the temp collection to $merge it back into the original collection.

db.temp.aggregate( [
    { "$set" : {
        _id : "$owner_id" ,
        updateId : "$_id" 
    } } ,
    { "$merge" : {
        "into" : "document" ,
        "on" : "_id" ,
    } }
] }

LATE ADDITION

It might be possible to set a view with the aggregation that creates the temporary collection. This might be more efficient since the resulting temp will not be written to disk.

5

Thanks for the comments/views. To be clear, the _id fields already exists on the document and the intention is to have a separate non _id field that for every update, updateMulti, etc. called, each individual record could be updated with a new unique ObjectId ideally generated by mongo itself.

6

What are you using this new field for? Is it an update check to see if you’re looking at an old version of data?

7

It’s being used as a nonce for paging for the purpose of syncing data to a mobile device (can’t use realm; we tried).

8

Does it have to be an objectID? You could take the current objectID convert to string and append a version of the current server time to create a new value?

db.getCollection("Test").update({},
[
    {
        $set:{
            NewFieldToSetOrUpdate:'Thingy',
            nonce:{
                $concat:[
                    {$toString:'$_id'},
                    {$toString:{$toLong:"$$NOW"}}
                ]
            }
        }
    }
])

It’s not like you’ll get a clash here as the objectID will be unique anyway.

I’ve not done this scenario before so apologies if I’ve got the wrong end of the stick!

9

@John_Sewell it’s a good idea and we have actually already been down that path of using a timestamp but ultimately it didn’t work. The reason is because we need a way to guarantee, when we’re paging through data, that the basis we’re using for paging will be greater than (or less than depending on the direction of paging). So even if we were to say prepend the $$NOW, the problem it that the value of $$NOW would be the same for all the values of the records that are being updated. However even if it weren’t, using a timestamp in this manner doens’t guarantee that each record being updated wouldn’t have the same value for the timestamp even if it were to nano-second precision.

Anyhoo, our solution for the time being is to basically not perform updateMulti’s but rather to perform bulk updates with each record being updated individually with it own objectId being generated by our java API. We will be writing an article detailing our approach. I’ll try to remember to post it back here to this thread.

10

Thank you @steevej for your feedback on the update, also for the proposed solution. What could be gathered so far is, new ObjectId() is called for each only on “insert”. In your proposal, when the temporary store is inserted, then the ObjectId will get generated. Please see another instance below via insertMany.

It all says the same, ObjectId() call will be made with respect to each insertion. It does not do so with respect “update”.

db.test.insertMany({zobjectId : new ObjectId()}, { zobjectId : new ObjectId()});
db.test.find();
[
  {
    _id: ObjectId('65e95f1cc1df6a2f9b6e243a'),
    zobjectId: ObjectId('65e95f1cc1df6a2f9b6e2438')
  },
  {
    _id: ObjectId('65e95f1cc1df6a2f9b6e243b'),
    zobjectId: ObjectId('65e95f1cc1df6a2f9b6e2439')
  }
]

Thanks
WeDotheBest4You

11

I do not understand what you trying to explain.

An new ObjectId is created when ever you call explicitely call new ObjectId(). Also a new ObjectId() is created when even ever a document that does not have the _id field is inserted in a collection. In your example:

You insert 2 documents that do not have the _id field. You call new ObjectId() twice. So you end up with 4 different $oid. This is expected. In

You call new ObjectId() once, so all documents will updated with the same value.

As mentioned, by John_Bonds1, none of the solution solve his issue. I originally, understood that he wanted to convert his data to have the new field. In his last post it became clear that he needs to do that whenever the data is updated. John_Sewell idea was the closest to match the requirement but does not work as explain by John_Bonds1.

I think that the post

is the only solution.

12

Hello @steevej , I agree with all of your points in your latest update.
I made a confusion in my latest post. Sorry for that. Kindly ignore it.