MongoDB.live, free & fully virtual. June 9th - 10th. Register Now

Failed to run query after 10 retries :: caused by :: version mismatch detected

I was running multiple aggregate/$merge (in python script) over the same collection and suddenly I got an error OperationFailure: “version mismatch detected for my_db.my_collection”

Now, in mongo cli, if a try a
db.my_collection.findOne()
I get

mongos> db.my_collection.findOne()
2020-03-31T16:48:32.280+0000 E  QUERY    [js] uncaught exception: Error: error: {
	"ok" : 0,
	"errmsg" : "Failed to run query after 10 retries :: caused by :: version mismatch detected for my_db.my_collection",
	"code" : 13388,
	"codeName" : "StaleConfig",
	"ns" : "my_db.my_collection",
	"vReceived" : Timestamp(857, 21),
	"vReceivedEpoch" : ObjectId("5e7e58930972ff979849eb57"),
	"vWanted" : Timestamp(855, 33),
	"vWantedEpoch" : ObjectId("5e7e58930972ff979849eb57"),
	"operationTime" : Timestamp(1585673310, 2),
	"$clusterTime" : {
		"clusterTime" : Timestamp(1585673311, 7),
		"signature" : {
			"hash" : BinData(0,"R0DXB77kTWRd47etf7iLrlreUq0="),
			"keyId" : NumberLong("6790649384410284063")
		}
	}
} :
_getErrorWithCode@src/mongo/shell/utils.js:25:13
DBCommandCursor@src/mongo/shell/query.js:696:15
DBQuery.prototype._exec@src/mongo/shell/query.js:111:28
DBQuery.prototype.hasNext@src/mongo/shell/query.js:282:5
DBCollection.prototype.findOne@src/mongo/shell/collection.js:255:10
@(shell):1:1

I tried to do a flushRouterConfig but it does not help.

Any idea of how to solve this problem and regain access to the collection ?

Context : mongodb 4.2.3, sharded replicated cluster

I had a look to the shards mongodb.log and in one of them (and only one), I have warnings like this one :

2020-03-31T15:48:23.552+0000 W SHARDING [conn1894082] requested shard version differs from config shard version for my_db.my_collection, requested version is 857|21||5e7e58930972ff979849eb57 but found version 855|33||5e7e58930972ff979849eb57

which correspond to the timestamps in the error message…