C# MongoDB.Driver - Replica Set with an instance in Recovery

[Stefano_Curcio]

I have a MongoDB Replica Set with 3 instances. One of them was in Recovery for a couple of weeks. A .net Core (C#) application continued reading from the instance in Recovery. This .net Core application uses MongoDB.Driver 2.11.6 .

My questions are:

1. Shouldn’t MongoDb.Driver have ruled out the instance in Recovery?
2. Is there a way to rule out instances in Recovery?

Thanks.

[James_Kovacs]

Hi, Stefano,

Thank you for contacting MongoDB. We understand that one member of a 3-member replica set was in recovering for 2 weeks, but your .NET Core application continued to read from this node.

If you connect to a replica set with the .NET/C# driver, it will select a replica set node to read from based on the configured read preference for the operation (possibly defaulted from the connection string). Even if the driver selected a recovering node due to stale topology information, the recovering mongod instance will not service reads and instead will return an error similar to the following:

{
  "topologyVersion": {
    "processId": ObjectId("6091db349e24f4ec09d5b60e"),
    "counter": NumberLong("5")
  },
  "ok": 0,
  "errmsg": "node is recovering",
  "code": 13436,
  "codeName": "NotPrimaryOrSecondary"
}

To read from a recovering node for data recovery purposes, you would have to restart the node as a standalone on a different port and then connect directly to that standalone node on that new port.

To answer your questions:

1. Not only will the .NET/C# driver not select a recovering node, but even if it did, that node would refuse read and write operations instead returning an error.
2. Ruling out recovering nodes should be done automatically by both the driver and the mongod instances.

If you are seeing different behaviour than described above, please provide a self-contained reproduction of the issue so that we can investigate further.

Sincerely,
James

[Stefano_Curcio]

We do not have a readPreference in the connection string, so the default is primary.
Our connection string has this shape: mongodb://user:password@x.x.x.x:27017,x.x.x.x:27017,x.x.x.x:27017/?authSource=auth_source
While Java applications logged an error (Command failed with error 211 (KeyNotFound): 'Cache Reader No keys found for HMAC that is valid for time: ) there were not errors or warning in the logs of C# applications.

I will investigate further.

Thank you @James_Kovacs.

[James_Kovacs]

Hi, Stefano,

Thank you for the follow-up. You are correct that the default read preference will be primary if left unspecified.

Reviewing your connection string, you do not specify the ?replicaSet=<<replSetName>> option. The driver will connect to the node as a standalone and will use the first host that can be resolved. This is why you continued to connect to the node after it went into recovering.

Please try specifying your replica set name in ?replicaSet=<<replSetName>> in your connection string for both the .NET/C# and Java applications. This should resolve the problem that you are observing.

Sincerely,
James