Broken Replica Set Config

I am not sure how it did it, but my replica set is broken, I am cannot figure out how to reset. I got to the point of the lesson where I set up the Sharding server and started to modify the node config files. But when I tried to connect to the replica set I found I had no primary server, and connecting to each server directly showed they are running, but not part of the replica set.
MongoDB Enterprise m103-example:OTHER> rs.config()
{
“_id” : “m103-example”,
“version” : 7,
“protocolVersion” : NumberLong(1),
“members” : [
{
“_id” : 0,
“host” : “192.168.103.100:27011”,
“arbiterOnly” : false,
“buildIndexes” : true,
“hidden” : false,
“priority” : 1,
“tags” : {

                    },
                    "slaveDelay" : NumberLong(0),
                    "votes" : 1
            },
            {
                    "_id" : 1,
                    "host" : "m103.mongodb.university:27012",
                    "arbiterOnly" : false,
                    "buildIndexes" : true,
                    "hidden" : false,
                    "priority" : 1,
                    "tags" : {

                    },
                    "slaveDelay" : NumberLong(0),
                    "votes" : 1
            },
            {
                    "_id" : 2,
                    "host" : "m103.mongodb.university:27013",
                    "arbiterOnly" : false,
                    "buildIndexes" : true,
                    "hidden" : false,
                    "priority" : 1,
                    "tags" : {

                    },
                    "slaveDelay" : NumberLong(0),
                    "votes" : 1
            },
            {
                    "_id" : 3,
                    "host" : "m103.mongodb.university:27014",
                    "arbiterOnly" : false,
                    "buildIndexes" : true,
                    "hidden" : true,
                    "priority" : 0,
                    "tags" : {

                    },
                    "slaveDelay" : NumberLong(0),
                    "votes" : 0
            }
    ],
    "settings" : {
            "chainingAllowed" : true,
            "heartbeatIntervalMillis" : 2000,
            "heartbeatTimeoutSecs" : 10,
            "electionTimeoutMillis" : 10000,
            "catchUpTimeoutMillis" : -1,
            "catchUpTakeoverDelayMillis" : 30000,
            "getLastErrorModes" : {

            },
            "getLastErrorDefaults" : {
                    "w" : 1,
                    "wtimeout" : 0
            },
            "replicaSetId" : ObjectId("5d6acd66f6cfcf90a95107fb")
    }

}
MongoDB Enterprise m103-example:OTHER> rs.status() { “state” : 10, “stateStr” : “REMOVED”, “uptime” : 285, “optime” : { “ts” : Timestamp(1567351227, 1), “t” : NumberLong(4) }, “optimeDate” : ISODate(“2019-09-01T15:20:27Z”), “lastHeartbeatMessage” : “”, “syncingTo” : “”, “syncSourceHost” : “”,
“syncSourceId” : -1,
“infoMessage” : “”,
“ok” : 0,
“errmsg” : “Our replica set config is invalid or we are not a member of it”,
“code” : 93,
“codeName” : “InvalidReplicaSetConfig”,
“operationTime” : Timestamp(1567351227, 1),
“$clusterTime” : {
“clusterTime” : Timestamp(1567351227, 1),
“signature” : {
“hash” : BinData(0,“KIGQxYZQfc2Oa/CE/g8rJ9opFd4=”),
“keyId” : NumberLong(“6731418439618920449”)
}
}
}
MongoDB Enterprise m103-example:OTHER>

How do I reset/fix I have one day before I need to complete this chapter.

Best thing would be to start from scratch
Did you complete rs.initiate() and rs.add() in first attempt
Did your rs.status() show all nodes working (one primary and 2 secondaries)

I did not have any issues in the replication chapter. rs.status() shows the servers have been removed, with an error “Our replica set config is invalid or we are not a member of it”.

Hi @joe.e.jamison,

This means that the nodes in your replica set are not able to talk to each other and that’s why you are seeing this error message. As you mentioned you are able to connect to individual nodes but the replica set as a whole is broken.

In this case, what you can do is you can reconfigure your current node by editing its members.

var rsconf = rs.conf()

Let’s say the current host to which you are connected to is the 0th member of the replica set then do this.

rsconf.members = rsconf.members[0] 

Now, reconfigure the current replica set using the rsconf variable.

rs.reconfig(rsconf, {force: true})

Once done, you can go ahead and add other nodes to the current replica set using rs.add() function.

I hope this helps. However, if the issue still persists please feel free to get back to us.

Happy Learning :slight_smile:

Thanks,
Muskan
Curriculum Support Engineer

Hi,

I’m having the very same issue as the one described by joe.e.jamison. I have followed the guidance from Muskan_47318 and can’t get through the command:

rs.reconfig(rsconf, {force: true})

I’m getting error:

{

“ok” : 0,

“errmsg” : ““members” had the wrong type. Expected array, found object”,

“code” : 93,

“codeName” : “InvalidReplicaSetConfig”

}

Can anybody help please?

thank you in advance!

The error is “members” had the wrong type. Expected array, found object

Because when you did rsconf.members = rsconf.members[0] you changed the type of members from an array to an object, the first object of the array.

Thank you for responding.

I believe that was the intention from the instruction posted by Muskan_47318.

Anyway, I have solved this issue now - my repl set name did not match the updated .conf files (the lecture instructed to update it to m103-repl, but the initial node1-node3 files had m103-example).
Once I’ve restored my previous node1-node3 files and log on to replica set (which was successfull) and changed the names on the newly updated files to match the already setup replica set name (m103-example), I was able to complete the task and reconfigure replica set.

I did not read Muskan message before replying. But it looks like he was missing a step. Replacing the array members with another array is possible you need an extra step.

temp[0] = rsconf.members[0]
rsconf.members = temp

The first line creates a temp array of 1 element from the 1st element of rsconf.members.
The second one replace the rsconf.members with the temp array.

thanks for that!

I will remember for the future, might come useful.

:slight_smile:

Re-quoting the problem line for brevity:

:arrow_up:
:arrow_down:
rsconf.members.splice(0, 50, rsconf.members[0])

… same as @steevej-1495’s but using a one-liner splice().
I chose 50 because of this, however, you can substitute it for any number >= the total number of items in the array.

1 Like

Thanks for remember me the splice method. And if I may add, more information about splice can be found at https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Global_Objects/Array/splice.