Why is node state "REMOVED"?

My CSRS node states “OTHER” when it should be PRIMARY, and when I ask for status, it says REMOVED. Why is this the case?

If you look at the rs.initiate() there’s an error message saying that there already was a replica set configuration present. So the init actually failed.

Hi David_23362,

@Tess_Sluijter is right!. The error message states that the “Replica Set has already initialized”.

Have you by any chance run rs.initiate() from any other node? If you have, then do rs.add("192.168.103.100:26001") from other node which has already been initialized.

And if you have not, check replset configuration as mentioned in the info key of the error message.

  1. Authenticate using user: m103-admin.

    use admin
    db.auth("m103-admin","m103-pass")
    
  2. Check for replset configuration.

    use local
    db.system.replset.find()
    

Kanika

1 Like

Thanks for the reply! I ran db.system.replset.find() and got this:

MongoDB Enterprise m103-csrs:OTHER> db.system.replset.find()
{ “_id” : “m103-csrs”, “version” : 3, “configsvr” : true, “protocolVersion” : NumberLong(1), “members” : [ { “_id” : 0, “host” : “192.168.103.100:27001”, “arbiterOnly” : false, “buildIndexes” : true, “hidden” : false, “priority” : 1, “tags” : { }, “slaveDelay” : NumberLong(0), “votes” : 1 }, { “_id” : 1, “host” : “192.168.103.100:27002”, “arbiterOnly” : false, “buildIndexes” : true, “hidden” : false, “priority” : 1, “tags” : { }, “slaveDelay” : NumberLong(0), “votes” : 1 }, { “_id” : 2, “host” : “192.168.103.100:27003”, “arbiterOnly” : false, “buildIndexes” : true, “hidden” : false, “priority” : 1, “tags” : { }, “slaveDelay” : NumberLong(0), “votes” : 1 } ], “settings” : { “chainingAllowed” : true, “heartbeatIntervalMillis” : 2000, “heartbeatTimeoutSecs” : 10, “electionTimeoutMillis” : 10000, “catchUpTimeoutMillis” : -1, “catchUpTakeoverDelayMillis” : 30000, “getLastErrorModes” : { }, “getLastErrorDefaults” : { “w” : 1, “wtimeout” : 0 }, “replicaSetId” : ObjectId(“5c11a91100adfe76cfa59db2”) } }

1 Like

Hi David_23362,

I have one concern after looking at your replica set configuration:

  • The replica set “m103-csrs” replica set has nodes: 27001,27002 and 27003 instead of 26001/26002/26003

Lets try to make the lab more clear:

  1. Replica Set: “m103-csrs” consists of 3 nodes: 26001, 26002 and 26003.
  2. You need to re-configure replica set: m103-repl to make it a shard. If you have not setup already.
    Replica Set: m103-repl
    Nodes: 27001, 27002 and 27003

Now, what you can do is, bring down nodes one by one. And then remove from the replica set “m103-csrs”. Then add 26001, 26002 and 26003 after they are up and running. And ofcourse after initiating replica set.

Let me know if I can be of more help!

Kanika

1 Like

i must have had the incorrect values in my conf files. Thanks much for looking at this.

I’m stumped at ports 27001 and 27003 Kanik. Can you explain this business of deleting the local data base so that I can run a fresh instance of the node1 and node3 csrs config files for these ports. They invariably fail in a replica set, becoming unreachable. Even when I add other nodes to permit a majority of healthy nodes, I can’t add the cluster as a shard in my mongos.

And my mongos can’t show dbs for the same reason apparently that it can’t add my second shard cluster, which was to be my first; that reason being ""Could not find host matching read preference { mode: “primaryPreferred” I can, however, use admin and use configure and use local. It’s just that I can know before time if those dbs are there. Is it from db.local that I delete local databases for those nodes causing problems?