Cannot restart a new node

MongoDB University M103: Basic Cluster Administration
#1 
vagrant@m103:~/sharding$ sudo mongod -f m103-csrs11.conf
about to fork child process, waiting until server is ready for connections.
forked process: 2432
ERROR: child process failed, exited with error number 14
To see additional information in this output, start without the "--fork" option.

In lab Configure a Sharded Cluster cannot restart a new node.
WAIDW?

#2

Please try what is suggested

#3

Hmmm, when I remove the fork option, it doesn’t return any additional information.

03%20PM
Screen Shot 2019-08-01 at 9.02.03 PM.png1022×116 41.1 KB

FYI, this is about halfway through Lab 1, after I change the clusterRole and the wiredTiger storage limit. I am able to start this node with configsvr.

#4

I also notice that I am unable to start the other two nodes after making the same change.

Here is what my first node conf file looks like after the change:

10%20PM
Screen Shot 2019-08-01 at 9.30.10 PM.png614×570 91.3 KB

#5

2019-08-02T10:24:15.528+0000 E REPL [replexec-0] Locally stored replica set configuration is invalid; See http://www.mongodb.org/dochub/core/recover-replica-set-from-invalid-config for information on how to recover from this. Got “BadValue: Nodes being used for config servers must be started with the --configsvr flag” while validating { _id: “m103-csrs”, version: 3, configsvr: true, protocolVersion: 1, members: [ { _id: 0, host: “192.168.103.100:26001”, arbiterOnly: false, buildIndexes: true, hidden: false, priority: 1.0, tags: {}, slaveDelay: 0, votes: 1 }, { _id: 1, host: “192.168.103.100:26002”, arbiterOnly: false, buildIndexes: true, hidden: false, priority: 1.0, tags: {}, slaveDelay: 0, votes: 1 }, { _id: 2, host: “192.168.103.100:26003”, arbiterOnly: false, buildIndexes: true, hidden: false, priority: 1.0, tags: {}, slaveDelay: 0, votes: 1 } ], settings: { chainingAllowed: true, heartbeatIntervalMillis: 2000, heartbeatTimeoutSecs: 10, electionTimeoutMillis: 10000, catchUpTimeoutMillis: -1, catchUpTakeoverDelayMillis: 30000, getLastErrorModes: {}, getLastErrorDefaults: { w: 1, wtimeout: 0 }, replicaSetId: ObjectId(‘5d440dd3f042be194659e37d’) } }
2019-08-02T10:24:15.528+0000 F - [replexec-0] Fatal Assertion 28544 at src/mongo/db/repl/replication_coordinator_impl.cpp 552
2019-08-02T10:24:15.529+0000 F - [replexec-0]

***aborting after fassert() failure

It’s only a part with error, but I can send warnings and info messages too, if necessary

#6

I suppose you write all logs into mongod.log file as you pointed in the message below.

#7

I see the same message in the log as well. My new question is why does it not recognize it as a shardsvr node after making the change to the clusterRole? I started the rolling restart with shutting down the third node first, but now it won’t start again because of this reason. Are there additional steps needed in the directions, because I think I set up my config file correctly using the example from the previous lecture (above).

#8

“Nodes being used for config servers must be started with the --configsvr flag” while validating”
Please check your config files again
clusterRole should be configsvr

#9

That’s right, @Ramachandra_37567, it should have configsvr as the clusterRole when first initiating the replica set. However, the directions for the lab (Configure a Sharded Cluster) and the previous lecture state that I need to change the clusterRole to shardsvr in order to alert the mongos that the set can be used as a sharded cluster. The problem seems to be that mongo does not recognize the node as being part of a sharded cluster after making the change in the conf file and attempting to restart.

1 Like
#10

Firstly all nodes contain configsvr, but then I should change it to shardsvr in order to start a sharded cluster. Looks like mongod thinks I didn’t configure replica set yet.

I also have noticed that in the lecture mongos configured for 26001,26002,26003 ports, though new restarted nodes are running under 27011, 27012, 27013, not mentioning that dbPath, logPath and other stuff are different from previously started nodes. There are so many inaccuracies between video, comments under this video and the lab that I don’t even know what to trust.

#11

Please don’t confuse with replicaset servers and config servers
For sharding you are using already up and running replicaset servers from previous lab
That’s why it says reconfigure(reconfiguring to add as 1st shard)
You have to bring down modify/add clusterrole params and restart the replica servers
For sharding you need 3 replica servers,3 config servers and one mongos
3 replica servers run on ports 27011, 27012, 27013
3 config servers run on ports 26001,26002,26003
mongos runs on 26000
Later on you will be adding 3 more replica servers as second shard for chunks lab
You have to follow the sequence and all your config files should be perfect as per lab requirements(replicasetname,ports,keyfiles etc
Any deviation will result in validation failures

1 Like
#12

Wow, that makes sense, now it works, who could know that three nodes are already running? ¯\ (ツ)/¯

#13

I’ve stopped “useless” servers from prev part :slight_smile:

#14
#15