Child process failed, exited with error number 1.0

@Simon_39939 or anyone

I’m receiving the below error when trying to stat mongodb on node 1:

mongod --config node1.conf

`` ERROR: child process failed, exited with error number 1

    To see additional information in this output, start without the "--fork" option.

    vagrant@m103:/var/mongodb/db$

new error message for the same node:

2019-06-26T14:50:13.237+0000 F CONTROL  [main] Failed global initialization: FileNotOpen: Failed to open "/var/mongodb/db/mongod1.log logAppend: true processManagement: fork: true security: authorization: enabled keyFile: /var/mongodb/pki/m103-keyfile replication: replSetName: m103-repl "
vagrant@m103:/var/mongodb/db$

Looks like write permission missing for the directory. Run ls -ld /var/mongodb/db to see current owner and permissions. Use chown or chmod to change owner or permission.

okay so group looks correct and even the permission looked correct to. Still changed it anyway to:

rw-rw-r-- 1 vagrant vagrant 54366 Jun 25 16:49 mongod.log
-rw-rw-r-- 1 vagrant vagrant   334 Jun 26 14:19 node1.conf
drw-rw-rw- 2 vagrant vagrant  4096 Jun 26 13:55 node1_dir
-rw-rw-r-- 1 vagrant vagrant   334 Jun 26 13:43 node2.conf
drwxrwxr-x 2 vagrant vagrant  4096 Jun 26 13:55 node2_dir
-rw-rw-r-- 1 vagrant vagrant   334 Jun 26 13:50 node3.conf
drwxrwxr-x 2 vagrant vagrant  4096 Jun 26 13:55 node3_dir
-rw------- 1 vagrant vagrant 36864 Jun 25 16:38 sizeStorer.wt
-rw------- 1 vagrant vagrant   114 Jun 25 12:34 storage.bson
vagrant@m103**:/var/mongodb/db$ mongod --config  node1.conf**
**2019-06-26T14:58:41.224+0000 F CONTROL  [main] Failed global initialization: FileNotOpen: Failed to open "/var/mongodb/db/mongod1.log logAppend: true processManagement: fork: true security: authorization: enabled keyFile: /var/mongodb/pki/m103-keyfile replication: replSetName: m103-repl "**
**vagrant@m103:/var/mongodb/db$**

I think this bit of the error is pertinent:

FileNotOpen: Failed to open "/var/mongodb/db/mongod1.log

I’m guessing that if you ls now, there won’t be a mongod1.log file? If there isn’t then maybe you don’t have write permission on the db folder, you could cd up a level and ls -ld to check the permissions and ownership of the db folder.

It’s odd though, you must have had write permission on this folder at some point in the past to have created the files and folders which are already there.

I think my file was out of order. Look at this

vagrant@m103:/data/db$ mongo --port 27001

MongoDB shell version v3.6.12

connecting to: mongodb://127.0.0.1:27001/?gssapiServiceName=mongodb

2019-06-26T19:03:57.390+0000 W NETWORK [thread1] Failed to connect to 127.0.0.1:27001, in(checking socket for error after poll), reason: Connection refused

2019-06-26T19:03:57.391+0000 E QUERY [thread1] Error: couldn't connect to server 127.0.0.1:27001, connection attempt failed :

connect@src/mongo/shell/mongo.js:263:13

@(connect):1:6

exception: connect failed

vagrant@m103:/data/db$

What have you changed since your previous post? The message

Failed to connect to 127.0.0.1:27001, in(checking socket for error after poll), reason: Connection refused

… says to me that you’re probably trying to connect the shell to a port where no mongod is listening. Are your mongod instances running? Try a

ps -ef | grep mongod

… to see whether the mongods that you need for this activity are actually running, and if they aren’t then start them. It can be a bit confusing at times, we need one set of mongods to follow along with the lectures and then we need a different set of mongods to do the labs, so make sure you’ve got the right mongods running for what you’re trying to do.

1 Like

WHOOPS ignore the first one its the wrong port this is the correct port but failed:

vagrant@m103:/data/db$ mongo admin --port 27001

MongoDB shell version v3.6.12

connecting to: mongodb://127.0.0.1:27001/admin?gssapiServiceName=mongodb

2019-06-26T19:42:05.465+0000 W NETWORK [thread1] Failed to connect to 127.0.0.1:27001, in(checking socket for error after poll), reason: Connection refused

2019-06-26T19:42:05.466+0000 E QUERY [thread1] Error: couldn't connect to server 127.0.0.1:27001, connection attempt failed :

connect@src/mongo/shell/mongo.js:263:13

@(connect):1:6

exception: connect failed

You briefly posted the output from a ps -ef | grep mongod command, but then retracted it. That’s what we need to see. That tells us what command line (and what config file) was used to start a mongod instance, and from the config file we can tell whether any of them are listening on port 27001 on the hostname you’ve suppplied. That’s how we can tell whether or not you’ve got a mongod (DB server) process running for your mongo (client) process to connect to.

So i think i figured out my issue its really strange. Even though I put into my config file port 27001 mongo would let me connect by just running: mangod --config (Node) for some reason when i put: mangod --config (Node) --port 27001 it finally worked. I wonder why it wasn’t reading my config file…

waiting for connections on port 27001
2019-06-26T19:51:15.073+0000 I NETWORK  [listener] connection accepted from 127.0.0.1:56992 #1 (1 connection now open)
2019-06-26T19:51:15.074+0000 I NETWORK  [conn1] received client metadata from 127.0.0.1:56992 conn1: { application: { name: "MongoDB Shell" }, driver: { name: "MongoDB Internal Client", version: "3.6.12" }, os: { type: "Linux", name: "Ubuntu", architecture: "x86_64", version: "14.04" } }

Who is the owner of the directory and whar are the permissions on the directory. It lloks you ran ls -l rather ls -ld. So we got the content of the directory not the directory. When mongod tries to move mongod.log to mongod1.log it needs write permissiin on the directory.

Hi @James_68753,

If you want to launch mongod process using config file use the following command:

mongod --config <config_filepath>

Then connect to the mongo shell using:

mongo --port 27001

Please let me know, if you face any problem in connecting to node running on 27001 port.

Thanks,
Sonali

@Simon_39939 @Sonali_Mamgain

What’s up guys! Okay this is regarding new issues i’m running into and wanted to see if you guys ran into the same one. Okay so first issue. I’m not sure how i set the secondary read replica’s. I’ve literally tried the different commands like: rs.add(":27002") but nothing. I also went to my second DB but by mistake set that node as a primary on that node as well. I’m having trouble with deleting that as a primary as well. I guess after figuring this part out i’m done with this project!!. Thanks!

My primary db
vagrant@m103:~$ mongo m103-admin --port 27001

MongoDB shell version v3.6.12

connecting to: mongodb://127.0.0.1:27001/m103-admin?gssapiServiceName=mongodb

Implicit session: session { "id" : UUID("373dd607-0cb3-415a-84ad-567662096ecf") }

MongoDB server version: 3.6.12

MongoDB Enterprise m103-repl:PRIMARY> 

secondary node

vagrant@m103:/var/mongodb/db/node1$ mongo --port 27002

MongoDB shell version v3.6.12

connecting to: mongodb://127.0.0.1:27002/?gssapiServiceName=mongodb

Implicit session: session { "id" : UUID("e8c36d20-95b3-4cf8-a4ef-6eca8ccaf9b6") }

MongoDB server version: 3.6.12

MongoDB Enterprise m103-repl:PRIMARY>

agrant@m103:~$ mongo --host “m103-example/192.168.103.100:27011” -u “m103-admin”

MongoDB shell version v3.6.12

Enter password:

connecting to: mongodb://192.168.103.100:27011/?gssapiServiceName=mongodb&replicaSet=m103-example

2019-06-28T16:16:54.643+0000 I NETWORK [thread1] Starting new replica set monitor for m103-example/192.168.103.100:27011

2019-06-28T16:16:54.644+0000 W NETWORK [thread1] Failed to connect to 192.168.103.100:27011, in(checking socket for error after poll), reason: Connection refused

2019-06-28T16:16:54.644+0000 W NETWORK [thread1] Unable to reach primary for set m103-example

2019-06-28T16:16:54.644+0000 I NETWORK [thread1] Cannot reach any nodes for set m103-example. Please check network connectivity and the status of the set. This has happened for 1 checks in a row.

2019-06-28T16:16:54.644+0000 W NETWORK [ReplicaSetMonitor-TaskExecutor-0] Failed to connect to 192.168.103.100:27011, in(checking socket for error after poll), reason: Connection refused

2019-06-28T16:16:54.645+0000 W NETWORK [ReplicaSetMonitor-TaskExecutor-0] Unable to reach primary for set m103-example

2019-06-28T16:16:54.645+0000 I NETWORK [ReplicaSetMonitor-TaskExecutor-0] Cannot reach any nodes for set m103-example. Please check network connectivity and the status of the set. This has happened for 2 checks in a row.

2019-06-28T16:16:55.146+0000 W NETWORK [thread1] Failed to connect to 192.168.103.100:27011, in(checking socket for error after poll), reason: Connection refused

2019-06-28T16:16:55.146+0000 W NETWORK [thread1] Unable to reach primary for set m103-example

2019-06-28T16:16:55.146+0000 I NETWORK [thread1] Cannot reach any nodes for set m103-example. Please check network connectivity and the status of the set. This has happened for 3 checks in a row.

2019-06-28T16:16:55.651+0000 W NETWORK [thread1] Failed to connect to 192.168.103.100:27011, in(checking socket for error after poll), reason: Connection refused

2019-06-28T16:16:55.651+0000 W NETWORK [thread1] Unable to reach primary for set m103-example

2019-06-28T16:16:55.651+0000 I NETWORK [thread1] Cannot reach any nodes for set m103-example. Please check network connectivity and the status of the set. This has happened for 4 checks in a row.

2019-06-28T16:16:56.153+0000 W NETWORK [thread1] Failed to connect to 192.168.103.100:27011, in(checking socket for error after poll), reason: Connection refused

2019-06-28T16:16:56.153+0000 W NETWORK [thread1] Unable to reach primary for set m103-example

2019-06-28T16:16:56.153+0000 I NETWORK [thread1] Cannot reach any nodes for set m103-example. Please check network connectivity and the status of the set. This has happened for 5 checks in a row.

@James_68753

First question… which lab / lecture are you trying to do here?

Second question, why are you trying to connect the mongo shell to ports 27001, 27002 and 27011? So far the ports in the 2700n range have been used for lectures and the ports in the 2701n range have been used for labs, so provided you’ve been using the same ports that are used in the lectures and labs, you shouldn’t be trying to use both ranges of port numbers at the same time.

Third question, how did you get ports 27001 and 27002 to both think that they’re a primary node? Unless I’ve missed something fundamental, they can’t both be part of the same replica set and both be primary nodes, i.e. a replica set can only ever have one primary node at a time. Is one or both of them a standalone node maybe?

Try connecting the mongo shell to each of ports 27001 and 27002 in turn, and running the following command against each of them:

MongoDB Enterprise m103-example:OTHER> rs.isMaster().hosts
[
        "192.168.103.100:27011",
        "192.168.103.100:27012",
        "192.168.103.100:27013"
]

(the only bit you need to type is the rs.isMaster().hosts, but when you share the results, it helps if you include the command prompt that you’re typing into, the command itself and the output, like I have above)

This command will at least tell us what the mongod instances listening on those ports think their replica set (if any) consists of.

Observant people will have noticed that my command prompt ends with :OTHER>, when it normally would end with :PRIMARY>, or occasionally :SECONDARY>. This is because I’m part way through making a right pig’s ear of trying to add my replica set to a sharded cluster, and have somehow convinced all the nodes in the replica set that the replica set configuration isn’t valid or that they’re not part of the replica set. I’m not asking for help with this right now, I’m going to have time over the weekend to work it out for myself, and if I decide that I need help then I’ll start a new thread for it. But it does go to show that we all make mistakes. I didn’t get where I am now by not making mistakes, but by learning from them :slight_smile:

1 Like

Lab - Initiate a Replica Set Locally

ab, but with the following adjustments:

type PRIMARY SECONDARY SECONDARY
config filename mongod-repl-1.conf mongod-repl-2.conf mongod-repl-3.conf
port 27001 27002 27003
dbPath /var/mongodb/db/1 /var/mongodb/db/2 /var/mongodb/db/3
logPath /var/mongodb/db/mongod1.log /var/mongodb/db/mongod2.log /var/mongodb/db/mongod3.log
replSet m103-repl m103-repl m103-repl
keyFile /var/mongodb/pki/m103-keyfile /var/mongodb/pki/m103-keyfile /var/mongodb/pki/m103-keyfile
bindIP localhost,192.168.103.100 localhost,192.168.103.100 localhost,192.168.103.100

OK, you’re setting up your first replica set. And I retract my statement

On reviewing my notes, this isn’t actually the case. Or maybe my notes are inaccurate, I must consider that possibility too :flushed:

So what does rs.isMaster().hosts return when run against each of ports 27001, 27002 and 27003?

lol its okay! okay look below

MY FIRST PRIMARY DB

MongoDB Enterprise m103-repl:PRIMARY> rs.isMaster()
{
	"hosts" : [
		"192.168.103.100:27001"
	],
	"setName" : "m103-repl",
	"setVersion" : 1,
	"ismaster" : true,
	"secondary" : false,
	"primary" : "192.168.103.100:27001",
	"me" : "192.168.103.100:27001",
	"electionId" : ObjectId("7fffffff0000000000000002"),
	"lastWrite" : {
		"opTime" : {
			"ts" : Timestamp(1561748479, 1),
			"t" : NumberLong(2)
		},
		"lastWriteDate" : ISODate("2019-06-28T19:01:19Z"),
		"majorityOpTime" : {
			"ts" : Timestamp(1561748479, 1),
			"t" : NumberLong(2)
		},
		"majorityWriteDate" : ISODate("2019-06-28T19:01:19Z")
	},
	"maxBsonObjectSize" : 16777216,
	"maxMessageSizeBytes" : 48000000,
	"maxWriteBatchSize" : 100000,
	"localTime" : ISODate("2019-06-28T19:01:28.591Z"),
	"logicalSessionTimeoutMinutes" : 30,
	"minWireVersion" : 0,
	"maxWireVersion" : 6,
	"readOnly" : false,
	"ok" : 1,
	"operationTime" : Timestamp(1561748479, 1),
	"$clusterTime" : {
		"clusterTime" : Timestamp(1561748479, 1),
		"signature" : {
			"hash" : BinData(0,"49cSiGezdECzeGIda93HPXiw3Lo="),
			"keyId" : NumberLong("6707582225824612354")

Here is my second. Now THIS SHOULD NOT BE A PRIMARY BUT IT IS. also what interesting is that when i tab up the servers seems to be connected. the same command shows

MongoDB Enterprise m103-repl:PRIMARY> rs.isMaster()
{
“hosts” : [
“192.168.103.100:27002”
],
“setName” : “m103-repl”,
“setVersion” : 1,
“ismaster” : true,
“secondary” : false,
“primary” : “192.168.103.100:27002”,
“me” : “192.168.103.100:27002”,
“electionId” : ObjectId(“7fffffff0000000000000002”),
“lastWrite” : {
“opTime” : {
“ts” : Timestamp(1561748577, 1),
“t” : NumberLong(2)
},
“lastWriteDate” : ISODate(“2019-06-28T19:02:57Z”),
“majorityOpTime” : {
“ts” : Timestamp(1561748577, 1),
“t” : NumberLong(2)
},
“majorityWriteDate” : ISODate(“2019-06-28T19:02:57Z”)
},
“maxBsonObjectSize” : 16777216,
“maxMessageSizeBytes” : 48000000,
“maxWriteBatchSize” : 100000,
“localTime” : ISODate(“2019-06-28T19:03:07.367Z”),
“logicalSessionTimeoutMinutes” : 30,
“minWireVersion” : 0,
“maxWireVersion” : 6,
“readOnly” : false,
“ok” : 1,
“operationTime” : Timestamp(1561748577, 1),
“$clusterTime” : {
“clusterTime” : Timestamp(1561748577, 1),
“signature” : {
“hash” : BinData(0,“6d3gXr32zaY7PCdfFhMueLFxHcY=”),
“keyId” : NumberLong(“6707588032620396545”)
}
}

My third that i have touched yet until now

agrant@m103:/var/mongodb$ mongo m103-admin --port 27003

MongoDB shell version v3.6.12

connecting to: mongodb://127.0.0.1:27003/m103-admin?gssapiServiceName=mongodb

Implicit session: session { "id" : UUID("83fdcf43-9ff1-42dc-97c4-38e42f7e8add") }

MongoDB server version: 3.6.12

MongoDB Enterprise > 

MongoDB Enterprise > rs.isMaster()

{

"ismaster" : false,

"secondary" : false,

"info" : "Does not have a valid replica set config",

"isreplicaset" : true,

"maxBsonObjectSize" : 16777216,

"maxMessageSizeBytes" : 48000000,

"maxWriteBatchSize" : 100000,

"localTime" : ISODate("2019-06-29T02:03:56.104Z"),

"minWireVersion" : 0,

"maxWireVersion" : 6,

"readOnly" : false,

"ok" : 1

Hi James_68753

Please check rs.status()

After you bring up first mongod you have to do rs.initiate()
Then add other two nodes

Please check the steps

Hello,

I’ve done both steps prior and again now and this is what i get:

vagrant@m103:/var/mongodb$ mongo m103-admin --port 27001
MongoDB shell version v3.6.12
connecting to: mongodb://127.0.0.1:27001/m103-admin?gssapiServiceName=mongodb
Implicit session: session { "id" : UUID("1db509dc-3464-42c9-a526-709c31949efb") }
MongoDB server version: 3.6.12
MongoDB Enterprise m103-repl:PRIMARY> rs.status()
{
	"operationTime" : Timestamp(1561775759, 1),
	"ok" : 0,
	"errmsg" : "there are no users authenticated",
	"code" : 13,
	"codeName" : "Unauthorized",
	"$clusterTime" : {
		"clusterTime" : Timestamp(1561775759, 1),
		"signature" : {
			"hash" : BinData(0,"quL5TxoFLtyNqaWRLel7QTVH5Ng="),
			"keyId" : NumberLong("6707582225824612354")
		}
	}
}
MongoDB Enterprise m103-repl:PRIMARY> rs.initiate()
{
	"operationTime" : Timestamp(1561775769, 1),
	"ok" : 0,
	"errmsg" : "there are no users authenticated",
	"code" : 13,
	"codeName" : "Unauthorized",
	"$clusterTime" : {
		"clusterTime" : Timestamp(1561775769, 1),
		"signature" : {
			"hash" : BinData(0,"SmI4Pu5VTQjfk3sxKhN+xTsZwk4="),
			"keyId" : NumberLong("6707582225824612354")