readConcern:majority warning shown for mmapV1 engine

Ops and Admin Replication
,
1

Hi Team,

I have a replica-set running with mmapv1 storage engine and mongo Version 3.6.17. Its a 4Member + 1 arbiter architecture. Please see the ps output from all the VMs for this replica-set. You can see the command args are same.

DEP1SM03
root 10080 1 31 Mar16 ? 05:32:14 /usr/bin/mongod --ipv6 --nojournal --noprealloc --smallfiles --slowms 500 --storageEngine mmapv1 --bind_ip_all --port 27727 --dbpath=/var/data/sessions.1/set02b --replSet set02b --fork --pidfilepath /var/run/sessionmgr-27727.pid --oplogSize 5120 --logpath /var/log/mongodb-27727.log --logappend --quiet

DEP1SM04
root 17705 1 26 Mar16 ? 04:46:38 /usr/bin/mongod --ipv6 --nojournal --noprealloc --smallfiles --slowms 500 --storageEngine mmapv1 --bind_ip_all --port 27727 --dbpath=/var/data/sessions.1/set02b --replSet set02b --fork --pidfilepath /var/run/sessionmgr-27727.pid --oplogSize 5120 --logpath /var/log/mongodb-27727.log --logappend --quiet

DAP1SM03
root 10053 1 11 Mar16 ? 01:58:58 /usr/bin/mongod --ipv6 --nojournal --noprealloc --smallfiles --slowms 500 --storageEngine mmapv1 --bind_ip_all --port 27727 --dbpath=/var/data/sessions.1/set02b --replSet set02b --fork --pidfilepath /var/run/sessionmgr-27727.pid --oplogSize 5120 --logpath /var/log/mongodb-27727.log --logappend --quiet

DAP1SM04
root 10053 1 1 Mar16 ? 00:18:19 /usr/bin/mongod --ipv6 --nojournal --noprealloc --smallfiles --slowms 500 --storageEngine mmapv1 --bind_ip_all --port 27727 --dbpath=/var/data/sessions.1/set02b --replSet set02b --fork --pidfilepath /var/run/sessionmgr-27727.pid --oplogSize 5120 --logpath /var/log/mongodb-27727.log --logappend --quiet

But when i connect to mongo replicas individually only for one Secondary the startup warning shows a warning that readConcern:majority is enabled. So im really confused if the readConcern is really enabled in this replica-set or not.

The db.serverStatus().storageEngine shows supportsCommittedReads is false.

{
        "name" : "mmapv1",
        "supportsCommittedReads" : false,
        "readOnly" : false,
        "persistent" : true
}

[root@DAP1SM04 ~]# mongo DEP1SM03:27727
MongoDB shell version v3.6.17
connecting to: mongodb://DEP1SM03:27727/test?gssapiServiceName=mongodb
Implicit session: session { "id" : UUID("fedcd09f-8a25-45d2-8d0c-c5fa779c9588") }
MongoDB server version: 3.6.17
Server has startup warnings:
2021-03-16T09:50:49.459+0000 I CONTROL  [initandlisten]
2021-03-16T09:50:49.459+0000 I CONTROL  [initandlisten] ** WARNING: Access control is not enabled for the database.
2021-03-16T09:50:49.459+0000 I CONTROL  [initandlisten] **          Read and write access to data and configuration is unrestricted.
2021-03-16T09:50:49.459+0000 I CONTROL  [initandlisten] ** WARNING: You are running this process as the root user, which is not recommended.
2021-03-16T09:50:49.459+0000 I CONTROL  [initandlisten]
2021-03-16T09:50:50.163+0000 I REPL     [replexec-0]
2021-03-16T09:50:50.163+0000 I REPL     [replexec-0] ** WARNING: This replica set was configured with protocol version 0.
2021-03-16T09:50:50.163+0000 I REPL     [replexec-0] **          This protocol version is deprecated and subject to be removed
2021-03-16T09:50:50.163+0000 I REPL     [replexec-0] **          in a future version.
set02b:PRIMARY> exit
bye

[root@DAP1SM04 ~]# mongo DEP1SM04:27727
MongoDB shell version v3.6.17
connecting to: mongodb://DEP1SM04:27727/test?gssapiServiceName=mongodb
Implicit session: session { "id" : UUID("c3392728-2a49-419c-9c6f-7d052b8e63ba") }
MongoDB server version: 3.6.17
Server has startup warnings:
2021-03-16T09:23:51.388+0000 I CONTROL  [initandlisten]
2021-03-16T09:23:51.388+0000 I CONTROL  [initandlisten] ** WARNING: Access control is not enabled for the database.
2021-03-16T09:23:51.388+0000 I CONTROL  [initandlisten] **          Read and write access to data and configuration is unrestricted.
2021-03-16T09:23:51.388+0000 I CONTROL  [initandlisten] ** WARNING: You are running this process as the root user, which is not recommended.
2021-03-16T09:23:51.388+0000 I CONTROL  [initandlisten]
2021-03-16T09:23:51.547+0000 I REPL     [replexec-0]
2021-03-16T09:23:51.547+0000 I REPL     [replexec-0] ** WARNING: This replica set uses arbiters, but readConcern:majority is enabled
2021-03-16T09:23:51.547+0000 I REPL     [replexec-0] **          for this node. This is not a recommended configuration. Please see
2021-03-16T09:23:51.547+0000 I REPL     [replexec-0] **          https://dochub.mongodb.org/core/psa-disable-rc-majority-3.6
2021-03-16T09:23:51.547+0000 I REPL     [replexec-0]
2021-03-16T09:23:51.547+0000 I REPL     [replexec-0]
2021-03-16T09:23:51.547+0000 I REPL     [replexec-0] ** WARNING: This replica set was configured with protocol version 0.
2021-03-16T09:23:51.547+0000 I REPL     [replexec-0] **          This protocol version is deprecated and subject to be removed
2021-03-16T09:23:51.547+0000 I REPL     [replexec-0] **          in a future version.
set02b:SECONDARY> exit
bye

[root@DAP1SM04 ~]# mongo DAP1SM03:27727
MongoDB shell version v3.6.17
connecting to: mongodb://DAP1SM03:27727/test?gssapiServiceName=mongodb
Implicit session: session { "id" : UUID("54e9cc35-0043-4272-803e-dd3f90f89f5c") }
MongoDB server version: 3.6.17
Server has startup warnings:
2021-03-16T09:50:48.430+0000 I CONTROL  [initandlisten]
2021-03-16T09:50:48.430+0000 I CONTROL  [initandlisten] ** WARNING: Access control is not enabled for the database.
2021-03-16T09:50:48.430+0000 I CONTROL  [initandlisten] **          Read and write access to data and configuration is unrestricted.
2021-03-16T09:50:48.430+0000 I CONTROL  [initandlisten] ** WARNING: You are running this process as the root user, which is not recommended.
2021-03-16T09:50:48.430+0000 I CONTROL  [initandlisten]
2021-03-16T09:50:48.630+0000 I REPL     [replexec-0]
2021-03-16T09:50:48.630+0000 I REPL     [replexec-0] ** WARNING: This replica set was configured with protocol version 0.
2021-03-16T09:50:48.630+0000 I REPL     [replexec-0] **          This protocol version is deprecated and subject to be removed
2021-03-16T09:50:48.630+0000 I REPL     [replexec-0] **          in a future version.
set02b:SECONDARY> exit
bye

[root@DAP1SM04 ~]# mongo DAP1SM04:27727
MongoDB shell version v3.6.17
connecting to: mongodb://DAP1SM04:27727/test?gssapiServiceName=mongodb
Implicit session: session { "id" : UUID("2e3d35e3-d001-49e5-86e8-5c69da316521") }
MongoDB server version: 3.6.17
Server has startup warnings:
2021-03-16T09:24:48.417+0000 I CONTROL  [initandlisten]
2021-03-16T09:24:48.417+0000 I CONTROL  [initandlisten] ** WARNING: Access control is not enabled for the database.
2021-03-16T09:24:48.417+0000 I CONTROL  [initandlisten] **          Read and write access to data and configuration is unrestricted.
2021-03-16T09:24:48.417+0000 I CONTROL  [initandlisten] ** WARNING: You are running this process as the root user, which is not recommended.
2021-03-16T09:24:48.417+0000 I CONTROL  [initandlisten]
2021-03-16T09:24:48.886+0000 I REPL     [replexec-0]
2021-03-16T09:24:48.886+0000 I REPL     [replexec-0] ** WARNING: This replica set was configured with protocol version 0.
2021-03-16T09:24:48.886+0000 I REPL     [replexec-0] **          This protocol version is deprecated and subject to be removed
2021-03-16T09:24:48.886+0000 I REPL     [replexec-0] **          in a future version.
set02b:SECONDARY> DAP1SM04:27727

Is it a bug? or the readConcern:majority is really enabled?

2

readConcern majority is not supported in MMAPv1 engine. Please have a look on below link

3

Ok Then why the mmapv1 replica-set showing this warning.?

4

@ROHIT_KHURANA, Do you have a reason why mmap replica-set showing this warning then?

6

@venkataraman_r

Are you geeting this warning for only instance running on DEP1SM04?
Please share output of below command
rs.conf();

Thanks
Braj Mohan

7

this warning occurrs on arbiter node.

8

@venkataraman_r
Please have a look on below jira which indicates mongo put this warning wherever we use arbiter node

We want to add a startup warning that alerts users if enableReadConcernMajority is set to true on a node that is part of a replica set that contains an arbiter

https://jira.mongodb.org/browse/SERVER-37557

9

Hi @venkataraman_r welcome to the community!

I believe @ROHIT_KHURANA is correct in this case. It is a startup warning, designed to inform you of any suboptimal settings for production instances.

Having said that, I would like to point out some things for your consideration:

  1. The MMAPv1 storage engine was deprecated and subsequently removed in MongoDB 4.2, so I would encourage you to not rely on any MMAPv1 specific tuning or operation characteristics (e.g. using nojournal, noprealloc, smallfiles, etc.).
  2. MongoDB 3.6 series is out of support in the upcoming April 2021, so I would encourage you to upgrade to a newer supported versions.
  3. Arbiters are designed to break election deadlock and allow you to have a primary should the secondary went offline in a 3-node replica set of Primary-Secondary-Arbiter setup. However, note that using arbiters does incur the cost of potential rollbacks and issues with majority read concern which underpins modern MongoDB features such as change streams and causal consistency. Please note that this doesn’t mean that Arbiters are not supported. It’s just that their use may be detrimental to performance in newer MongoDB versions.
  4. Please be very, very careful in using bind_ip_all setting without using auth. If these servers are open to the internet, anyone can connect to them without password and perform admin tasks (do any query, drop all databases, etc.). Even if they are not connected to the internet, anyone with access to the servers can do all those things. Please see the security checklist for more information.

Best regards,
Kevin

2 Likes
10

The output i shared is from PRIMARY and SECONDARY. As per your statement, im using arbiter in the RS so it should show for all the SECONDARY. But it shows only for one SEC member.

By point is if the warning is correct, then it should show for all 4 members of a RS. but it shown only for one SEC.

@kevinadi,

Thanks for your reply. Yes we undetstood mongo 3.6 is EOL by next month. I’m in the process of migrating to the latest one. But we are looking this discepency in the existing deployment with 3.6. So raised this question.

11

Hi @venkataraman_r

The message about readConcern majority should only show up in a PSA (Primary-Secondary-Arbiter) setup as per SERVER-42573. This was implemented in MongoDB 3.6.1, thus you are correct that it should not show up in a 5-node PSSSA configuration like yours.

There may be some other underlying issues, like whether the replica set is seeing itself properly (from the output of rs.conf() and rs.status(), for example), whether all mongod binaries involved are verified to be 3.6.17 and not an accidental mix of versions, or other issues. However if everything is verifiably correct and the message still shows up in your deployment, especially only in one node as you observed, it may be a new issue.

Having said that, there are some points against this:

  1. Since the 3.6 series will be out of support very soon, it’s unlikely that this issue will receive attention. Especially since the issue involves the MMAPv1 engine which was already removed in newer MongoDB versions.
  2. The startup warnings have no bearing on the operational side of the database. They are to warn you about suboptimal settings.

In short, in this specific case I would not worry about what appear to be an erroneous startup warning if it doesn’t affect the operations of the replica set. I would recommend you to instead migrate from MMAPv1 to the WiredTiger storage engine and upgrade to the latest MongoDB version (currently 4.4.4) so you’ll receive the most up to date version with lots of bugfixes and improvements. Also, please consider turning on Auth for your deployment.

Best regards,
Kevin

2 Likes
12

setup have one arbiter node with 4 data nodes.