MongoDB authentication failed for Replica Set

Hi Team,

We are getting below errors sometimes when we try to connect from PHP client. This goes away when we try to login as single node rolling back from distributed login method to single node login. And by trying to login as a different user.

This is really concerning as single node login always works for us but login as Replica Set with read preference as slave fails in between. It is very scary to move with this development forward.

Error at application health check :

{​​​​​​​​"status":“Down”,“message”:“Something went wrong!!!”,“error”:“No suitable servers found (serverSelectionTryOnce set): [Failed to resolve ‘mongodb-1’] [Failed to resolve ‘mongodb-2’] [Failed to resolve ‘mongodb-3’]”}​​​​​​​​

Setup :

PHP client 7.3
MongoDB 4.2.10 (Replica Set cluster 3 nodes(no arbiter))
(mongodb-1,mongodb-2,mongodb-3)

Our ground check :

Firewall not enabled
Selinux disabled

MongoDB replication stat (replicating working normally) :

ReplicaSet:PRIMARY> rs.printSecondaryReplicationInfo()
source: mongodb-2:27017
syncedTo: Fri Oct 16 2020 05:33:11 GMT+0000 (UTC)
0 secs (0 hrs) behind the primary 
source: mongodb-3:27017
syncedTo: Fri Oct 16 2020 05:33:11 GMT+0000 (UTC)
0 secs (0 hrs) behind the primary 

PHP config :

return [
 'default' => 'mongodb',
 'connections' => [
 'mongodb' => [
 'driver' => 'mongodb',
 'host' => ['Comma separated Ips'],
 'port' => env('DB_PORT'),
 'database' => env('DB_DATABASE'),
 'username' => env('DB_USERNAME'),
 'password' => env('DB_PASSWORD'),
 'options' => []
 
 ],
 ],
 'migrations' => 'migrations',
 ];

MongoDB Logs :

2020-10-16T05:12:26.877+0000 I  ACCESS   [conn134] Successfully authenticated as principal mongo_styli_coupon on styli_coupons from client 10.60.5.198:42038
2020-10-16T05:12:57.378+0000 I  NETWORK  [listener] connection accepted from 127.0.0.1:58282 #135 (12 connections now open)
2020-10-16T05:12:57.378+0000 I  NETWORK  [conn135] received client metadata from 127.0.0.1:58282 conn135: { application: { name: "MongoDB Shell" }, driver: { name: "MongoDB Internal Client", version: "4.2.10" }, os: { type: "Linux", name: "CentOS Linux release 7.8.2003 (Core)", architecture: "x86_64", version: "Kernel 3.10.0-1127.19.1.el7.x86_64" } }

User details :

ReplicaSet:PRIMARY> db.getUser("mongo_styli_coupon")
{
	"_id" : "styli_coupons.mongo_styli_coupon",
	"userId" : UUID("f9cc744e-1516-4106-9fe0-c2a2426b9f9b"),
	"user" : "mongo_styli_coupon",
	"db" : "styli_coupons",
	"roles" : [
		{
			"role" : "read",
			"db" : "admin"
		},
		{
			"role" : "readWrite",
			"db" : "styli_coupons"
		}
	],
	"mechanisms" : [
		"SCRAM-SHA-1",
		"SCRAM-SHA-256"
	]
}

Hi,

the error you’re receiving is what we call a server selection error. When you run any operation, the driver selects a suitable server to run the command on, based on the type of command and the read preference you’ve stated. In your case, the error detail (“Failed to resolve ‘mongodb-1’”) indicates that this error originated in the base implementation of libmongoc, see here: https://github.com/mongodb/mongo-c-driver/blob/282f110565239a0d5d91f1e5e2f91ba4bee57dc7/src/libmongoc/src/mongoc/mongoc-client.c#L655..L659.

Looking at the replication stats that you posted, I can see that the host names for the two secondaries are mongodb-2 and mongodb-3, which is consistent with the error message. It looks like these host names don’t resolve to any hosts on your application server, leading to the connection error seen above as the server is not able to connect to any of the hosts in the replica set.

Please either reconfigure your replica set to use IP addresses, or ensure that the host names used in the replica set config resolve on all machines that are supposed to connect to any server from the replica set.

1 Like

Thanks Andreas,

I will try these changes and confirm.

Regards,
Pritiranjan

Hi Andreas,

We have made those changes and it has fixed the issue. Thanks a lot.

Regards,
Pritiranjan