Hi,
When I enable clusterAuthMode: x509 for replica set authentication, with the proper certs in place per this documentation, the replica members can authenticate with each other just fine. However, when I also enable authorization: enabled, the members are unable to communicate with each other. In the logs I see entries like this:
authenticate db: $external { authenticate: 1, mechanism: "MONGODB-X509", user: "CN=mongod1", $db: "$external" }
2020-09-11T19:15:50.644+0000 I ACCESS [conn10] Failed to authenticate CN=mongod1@$external from client 172.19.62.122:37102 with mechanism MONGODB-X509: UserNotFound: Could not find user "CN=mongod1" for db "$external"
This would suggest that a corresponding user needs to be in place for each member of the replica set. In the documentation, it’s clear that a user is needed for clients to use when connecting to the replica set and I do see log entries indicating that my client is successfully authenticating using the user I’ve setup, but I can’t find anything in the docs showing that such a user is needed for internal/membership authorization.
Is such a user needed for each member of the replica set for internal/membership authorization, or am I missing something else here? If a user for each member is needed, what db should the user be created in, and with what permissions/roles; where are the docs I’ve overlooked that specify this?
Hi @DanielP and welcome in the MongoDB Community !
Are the cluster IP addresses correctly setup in the bind_ip list?
There are definitely no “internal users” for internal membership authorization, etc. Users are just for real users. Not for internal purposes.
Can you please share your config files & command lines and walk us through the steps you have been through so we can maybe spot the issue?
When I comment the two lines in the security section, the replica set members can communicate just fine, so it seems to be an issue relating to membership authentication.
Thank you for confirming there are no users needed for internal membership authentication; that was my initial assumption but I’ve been grinding on this problem for several days now so I didn’t want to rule anything out, particularly since the error messages I’m seeing in the logs seem to indicate it’s in fact expecting a particular user named after the member that’s trying to connect.
When you comment security section members are able to communicate means some issue with certificates
From mongo documentation
“…Certificat Requirements: The Distinguished Name (DN), found in the member certificate’s subject, must specify a non-empty value for at least one of the following attributes: Organization (O), the Organizational Unit (OU) or the Domain Component (DC).”
It looks like mongod2 is refusing to communicate with mongod1 because it can’t find the user CN=mongod1. Based on what @MaBeuLux88 stated earlier about users not being needed for membership authentication, should I assume this error message is a red herring? I have no idea why this isn’t working.
per @Ramachandra_Tummala’s comment, I’m further scrutinizing the certs. I don’t know if this is relevant, but I’m creating the certs from an easyrsa CA with the following X509 extensions:
Anything here that might cause mongod to fail when attempting to use the certificates?
Also, can anyone confirm that error messages like with mechanism MONGODB-X509: UserNotFound: Could not find user "CN=mongod1" for db "$external" can indeed be indicative of problems with the certificates?
Maybe we can work this from the other end; can anyone that is usingclusterAuthMode: x509 with certificates generated with easyrsa3 tell me how they’re doing so, and with what attributes (e.g. which x509-types, etc.)?
@Ramachandra_Tummala thanks for the response; it’s perhaps important to note that the CN value of the certs is the FQDN of the hosts, not the regular hostname (e.g. hostname is mongod1 whereas the CN value is mongod1.my.domain) Is this perhaps the problem? Would it help to generate a cert that has both the FQDN hostname and the regular hostname in the SAN list?
Also, your comment about needing to add a user, that seems to be solely for client authentication, not membership authentication.
I’m having the exact same problem as yours where replica is returning NoUser error. I have tried both certificates with only clientAuth and both clientAuth and serverAuth
I did not get this error.I was trying to help another student in the thread you have referred
Please show screenshot of the exact error you got
Check all your certificates are fine .Values of CN,O fields etc
Refer to mongo documentation for more details
It’s okay, I found the error. It was mentioned in the docs but it was all cluttered within sentences so I overlooked.
The Organization attributes ( O ’s), the Organizational Unit attributes ( OU ’s), and the Domain Components ( DC ’s) must match those from both the net.tls.clusterFile and net.tls.certificateKeyFile certificates for the other cluster members (or the tlsX509ClusterAuthDNOverride value, if set).
I had a different certificateKeyFile generated by letsencrypt when I only enabled TLS for client to server communication.