Hello, I was trying to test out client field level encryption following this example field-level-encryption-sandbox/golang_fle_install.sh at master · mongodb-labs/field-level-encryption-sandbox · GitHub. I didn’t make any modification as I wanted to see if it will work on my computer. Well, when I run it I get this error:
Connect error for client with automatic encryption: exec: "mongocryptd": executable file not found in $PATH
I installed the required libmongocrypt following these instructions for Ubuntu GitHub - mongodb/libmongocrypt: Required C library for Client Side and Queryable Encryption in MongoDB
I am missing something? my $PATH is:
/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/usr/games:/usr/local/games:/snap/bin:/usr/local/go/bin:/usr/local/go/bin:/usr/local/lib/
Hi,
Automatic encryption requires both libmongocrypt and mongocryptd. libmongocrypt is a C library that the driver uses to do encryption, decryption, and key caching. mongocryptd is a server process that is automatically spawned by the driver when performing auto-encryption. When encrypting a command, the driver first sends the command to mongocryptd, which analyzes it and marks the fields that need to be encrypted. The marked version of the command is then sent to libmongocrypt to do the actual encryption.
My guess is that you’re running the community version of the server. As noted in the mongo package docs (mongo package - go.mongodb.org/mongo-driver/mongo - Go Packages), automatic encryption is only available for the enterprise version of MongoDB. This is because the mongocryptd binary is only available on enterprise builds.
If you are not able to use an enterprise build, you can disable auto encryption and do explicit encryption instead. The data will still be automatically decrypted, as the decryption process does not require use of mongocryptd. See mongo-go-driver/client_side_encryption_examples_test.go at master · mongodb/mongo-go-driver · GitHub and mongo-go-driver/client_side_encryption_examples_test.go at master · mongodb/mongo-go-driver · GitHub for examples of explicit encryption.
– Divjot
1 Like
Thanks @Divjot_Arora,
The 2 links you shared were useful.
Best!
@Divjot_Arora,
I have some follow up questions
Why is that (from the example on GH) it requires to first compile the code with a cse tag? It also creates so many collections, about 3 in total, can’t it just that be saved into a single collection?
Unrelated to the above, what is the best approach to have a custom TTL for Gridfs documents? any guidance on this?
The cse build tag stands for “client-side encryption”. We require this build tag because this feature requires linking against an external C library. The build tag allows users who don’t need this feature to compile and use the driver without worrying about installing libmongocrypt. If you’re curious, this is done by keep two separate copies of the source code, each conditionally compiled based on the build tag. One copy has actual logic to interact with libmongocrypt and implement the feature and the other contains function stubs that panic if called. You can see this by running the script without specifying the build tag.
As for the collections created, I believe both examples create two collections:
-
encryption.testKeyVault: This is the key vault collection and is used to store the data key created by ClientEncryption.CreateDataKey. These data keys are used to encrypt/decrypt fields. Note that the key material for these keys is also encrypted using the master key. In the examples I linked above, this is localMasterKey. The key vault is necessary collection for client-side encryption.
-
test.coll: This is the collection where the application stores its data. In the example, this data looks like {"encryptedField": <encryptedField>}.
For your question about GridFS, can you provide more details? Do you want to delete the entire GridFS file (i.e. delete all of the chunks for the file and any other information related to it) after a TTL?
If you have any follow-up questions about client-side encryption, it may be helpful to create a new topic with the #go-driver tag for the GridFS question.
– Divjot
1 Like
Thank you,
I will keep this thread for client-side encryption only and,
You said I will need to install an different (go) drive?
For client-side encryption, you do not need to install a different driver. You can install the Go driver using Go modules as documented at GitHub - mongodb/mongo-go-driver: The Official Golang driver for MongoDB and then install libmongocrypt. After everything is installed, you can put either of the examples I linked earlier in your application and everything should work if you compile with -tags cse.
1 Like
Yes, that’s exactly how I tested it when you first shared the links. I thought from your previous response I needed a different go driver.
I see that forcing the -tags cse isn’t a friendly way of using CSE. I’d imagine how it fits in unit testing. We can’t always compile each time you need to run a certain portion of your code because the whole program is now dependent on the tag to be specified. Wouldn’t it be best to simply provide an API interface of the process that will instantiate the CSE?
Unfortunately, it’s not that simple. Files like mongo-go-driver/mongocrypt.go at master · mongodb/mongo-go-driver · GitHub need to be behind some sort of build flag or other build constraint. If they’re not, the #cgo lines will try to link against libmongocrypt, which will immediately fail if the user has not installed libmongocrypt on their system. The idea was to make users who wanted this feature opt-in via the build flag so that users who don’t need it didn’t have to install libmonogcrypt for no reason. Additionally, there are certain Docker images that don’t have cgo support at all, so it’s important that any cgo-based features in the driver are not automatically compiled.
For your point about always requiring compilation, I don’t think this is true. Both go run and go test support adding -tags cse command line option. We take advantage of this in many of our CI tasks, which call go test -tags cse ... rather than compiling anything.
1 Like
thanks, @Divjot_Arora. I appreciate you clarifying this.