Automatic Client Field Level Encryption with AWS Lambda

We are trying to integrate Automatic client side field level encryption with AWS Lambda. I wrote a blog post about a POC, overcoming some of the obstacles around the mongocryptd process here: https://medium.com/@floriannorbertbischoff/using-mongodb-client-field-level-encryption-with-aws-lambda-5968a5a1a98a

However, convert this POC into production ready code proves to be challenging. My lambda randomly exists prematurely, but when run again it sometimes works, sometimes not. It seems like there is some sort of race condition going on. Since it works fine without field level encryption, my best guess is, that the mongocryptd process is not fully up and running.

Has anyone had success in integrating automatic FLE with lambda?
How can I configure the mongocryptd process in a way that it offers some basic debug logs, so I can see the point of failure?
I know that according to the mongo docs one should set context.callbackWaitsForEmptyEventLoop = false; and cache connections (in order to not flood the cluster with unused connections). But I wonder: Does this create problems with hanging mongocryptd processed?

Generally it seems the mongocryptd approach favors a “tranditional” approach of a long running process. What would be the advice for using it in ephemeral function containers (like lambda, Google Cloud functions, etc.)

Sometime the error is MongoServerSelectionError: connect ECONNREFUSED 127.0.0.1:27020 at Timeout._onTimeout, but most of the error are "MongoError","errorMessage":"Current topology does not support sessions" which is strange as I am on an Atlas cluster 4.4 with MongoDB Driver 3.6

There is a bug in the node bindings of the libmongocrypt page: https://github.com/mongodb/libmongocrypt/blob/8530af06643daa28259e5830ce1dff22b6be326a/bindings/node/lib/mongocryptdManager.js#L44-L66

The spawning function does not wait until the process is up, it merely waits until the spawning of the process has been handed off by Node to the OS land. At least to my understanding of how the node event queue works. This can create a race condition where the driver tries to connect to the cluster while the mongocryptd process is not ready to accept connections yet. In my case it sometimes did, sometimes not.

I detailed a fix in the article above (at the very bottom), but it is not pretty and I would love to hear from the devs.

Hi @Florian_Bischoff and welcome to the forums!

Thanks for sharing the knowledge and also raising an issue relating to libmongocrypt. There is now a spec discussion changes to accommodate this (default timeout value).

Please see NODE-2794 for more information. Feel free to upvote/watch the issue ticket to receive notifications on the ticket.

Regards,
Wan

Hi Wan, thank you. The link you provided leads to a protected resource.:

You can’t view this issue

It may have been deleted or you don’t have permission to view it.

Hi @Florian_Bischoff,

Apologies for that, as the ticket is linked to your reported ticket NODE-2794 you should be able to receive notifications on the progress. The ticket is linked as a dependency.

Regards,
Wan

This topic was automatically closed 5 days after the last reply. New replies are no longer allowed.