Atlas + Next.js + Now Connection Issues

[Andrew_Lisowski]

I am trying to using mongo in a serverless environment (vercel’s now). But I am constantly deailing with my connection just dying.

What does my code do?

• I have a list of 1000 blogs
• I scan the blog for new posts via RSS feed
• I potentially add the post to my db if some criteria are met

So a call to my /scan-blogs API endpoint will potentially scan and add hundreds of blog posts. All of this was working fine on mLab, but after switching to Atlas I’ve had to really put a limit on concurrency of these scans to dance around the connection limit.

Here is my shared db utility that I use in all of my serverless function

and here are the errors that I keep getting, about 1/3 of the requests fail.

[ado (Ado Kukic)]

Hey Andrew,

Like from my Twitter message, if you can please try this utility instead https://github.com/vercel/next.js/blob/canary/examples/with-mongodb/util/mongodb.js and let me know if it helps. I think that would be a good first step to helping troubleshoot this.

Thanks,
Ado

[Andrew_Lisowski]

and my connections shoot way up to 500

[Andrew_Lisowski]

It didn’t work

Same error

[Andrew_Davidson]

I think what’s happening here is that the Atlas M0 free sandbox allows only 500 concurrent connections and Vercel is spawning >500 concurrent Lambda functions each making a separate connection: Is there a way to reduce the concurrency a bit or re-use Lambda contexts?

[h_b]

same error . Have you find the fix of this issue?

[Andrew_Davidson]

We are still trying to reproduce this but one thing you could try is to close the MongoClient via close() when your process exits. You do not want to do this after each operation since you want to re-use the MongoClient object while the lambda function is warm, but you should call .close() when your function goes cold.

[ado (Ado Kukic)]

Hey Andrew,

In your next.config.js file, is your target set to “serverless” or “server”? I’ve been trying to get to the bottom of this for a little while, and have found that if your target is “server”, when your app is deployed to Vercel, it will create just two lambda functions that will bundle the /api/ routes and pages that use getServerSideProps(). If you have it set to “serverless” mode then each page will create a unique lambda function, thus potentially adding too many connections.

In my testing, with 10 different API’s and multiple pages using getServerSideProps, I never go over 10 connections. The other thing to consider is shutting down and deleting prior builds.

Hope this helps.

[Angela_Shulman]

Andrew_Lisowski - We reached out to you over MongoDB Atlas’ in-app chat app so that we can provide more tailored help with this connection limit issue. Could you kindly log into https://cloud.mongodb.com/ and look for the chat that we opened?

[Angela_Shulman]

h_b - It looks like one of our Atlas support agents already helped you solve the issue you were having connecting with Mongoose. Please open another chat if you still need help!