Mongos throws error around 7000 connection

Sam_Lin · December 23, 2020, 6:46am

[mongodb-sharded] connection drops and performance issue

opened 09:36PM - 11 Dec 20 UTC

closed 01:16PM - 20 Oct 22 UTC

solved

**Which chart**: mongodb-sharded **Describe the bug** I don't know if t…his is the bug for this chart but If I create a lot of connections like 7000 ish Mongos starts to fail. ``` ":{"$date":"2020-12-11T21:02:07.433+00:00"},"s":"W", "c":"EXECUTOR", "id":22993, "ctx":"conn20145","msg":"Terminating session due to error","attr":{"error":{"code":1,"codeName":"InternalError","errmsg":"failed to create service entry worker thread"}}}{"t":{"$date":"2020-12-11T21:02:07.433+00:00"},"s":"I", "c":"NETWORK", "id":22944, "ctx":"listener","msg":"Connection ended","attr":{"remote":"10.42.80.0:47203","connectionId":20145,"connectionCount":7005}}{"t":{"$date":"2020-12-11T21:02:07.434+00:00"},"s":"I", "c":"NETWORK", "id":22943, "ctx":"listener","msg":"Connection accepted","attr":{"remote":"10.42.80.0:10301","connectionId":20146,"connectionCount":7006}}{"t":{"$date":"2020-12-11T21:02:07.434+00:00"},"s":"I", "c":"-", "id":22948, "ctx":"listener","msg":"pthread_create failed","attr":{"error":"Resource temporarily unavailable"}} ``` I can't figure out why. I added this to initContainers of mongos but it doesn't make any difference ``` - name: "adjust-max-map-count" image: "busybox:latest" securityContext: privileged: true command: ["sysctl", "-w", "vm.max_map_count=262144"] ``` **To Reproduce** Start a mongo cluster with 4 shards/3 replica set use jmeter to connect to mongodb like below and set the thread count like 70 and run 100 times (so it creates 7000 connections) https://www.blazemeter.com/blog/mongodb-performance-testing-with-jmeter

I used bitnami’s chart to create 3 mongos / 3 replica set.

However, when I run it, it always fails around 7000 connection.

(Here is the unit test code)

Any idea? thx!