, free & fully virtual. June 9th - 10th. Register Now

Insert-heavy load performance questions

Hey there. I’ve performed a couple of simple benchmarks to see how Mongo behaves when inserting documents with j:true and w:1. The environments I used were my local machine with 8 core CPU and SSD, and Azure hosted VM with 4 core CPU and dedicated 256 GB SSD disk. I use the C# driver with shared singleton Mongo Client. The documents I insert are fairly small ones - no more than 2kb. The number of writers I tested are 1-5, 100 locally and distributed tests with up to 600 concurrent writers on Azure.
There are a couple of questions I’d like to clarify:

  1. When I increase the number of writers, Mongo is able to insert mode documents per second, as seen using mongostat. However, the IOPS do not increase linearly with the number of documents. For example, 200 documents correspond to 200 IOPS, 400 to 400, but inserting 500, 1k or 2k documents per second results in 500 IOPS consumed. Furthermore, the IOPS used are only a fraction of total available for disk: Azure disk can handle up to 2k IOPS and my local one is capable of thousands as well. Is there any internal buffering Mongo performs in order to write the entire batch of documents in one operation? I would expect the number of documents to be equal (or less than) to IOPS since j:true implies the document is flushed right away.
  2. Why does the number of inserted documents not grow linearly when increasing the number of writers? There’s plenty of CPU, RAM and IOPS left and the only locks I observed are IX ones for the db and collection.
  3. The most curious one: when I start a second mongod on the same machine and use basic hash to rotate between writing to them, IOPS increases and the number of written documents increases as well. When I use one mongod and multiple dbs/collections, nothing changes. Basically, somehow two mongod processes are able to provide better performance with the same resources dedicated. Is there anything which would cause this? I’d prefer not to start a new mongod for each 10 writers.

Further notes: the local file system is NTFS and it’s a Win 10 machine, the Azure one is the latest mongo docker image with the XFS filesystem on the dedicated Az disk.