Asya - Thank you for responding. My database is hosted on MongoDB Atlas with version 4.2.13 Enterprise. Clearly I should upgrade to 4.4. The stats you referenced will be very helpful.
The most important collection is a set of roughly 1.4MM timed events, growing about 25K events per day. I’m implementing a dashboard that shows various views of the events, including time-based distributions.
Users can configure the duration of the time series and how many divisions per day. The most fine grained option is 40 days of data, divided into 1-hour divisions, or 960 ‘buckets’ total.
The $group stage is problematic. I use $addFields to give each event a ‘bucket number’, then group by bucket number. So, I am grouping the events into 960 separate buckets. I’m very interested to know the time spent on this stage, and looking forward to trying 4.4 to discover this.
I would be happy to share one or more of my pipelines, if you would review them. LMK and I’ll send a couple of examples over. Best - Peter.