Auditing in MongoDB

First of all, thanks guys and gals for putting this course together! :clap:

So I completed this course two days ago and wanted to raise a few general queries re Auditing in MongoDB. Here goes:

  1. Collection as a destination for audit logs: Currently, there are 3 destinations for audit logs; file, syslog and console. It would have been beneficial to have the option of outputting to a special audit collection and separate db. Oplogs are stored in a collection, so a similar concept could have been applied to audit logs. That way we can query audit logs (very useful), have a capped audit collection, encrypt, compress, TTL at collection level etc… and do other nice things that come with the WiredTiger and MongoDB alliance.
  2. Collection as a source for configuring auditing params: Considering the nature of MongoDB schemas, it’s a perfect structure (besides YAML) for setting up the configs for audit logs. Abstract the audit config from the DB config file!
  3. Changing filter params on-the-fly: Without doing a rolling upgrade, it’s quite tedious to add/edit/drop filters. This is because the config for audit logging is tied to the DB config file! In some organisations, auditing operations are not periodic/static and can change fairly frequently. If audit logs were configurable in a collection as per point 2, it would have been so much easier to add/edit/drop multiple auditing filters, and have timestamps of when a filter was setup, by whom, on what resource, and for how long.
  4. TTL on filters and Rotate Logs: Imagine a fictitious scenario where I want to audit the following users: @kanikasingla, @Sonali_Mamgain, @Shubham_Ranjan and @Muskan_47318 for one week :slight_smile: , I can’t do that with ease :unamused: . I have to remember to remove the filter a week later and in addition, perform a rolling upgrade each time. There’s no reference of this in the docs for Ops/Cloud Manager. No references to Stitch (Triggers) please!
  5. Cap log size: The ability to set the max size of the log file to stop it from outgrowing set limits and/or overwriting data when a certain quota is reached. As per point 1, if audit logs were stored in a collection, that’s also backed by a compressed DB, this would have been possible using the existing concept of capped collections. And with this, it would have been easier to rotate the log when a quota is hit.
  6. Alerts and Monitoring: Being able to monitor and respond to events related to auditing seems very limited in this regard. Even with the integration with New Relic, I can’t see anywhere in the Ops/Cloud Manager docs that describes a robust way of handling this.

The documentation for Ops Manager and Cloud Manager touches on a few of these pain-points but I don’t see much improvement.

In my humble opinion, auditing should be detached from the DB config file. And a fresh approach on how Audit Logging in MongoDB is structured and handled should be considered; one that really leverages the power and resources of existing MongoDB/WiredTiger capabilities.

Thoughts, comments? Anyone?

Disclaimer: for your sanity, ignore any typos/grammatical errors :wink:

Hi @007_jb,

First of all congratulations on completing this course :clap:. And also I’m glad you found the content helpful :slight_smile:

Thanks for sharing your valuable feedback. We are looking into it and we will get back to you shortly.

Is there anything that I can do to get my name off of the auditing list ? :joy:

Thanks,
Shubham Ranjan
Curriculum Support Engineer

It’s going to be difficult because I have to do a rolling upgrade :rofl:

2 Likes

Any updates from your Dev/Product team @Shubham_Ranjan?

Hi @007_jb,

First of all, I would like to thank you for your valuable contribution to the community :slight_smile:

I have passed on your feedback to the product team. If they need any additional information on this then I would get back to you.

Sometimes we may not get back to you but be assured that we are capturing your feedbacks and passing it on to the relevant teams.

Keep up the good work :slight_smile:

Thanks,
Shubham Ranjan
Curriculum Support Engineer

No problemo @Shubham_Ranjan