Why does this tool exist?

I am new to MongoDB and trying to understand the tools and offerings. One of the first things I try to understand is what is the purpose or usage of a particular tool. It seems we have the ability to query MongoDB using the collection find() method. It allows filtering and projections.

So my question: why does the Aggregation Framework exist? Is this used internally by the find() method? Is the purpose to derive summation data such as percentages, or counts, etc? Can we not use find() to do the same?

I believe that’s why you took this course to know why this tool exists.

While it’s true that you can do with find() method what the current lesson on Aggregation is teaching. But wait, there’s more. Let’s follow the lessons to learn more. You can also view the manual reference: https://docs.mongodb.com/manual/reference/operator/aggregation/ to learn ahead what functions/methods can be found on Aggregation.

As a side note, I’m also a beginner and I’m learning that there’s this $project function that let’s you do data manipulation – I’m not sure if it can be used on a find() method.

Hope this helps,

Ben

1 Like

Hi,

As @Benjamin_93799 pointed out, the aggregation framework provides the ability to create complex data processing pipelines, where you can filter, group documents, compute aggregated fields, re-shape documents, etc., a lot of operations which cannot be done using find(). Before the aggregation framework was released, all these operations were programmed in the application layer.

José Carlos

2 Likes

Well, this course describes “what” the tool is, it does not describe “why”. I must base my understanding of “why” on my very limited Mongo experience. I think the answer by @jcarlosgarcia is good in that he describes “why”.

Simply put, there was a need. Processing on the client side could be pushed to the server side, specifically for operations which cannot be performed using “find()”. With the answer by @jcarlosgarcia I can now start to form an opinion about the “find()” tool, and how it is not full featured. Before his answer I did not know there are limitations with find() which can be overcome using aggregation().

Learning the limits of tools is difficult because people don’t seem to want to discuss the weaknesses. These limits will be learned through experience and pain. So I ask “why” in an academic setting so I can understand the limits before I encounter a problem in a profession environment. I hoped these tutorials would explain “why” but they seem to miss this point.

My approach to learning mongoDB, before I took this course, was by tutorial on another framework I’m learning, Meteor.

My background is based on Linux Apache MySQL PHP (LAMP). I’m used to DBRMS way of setting up my data. So I tend to relate MongoDB to MySQL when designing database. Naturally this takes some mind bending on my part as I’m ‘hardwired’ to MySQL way. MongoDB is nothing like MySQL in many ways. I also understand that MongoDB has address issues that is superior to MySQL in today’s technology.

I’m used to MySQL JOINS to relate information across tables. MongoDB find() cannot do this. That’s when I stumble on MongoDB Aggregate() – by looking into a MongoDB equivalent to MySQL JOINs. As I’m currently learning now, Aggregate is more than a MySQL JOIN. And also, I need to re-evaluate my database designs if I’m going to use MongoDB – very different. It would be helpful if there’s some easy to ready information on how to design database MongoDB style. :smiley:

I think the why for me comes after the fact. I may not learn MongoDB Aggregation if I don’t need it in my App devs.

Hi,

You’ll find this very helpful:

https://docs.mongodb.com/manual/data-modeling/

Hope it helps.

José Carlos

I come from a classic database background and it does require a slight adjustment to your thinking, more a 90 degree shift.

So I don’t look at the find/delete/update etc as lacking features. Like most things, it is about using the right tool for the job. Some things are easier to do in the client language, where you can change the data. I think when I first looked at MongoDB 3+ years ago,

I think I remember reading that the aggregation pipeline was developed separately, so it didn’t sit in the core of MongoDB, like find, update etc. Which has the benefit in that it could be maintained independently, without impacting other facets of the code base, of course that comes with limitations of course, but it does mean it can be fine tuned for server side performance.

Aggregate is about keeping the heavier data manipulation on the server, and reducing network traffic. I tended to look at it as a reporting tool initially, however that simplifies what you can achieve with it, and now I would use it for a lot more complex uses.

I think the strength of aggregate is the ability to pipeline data, from one operation into another, into another, reusing pipeline operators, grouping, joining, splitting, rejoining etc.

I think you will appreciate it more as as you move through the course.

For MongoDB, it is about how your data is read. What data best sits together, not about normalising the crap out of it :slight_smile:

NOTE - have a look at the aggregate feature in compass (next to the schema view), which provides a gui look at aggregation pipeline. I don’t know if it is as fully formed as the shell command, but I do anticipate I will use it to re-review my data.

I will also be looking to see if the Cosmos DB extension for Visual Studio code will help with intellisense for MongoDB methods.

Thanks for the response. Your comments make sense and will solidify my opinion of the tool set. Like you said, the tool becomes more appreciated with use. There is no substitute for experience…

While looking at the material from the last mongo world conference in 2018, I noticed this, which may be of use. Obviously take things with a pince of salt, but it should be useful.

It it is a shame it doesn’t have the video of the actual presentation.