Batch Processing

I know this question is not course related. Hopefully someone can help me on this. Thanks in advance.

I have a students collection which has thousand of documents. It looks like this:
{
name: “James bond”,
class: “A1”,
math: 95,
english: 90,
social: 80
}

I’d like to modify all the documents which would look like:
{
name: “James bond”,
class: “A1”,
exam: {
math: 95,
english: 90,
social: 80
}
}

Also, If I wonder to get the total score of each course by class in other collection, like:
{
class: “A1”,
math: 2865,
english: 2973,
social: 2643
}

I know there is aggregation for doing this. But I prefer to use batch processing.

Thanks a lot.
Richard

The aggregation is batch processing performed by the server. What kind of batch processing do you have in mind? The only alternative I see is to write an application that:

1.read documents one by one
2.apply the desired transformation to each documents
3.keep a running total for the classes
4.write back all documents

With aggregation, no documents has to be transmitted in and out of the server, except for the total. You would be writing a program. May be we can call that batch processing because there will be no UI. In pseudo code it would be something like:

totals = { "A1" : { math : 0, english:0, ...} , "A2" : {...} .... "An" : { ...} }
cursor = db.students.find()
while cursor has next
{
  document = cursor.next
  modified-document = { name : document.get("name") ,
    class : document.get("class" )
   }
  modified-document.set( "exam" :
  {
     math : document.get( "math" ) ,
     ...
    social : document.get( "social" )
  } )
  db.modified-students.insert( new-document )
  class-totals = totals.get( document.get( "class" ) )
  class-totals.set( "math" : class-totals.get("math") + document.get( "math" ) 
}
insert all documents from totals into other-collection

Thanks for your response.

Yes, you are right. I do want to program from scratch for processing documents one by one.

The four steps solution is absolute what I want.

However, the pseudo coding not work for me.
For example,
cursor = db.students.find()
will provide a bunch (batchsize) of student document. I have no idea how to process it one by one.

Richard

At least 2 ways for JavaScript:

  1. https://docs.mongodb.com/manual/tutorial/iterate-a-cursor/
  2. https://docs.mongodb.com/manual/reference/method/cursor.forEach/