How are you doing today? I hope that everybody is safe and sound! I have a necessity to interact over a large database, we are talking about 16 MM records. I’m using python with mongoengine to interact over inside my application. I’m using a simple queryset interacting using a for loop.
But it is costing many hours to process all queryset and it is causing a bottleneck in my system
I would like know if mongo (through) pymongo has alternatives to interact more faster where it can be reduced from hours to minutes.
This is the code:
contacts = Contact.objects.filter(__raw__=query).only('id', 'email', 'lead_score').no_cache().batch_size(10000) # query is a simple dictionary where it is place the search criteria Estou iterando pelo for abaixo: for index, contact in enumerate(contacts, 1): print contact.name
Thank you so much for your help guys!