, free & fully virtual. Register Now, free & fully virtual. Register Now

Atlas search with pagination slow

We have 600,000 documents in our atlas collection, soon to be over 2M. Once we created the proper indexes, an atlas search with" text" or “autocomplete” returns in less than a second. We need pagination so we tried mongoose library mongoose-paginate-v2, it takes 15 seconds!

Any suggestions?

Out of curiosity, have you tried using the $skip aggregation stage?

This can help you for pagination logic fast with $skip and $limit, just convert it to JS or NodeJS :stuck_out_tongue:

def idlimit(page_size, last_id=None, query=None, collection=None):
"""Function returns `page_size` number of documents after last_id
and the new last_id.

if query is None:
    query = {}

if last_id is None:
    # When it is first page
    cursor = collection.find(query).limit(page_size)
    query['_id'] = {'$gt': last_id}
    cursor = collection.find(query).limit(page_size)

# Get the data
data = [x for x in cursor]

if not data:
    # No documents left
    del cursor
    return None, None

# Since documents are naturally ordered with _id, last document will
# have max id.
last_id = data[-1]['_id']

# Return data and last_id
del cursor
return data, last_id