Realm functions limit() and skip() are badly broken

Hello! To implement GraphQL pagination, there appear to be two options: offset-based or cursor-based. I decided to try offset-based pagination to keep my test client a little simpler. This thread indicates pagination has to be done with a custom resolver, so I started with a simple pagination function.

…or rather, I thought it would be simple. See the following function code:

exports = async ({limit = 10, offset = 0}) => {
  const db = context.services.get('mongodb-atlas').db('DATABASE_NAME');
  const results = await db.collection('COLLECTION_NAME')
      .find({})
      .sort({id: 1})  // Sort by ID
      .skip(offset)   // Skip `offset` results.
      .limit(limit)   // Limit to `limit` results
      .toArray();

  console.log('results returned', results.length);
  return results;
}

For a collection of size 100 with default limit 10 and offset 0:
Expected number of results: 10 (matching limit)
Actual value printed: 90 – what?

If you flip the order of operations and do the limit before the skip, the result is always 0.

This seems like it should work. The API limitations guide does not list limit or skip as unsupported options for the find command. The Realm docs don’t mention limit/skip at all.

The only docs I could find on skip/find in JS explicitly state:

The order in which you call limit and sort does not matter

This is definitely not true in Realm Functions (although I’m not sure if these docs apply, which is confusing as this is MongoDB running in Node, but the find() API is a little different with projections in Realm vs. args in Node).

Concrete suggestions:

  • If skip and limit don’t work, (1) don’t let them be called at all – returning an invalid result is worse than an error, (2) update the find unsupported operation list
  • Update the MongoDB Realm Atlas service docs to explicitly mention these aren’t supported.
  • In the tutorial for custom resolvers, make it clear the access to MongoDB is different than a NodeJS driver + link the relevant docs.
  • Provide examples for implementing basic pagination with a custom resolver if that’s the recommended solution (even if it can’t be fully implemented on the server yet)

Whew! That ends my functions adventure for the evening. :wink: Thanks for all of your work on Realm, and I hope these suggestions are helpful for the future. :smiley:

1 Like

Hey Matthew -

Sorry that you had to run through so many hoops while implementing pagination.

The API limitations guide that you linked does show skip as an unsupported operation with find. It also doesn’t show skip as a supported operation at all. I’d love to get your feedback on how we can make this a more clear on our end and pass this onto the documentation team.

The fact that limit() does not work when preceded by a skip() is a bug we’re investigating. I’m happy to keep this thread updated when it is resolved.

In general (as pointed out in the thread you linked), we recommend using find() and limit() when implementing pagination due to the skip() limitation.

  • In the tutorial for custom resolvers, make it clear the access to MongoDB is different than a NodeJS driver + link the relevant docs.
  • Provide examples for implementing basic pagination with a custom resolver if that’s the recommended solution (even if it can’t be fully implemented on the server yet)

This is also feedback I will pass onto the documentation team.

3 Likes