Range operator on ObjectId, string

Currently, some methods of pagination in atlas search are difficult due to the limitations available operators. By this I mean that it is difficult to implement a keyset or cursor based pagination using object id’s.

In keyset pagination, you use filtering instead of skip + limit. To use this in Atlas Search, you might do a query like the following, where you know the _id from the previous set of results:

{
  compound: {
    must: {
        text: {
            path: "..."
            query: "my search query" 
        }
    },
    filter: {
        range: {
            path: "_id",
            gt: ObjectId('...')
        }
    }
  }
}

To make this possible, it would be great if we could use the range operator with more data types, like objectid and strings.

4 Likes

Exactly, and originally, I thought this operator could do more than it does right now, like working with more data types, as you mentioned, and also I was thinking maybe it would be better if we could just wrap up our actual query around with the Range operator, will that make more sense? :slight_smile:

Spending some time last night considering it, and here is my thought: by the time I’m writing, we still face the limitation of the range operator( only can compare number and date types), so I’m thinking we can make a compromise if we still wanna implement cursor-based pagination by adding “$match” stage right after “$search” stage :

{
    $search:{
        index:'your index',
        text:{
            path:'your path...',
            query:'your query'
        }
    }
},
{
    $match:{
        _id:{
            $lt:'ObjectId(...)' 
        }
    }
},
{
    $sort:{
        _id:-1
    }
},
{
    $limit:5
}

basically, so far, it’s a bit difficult to add conditions to the search stage, but we can still use the match stage to help us do the conditions.

Hopefully, here could be helpful for anyone who comes here looking for pagination :slight_smile:

I’m having a similar issue. How can we implement cursor based pagination in this use case?