$count and $sort inside $search

MongoDB Atlas Atlas Search
,
1

Hello!
I believe having the $count and $sort inside $search has been asked by lot of people.
Wanted to know how others are doing it until the feature is provided.
I am observing high memory usage by doing $count and $sort after the $search. If there are 2 different processes mongot for $search and mongod for $count and $sort what is causing this high memory?
Any ideas?

Best,
Supriya

2 Likes
2

Hi again @Supriya_Bansal!

Customers are using the near operator to sort numeric results today. Most search use cases want to be sorted by relevance, so near influences relevance based on numeric order. However, if you want an absolute sort, you can boost the near clause with an arbitrarily high number like:

{
  $search: {
    "compound": {
      "must": [
        {
         "text": {
                  "query": "Adam",
                  "path": "A"
        }},
        {
         "range": {
                  "gt":0,
                  "lt":12,
                  "path": "B"
        }},
        {
         "near": {
                  "path": "B",
                  "origin":12,
                  "pivot": 1,
                  "score": { "boost": { "value": 999} }
                  }
        }
      ]
    }
  }
}

We are releasing a search optimized count operator in a couple months, and search optimized sort soon after.

1 Like
3

Thanks @Marcus!
I am using the “wildcard” operator and our clients want the results to be sorted alphabetically.
Would you have an example?

4

We don’t support that today using the $search operator alone. It is coming soon.

5

Thank you @Marcus for the update!!

6

@Marcus Still struggling with counting search result after full text search. A strange behavior I noticed is when I search for a query “harry potter” it counts fast as compare to “the harry potter”. What difference “the” makes in query which make it too slow.
Any solution?

7

Hi @Marcus, this there an update on this? Or has it been released yet?

1 Like
8

$count with $search is now available. Still waiting on the $sort inside $search.

1 Like
10

Hey @Marcus, any update regarding $sort inside $search? In my use case not having $sort available inside $search will heavily affect performance

11

@Marcus Any idea when faster sort will be coming to atlas search? stored source fields do not provide the speed that is acceptable.

2 Likes
12

My use case would also benefit from “search optimized sort”. I’ll check back here and the changelog for any updates. Is this feature still coming soon?

13

@Elle_Shwer @Andrew_Davidson could you provide any ETA on “search optimized sort” coming to the platform? What is the best place to read about upcoming features?

14

We will keep our Feedback Portal and the changelog you linked up to date. Changelog most reliably.

@Daniel_Beaulieu, if this is a challenge for you, do you mind sharing the following information:

  1. A sample document
  2. A sample query
  3. Data size
  4. number of Results limited to
  5. Acceptable query latency
  6. if you tried Stored Source, and the query latency with that solution
15

I’m implementing paging/sorting on a multi-tenant collection. Any reasonable filter on top of the full data set where results are in the tens or low thousands performs very will. Worst case is there is no filter (other than tenantId filter that gets added automatically). In this case, 750k total documents takes 43 seconds. I’m happy to provide more info or get on a call to discuss more, but figured i’d post what i have collected so far.

db.mycollection.aggregate(
  [
    {
      $search: {
        index: "default",
        returnStoredSource: true,
        count: {
          type: "total",
        },
        compound: {
          must: [
            {
              text: {
                query: "60140fae076eeb001176245c",
                path: "tenantId",
              },
            },
          ],
        },
      },
    },
    {
      $sort: { displayName: 1 },
    },
    {
      $facet: {
        totalCount: [
          { $limit: 1 },
          { $project: { meta: "$$SEARCH_META" } },
          { $group: { _id: null, count: { $sum: "$meta.count.total" } } },
        ],
        results: [
          { $skip: 0 },
          { $limit: 10 },
          {
            $lookup: {
              from: "mycollection",
              localField: "_id",
              foreignField: "_id",
              as: "document",
            },
          },
          {
            $unwind: {
              path: "$document",
            },
          },
          {
            $replaceRoot: {
              newRoot: "$document",
            },
          },
        ],
      },
    },
  ],
  {
    allowDiskUse: true,
  }
);
16

43 seconds with Stored Source? And what is your ideal response time?

17

Yes, 43 seconds is with stored source. The ideal performance is at least the same as non $search based that uses normal mongo index. This runs in 6 seconds.

db.mycollection.aggregate([
  {
    $match: {
      tenantId: '60140fae076eeb001176245c'
    }
  },
  {
    $sort: {
      displayName: 1
    }
  },
  {
    $facet: {
      results: [{$skip: 0}, {$limit: 10}],
      total: [{$group: {_id: null, count: { $sum: 1 }}}]
    }
  }
])
18

Thanks for the response, may respond directly but in the mean time our team is evaluating this. Improving the performance for pagination and sort are extremely top of mind for us at the moment.

2 Likes
19

@Marcus could you please tell me how we can use sort inside the $search pipeline?

20

seems to be not possible right now, only can sort after search, which is slow

21

sorting on strings after $search is indeed slow. Atlas search results are sorted by relevance/score. You can boost the score to sort on numeric or date fields, but not on strings. For example, this will sort on createdDate descending

{
          $search: {
            compound: {
              should: [
                {
                  near: {
                    origin: new Date(),
                    path: 'createdOn',
                    pivot: 1,
                    score: { boost: { value: 999 } },
                  },
                },
              ],
            },
            count: { type: 'total' },
            index: 'default',
            returnStoredSource: true,
          },
        }
1 Like