Generating Unmanaged Realm Objects from Equivalent MongoDB Atlas BSON Documents (Cross-Post)

This is a repost of a question I asked in the wrong forum. I’m reposting here since it has much more to do with the Swift SDK than MongoDB Realm in general.

I’m working on a Swift iOS app using Realm Sync and MongoDB Atlas. It’s a photo editing app, and I want people to be able to create filters that they have write access to, and be able to share them, so that other users can have read-only access to them to download them on their phone.

I’m able to sign in, open a realm, create filters, store them, and access them.

However, I’d like to run a query for all filters available to download (i.e. those which aren’t owned by me). My data is partitioned by the user_id property.

Here is the schema for my filters:

{
  "title": "DFFilter",
  "bsonType": "object",
  "required": [
    "_id",
    "user_id",
    "name"
  ],
  "properties": {
    "_id": {
      "bsonType": "objectId"
    },
    "user_id": {
      "bsonType": "string"
    },
    "name": {
      "bsonType": "string"
    },
    "edits": {
      "bsonType": "objectId"
    }
  }
}

And here is my equivalent swift Object:

@objc public class DFFilter : Object {
    @objc dynamic public var _id: ObjectId = ObjectId.generate()
    
    // Partition Key
    @objc dynamic var user_id: String = ""

    @objc dynamic public var name: String = ""
    @objc dynamic public var edits: DFEdits? = nil
    
    override public static func primaryKey() -> String? {
        return "_id"
    }
    // ...

And here is how I’m performing the query:

        let collection = database.collection(withName: "DFFilter")

        let filterDocument : Document = [
            "user_id": [
                "$ne": AnyBSON(forUser.id)
            ]
        ]

        collection.find(filter: filterDocument) { result in
            switch result {
            case .failure(let error):
                print(error)
            case .success(let documents):
                print(documents)
                completion(documents.map({ document in
                    let dfFilter = DFFilter(value: document)
                    print (dfFilter)
                    return "foo" // just a placeholder for now
                }))
            }
        }

However, the initializer is failing to create a local unmanaged DFFilter object from the BSON Document I’m getting from Realm:

Terminating app due to uncaught exception 'RLMException', reason: 'Invalid value 'RealmSwift.AnyBSON.objectId(6089b6e38c3fafc3e01654b1)' of type '__SwiftValue' for 'object id' property 'DFFilter._id'.'

Here’s what the BSON document looks like when I print it in the console:

(lldb) po document
▿ 4 elements
  ▿ 0 : 2 elements
    - key : "_id"
    ▿ value : Optional<AnyBSON>
      ▿ some : AnyBSON
        - objectId : 6089b6e38c3fafc3e01654b1
  ▿ 1 : 2 elements
    - key : "name"
    ▿ value : Optional<AnyBSON>
      ▿ some : AnyBSON
        - string : "Hey Hey"
  ▿ 2 : 2 elements
    - key : "user_id"
    ▿ value : Optional<AnyBSON>
      ▿ some : AnyBSON
        - string : "6089b62f9c0f6a24a1a5794b"
  ▿ 3 : 2 elements
    - key : "edits"
    ▿ value : Optional<AnyBSON>
      ▿ some : AnyBSON
        - objectId : 6089b6e38c3fafc3e01654b2

I’ve tried search around for answers but I’m coming up blank. This indicates to me that potentially my whole approach to this problem might be mistaken?

It is worth pointing out that the edits property of DFFilter which you see in the schema referenced by an object_id is a different object of type DFEdits. I’m also not sure how I can query an object and its dependencies at once? Does MongoDB resolve these automatically?

It seems like it might even be easier to just write a GraphQL query directly and decode the response into my local object types?

Just not really sure which direction to head.

Are you using RealmSwift, the Swift SDK?

I’m using @import Realm; in my objective-c code which is consuming my persistence-layer API.

Within the persistence-layer, where all the aforementioned code snippets are, and which is built in Swift, I’m using import RealmSwift

Here is what my SPM include looks like:

Ok. Just needed clarification so we answer in the right language; your coding in ObjC, not Swift.

Objc
let collection = database.collection(withName: "DFFilter")

Swift
let results = realm.objects(DFFilter.self)

Oh, just a fyi, and this isn’t valid in Swift as Realm doesn’t have public swift vars, just vars and private vars

@objc dynamic public var _id: ObjectId = ObjectId.generate()

I’m not sure I understand your objc/swift distinction. Are you suggesting that I’m using the RealmObjC SDK when I use the database.collection API, and I’m using the RealmSwift SDK when using realm.objects(DFFilter.self)

Per my reading of the docs, realm.objects(DFFilter.self) will return all the DFFilter objects in that realm, which is partitioned to my user_id, so this realm isn’t aware of publicly-available filters, not owned by me (I.e. do not share my partition key).

As @Andrew_Morgan suggested in the original thread, I could download all the publicly available filters in my system to each client device but that’s not really applicable here, there’s going to be an ever-growing volume of those, and it doesn’t seem memory nor disk nor bandwidth efficient to download my entire backend to each client device.

I just need to query my backend for some data, and generate local unmanaged versions of that download. It’s sounding more and more like I need to just process the BSON as if it were coming from a generic backend, and populate unmanaged DFFilter objects manually?

Just really curious why a Document backed by a schema, doesn’t allow me to generate a local object that defined that schema.

Apologies if I asked the same question again, I just want to be extra explicit to make sure we don’t talk past each other.

Appreciate the help, thank you so much.

        guard let realm = inRealm.realm else { return }
        
        let results = realm.objects(DFFilter.self)
        let filteredResults = results.filter("user_id != '\(forUser.id)'")
        print(filteredResults.count) // Printed 0 in the console (Incorrect)
        
        
        //==============================
        
        let client = forUser.mongoClient("mongodb-atlas")
        let database = client.database(named: "Darkroom")
        let collection = database.collection(withName: "DFFilter")

        let filterDocument : Document = [
            "user_id": [
                "$ne": AnyBSON(forUser.id)
            ]
        ]

        collection.find(filter: filterDocument) { result in
            switch result {
            case .failure(let error):
                print(error)
            case .success(let documents):
                print(documents.count) // Printed 3 in the console (Correct)
            }
        }
1 Like

Correct. Just wanted to establish which SDK was being used as your coding platform so if we answered with code it would be applicable as the two SDK’s are quite different.

As far as the SDK’s go, when you touch a partition, everything in that partition is sync’d. So if you want to query for your DFFilter objects, you will have to open Realm (a partition) that has those filters and they will all be sync’d to your device. Query based (partial) syncs are not currently available (as they were in the prior Realm). I other words, for the SDK’s there’s no “server data” and “client data”; only “sync’d data” which exists on both the server and the client. When a query is run, no data is ‘pulled’ from the server, it runs against the local data (which is why it’s fast as it’s not waiting on the server or the internet’

You can’t query (only) the backend for data (from the SDK) because it will exist locally but you can do the latter. It’s the same process - to query against a Realm you have to open that Realm (partition), which then sync’s everything in that partition.

I am curious - your question has a local Realm Object that defines the schema of the object. Right? What does the ‘generate a local object’ part of that mean? - like if you create an object in the the Realm console an object is generated in code? (that would be cool).

@objc public class DFFilter : Object {
    @objc dynamic public var _id: ObjectId = ObjectId.generate()
    ...

I do the opposite, or at least I think I am doing the opposite lol. I create my objects in code and that in turn generates/creates those objects in the Realm Console (Atlas) when it syncs.

That being said, you CAN store data on the server that’s independent of the client. As long as the client doesn’t touch that data then it won’t sync (download). You could then access that via REST calls or server-side functions. That really sounds like a solution - keep your filters on the server only, make a server side function call and then parse the local data and instantiate ObjC or Swift objects to hold the data.

1 Like

I appreciate your responses, Jay, thanks again for taking the time. I think I understand the nuance between Realms being synced automatically to their partitioned data in a collection, so let me take some time to share some diagrams that might shed some light on what I’m trying to do, and where I’m struggling.

In this scenario, there are two users: A, and B. Each has their own Realm in an iOS app. Their realms are initialized with their User ID as their partition value. They automatically get their own documents synced to the client. This is all working as expected.

If User B shares a filter, it should create a new Document in the cluster which references user B, but is accessible by anyone.

When a filter is shared, it will create a link that includes the ID of the shared filter (in this digram, 5)

User A can use that link to install the filter. I would like to get the filter details, and create a filter, owned by User A, with the data of the shared filter.

My question is: How do I get the details of that shared file in my swift app, since I can use the realm to do it. I tried querying it directly via the collection, and that worked, but I’m getting a Document type response, which I can’t initialize a Filter object from.

I have seen the suggestion that I open a second realm on User A’s app, which can sync all the shared filters, and access it that way, but I don’t want to sync every shared filter in the cluster, it’d be wasteful and it will keep growing over time.

It sounds like I need to just fetch the shared filter from the collection by its id (like I’m doing in my snippet), and then instantiate a filter object and populate it with the values in the Document BSON manually?

I was able to just use the GraphQL API to access the object in the Atlas cluster directly including its nested objects. Then I can construct filter objects and add them to my realm manually. Seems like that the safest/easiest way to go

Unfortunately, that won’t work. From the SDK, user B won’t have access to the Shared Filters if they are only located on the server. All objects that are sync’d must contain a partition key. For user B, or any user, to access the filters they must access the filters partition, which means the filters sync to all devices.

It’s a little unclear why, when a user wants to share a filter, additional data is created. Perhaps having all filters stored in one partition, and then setting up permissions that define which users can access which filters. Then sharing would just be a matter of adding the users uid to the filter. At a high level, something like this example

class DFFilter : Object {
    @objc dynamic var _id: ObjectId = ObjectId.generate()
    @objc dynamic var _partitionKey = ""
    @objc dynamic var owner: UserClass? //the owner of the object

    let userList = List<UserClass>() //users who can access it
    //or
    let uidList = List<String>() //a list of user id string that can access this object

If you want to sync an object in a partition (a Realm), all objects in that partition are sync’d.

When a user shares B a filter with user A, they’re not sharing access to the same document. If that were the case, then any changes either user made to the document would be synced. That is not the sharing model we’re building.

In our sharing model, when user B shares a filter, they create an immutable copy. Otherwise, you might install a filter that increases your saturation, then try it again the next day, and it might make your photo black & white.

I’m coming around to the notion that our sharing model needs to work outside of the scope of MongoDB Realm Sync.

I’m coming to the conclusion that when a user shares a filter (and “share” in this context is used in the UX, app-based sense, not the Realm partition-sharing sense), I can run a GraphQL mutation to create a new document on the server, and then use GraphQL to access that document when installing that filter, and reconstruct a new DFFilter object, owned by the user who installed that filter.

I want to circle back around on this as I was assuming you were using the RealmSwift Sync SDK, but you’re not; your also using MongoDB Remote Access - iOS SDK - not sure how I overlooked that.

So flipping my suggestion around, if you are NOT sync’ing any of the Filter data, then that part of the SDK should enable you to do what you want. It may require a server side function as well - have to give that some thought.

I’m finding myself quite confused by the various SDKs to be honest:

  • RealmSwift Sync SDK
  • Realm Objective C SDK
  • MongoDB Remote Access - iOS SDK

All I enabled was add Realm as a swift package manager, I’m not making an explicit decision on which SDK to use, and the boundaries between these aren’t clear to me. I’m simply using the APIs available to me.


CleanShot 2021-05-04 at 14.00.44@2x

@Majd_Taby Let me see if I can help. In the RealmSwift SDK that you have installed there are two methods for accessing data from MongoDB Atlas

  1. Use Realm Sync, this will automatically translate MongoDB documents into Realm Objects and store them locally in a Sync Realm. You fetch these documents by opening sync realms with a partitionKey, which is a field you define on the Atlas document - you pass in a value and Realm Sync grabs these documents and translates them to Realm Objects - syncing any deltas on the documents back and forth between Atlas and the synced Realm you just opened - https://docs.mongodb.com/realm/sdk/ios/examples/sync-changes-between-devices/

  2. You can fetch documents using RealmSwift’s MongoDB Remote Access APIs - this will fetch documents from Atlas based on a MQL query. The results will be returned to you as JSON document just as it would if you were using a MongoDB driver. Think of this as a REST API request-response. If you want to store them in a realm you will need to convert the documents into Realm objects in a separate non-syncing local realm. We do not provide this mapping for you because well, that’s what sync does. There are two types of realms, syncing and non-syncing - you would open separate realms (separate variables with separate configurations) and access and store data in them indepedently.
    https://docs.mongodb.com/realm/sdk/ios/examples/mongodb-remote-access/

I hope this helps. In general, I recommend using the Realm Sync APIs as it simplifies the architecture. But there are some use cases where using the Remote Data Access APIs might be useful. Depends on your usecase

1 Like

To go along with @Ian_Ward excellent explanation

Much (all?) of Realm is based in ObjC and when Swift came out a Swift interface was added to enable Swift Developers to interact with Realm objects in a more Swifty way.

If you are coding in ObjC, the ObjC SDK is for you. If you are a Swift developer, RealmSwift is for you.

MongoDB Remote access is built into both and enables interaction with the Realm server in a less structured way.

Both ObjC and RealmSwift SDK’s rely on your custom objects whereas the Remote Access aspect doesn’t have custom objects at all and uses BSON Documents to interact with your server data.

The ObjC and RealmSwift SDK’s require far less code to retrieve, update and write data. The Remote Access aspect (to me) is much more raw and low level.

For your situation, combining the two I think may be a solution as with Remote Access you can get and write data without sync’ing everything in a partition. So, for example, you could download a filter, do something with it in code and write it to the server without ever sync’ing it to the device. I am pretty sure using that process, you could write it to a partition that would then sync to another user or enable shared access in some other creative way.

Ah. Thank you @Jay and @Ian_Ward for your explanations.

I must admit I have never investigated Realm before the MongoDB acquisition so the historical context was missing and this context helps me follow your earlier responses.

Yes based on the sharing model we’re pursuing, using a synced Realm for user-data makes sense, and using raw queries using Remote Access for downloading publicly-available documents makes sense too. I suppose the missing part in my approach was the expectation that I ought to be able to decode a Mongo DB DFEdits BSON Document to a Swift DFEdits object directly, but I can also just map the fields manually. That’s not an issue.

And I presume that read-access restrictions would have to be implemented in a server-side function that ensures for only shared filters are accessible via Remote Access?

You can apply permissions by using Realm Rules - https://docs.mongodb.com/realm/mongodb/define-roles-and-permissions/

I’d also say that you can have read-only sync realms, this is a common use case for public data for mobile apps, for instance, a catalog for an inventory or shopping app. See here:
https://docs.mongodb.com/realm/sync/partitioning/#partition-strategies

Also, a new partitioning guide is here -
https://www.mongodb.com/how-to/realm-partitioning-strategies/

I know this thread is a bit old, but since I am having essentially the exact same issue, I will try my luck. Mapping this manually turn out as being pretty inefficient for me (a query for around 10 objects taking 3s+ to initialise the swift objects). Is there a way to use Decodable or some other efficient method? Also, why is RealmSwift not allowing the conversion? Since I imagine this has to be done somehow under the hood for the synced Realms…
@Ian_Ward @Jay

@David_Kessler Trying to keep all of the data in one spot. Please see the StackOverflow questions as there may be more to this part of the question: