Developer feedback for ease-of-use improvements for Go Driver

I’d like to continue the conversation we started in the Google Group about an ease-of-use feature-set - making day-to-day operations easier/more compact.
I can copy the contents over, but it sounds like that thread will be archived.
Ultimately, I’d love to work with the team and the community to help develop a really straightforward interface and I am happy to use my own hours to get it done.

Also adding this ticket as an example of ease-of-use features for the library: https://jira.mongodb.org/browse/GODRIVER-903
ListCollections() and Indexes().List() both return a cursor that you have to define a custom type and unmarshal into. Why don’t they just return a slice of Collection{} and a slice of IndexModel{} instead, so that I as a developer can access things WAY more efficiently.

1 Like

Hi Christopher,

We currently have our hands full working on the 1.4.0 driver release, which will correspond with the MongoDB 4.4 server release. Once that’s over, I plan to organize a meeting with the team to go over the feedback you outlined and get you an answer about which tickets we can re-open/accept PRs for now and which ones are already planned for cross-drivers work in the future (e.g. client side operations timeouts). Does this sound OK to you?

– Divjot

Actually Divjot, I was hoping we could take a slightly different tack.
I understand that the driver tries to use the API fairly exclusively and at this point we can’t introduce breaking changes, and my primary qualm is with how hard the functions themselves are to use. It takes me forever to get teammates spun up and it’s hugely error prone because there are just so many knobs you have to turn and so many gotchas you have to check for.

I would like to add another package to mongo-go-driver - mongo-for-humans or something similar which still uses mongo-go-driver, but abstracts a lot of the complexity away from the developer. That way, if you want, you can have full control, or if you are just doing something common (e.g. SelectAll with no timeout), then it’s really easy. If you support this route, I would love to just come up with an interface spec together here and I’m happy to do a majority of the work.

1 Like

I understand that there are things that are more difficult to do in our API compared to mgo and many of these are because we have a set of specs to follow and have to ensure that we offer a way to work with both raw BSON and decode BSON into native Go types so an application can choose what it wants to do based on performance constraints. I’m not convinced that another package is the right way to proceed, though. Many of the BSON issues you linked in your original post have been solved by the mgocompat.Registry, released in v1.3.0 of the driver. This new BSON registry was introduced to address the issues users were facing when migrating BSON libraries. Some of the other BSON features linked would have to be solved in our BSON library, not in a helper. An example is a struct tag to treat string fields as ObjectIDs. Many of the CRUD helpers in the tickets you linked (e.g. FindAll) would be added to the main mongo package rather than having users remember that some helpers are in mongo and others are in a separate helper package.

1 Like

Hey Divjot -

Happy Monday :slight_smile:

I truly appreciate you working the tickets I linked - that will make life easier for most.

My fundamental issue is how verbose normal functions are to implement.

Ultimately, I’m about the least about of key strokes - I like to write lean, clean code, and unfortunately it’s not possible to be DRY with the current go library for standard tasks.

The problem is there are too many knobs exposed that have to be turned. I want to leverage all these awesome and cool features you’ve exposed to make some really simple, less error prone functions. Look at how clean and readable this line is. It’s really easy for me to teach new developers - they just check a single error object. Forget about how we would do it for a moment, and just compare:

if err = coll.Find(query).All(&sliceToUnpackTo).WithTimeout(time.Minute); err != nil {}

Here’s how I have to do that in today’s world:

   ctx, cancel := context.WithTimeout(context.Background(), time.Minute)
   defer cancel()
   cursor, err := coll.Find(ctx, query)
   defer cursor.Close(context.Background())
   for cursor.Next(context.Background()) {
       if err = cursor.Decode(&someObj); err != nil {
           break
       }
       sliceToUnpackTo = append(sliceToUnpackTo, someObj)
   }
   if err == nil {
       err = cursor.Err()
       if err == nil && len(sliceToUnpackTo) { err = ErrNotFound }
   }
   if err != nil {
       return err
   }

No need to deal with using context or cancel functions or closing the cursor. (They always forget the defers). No need to play with defering the close of the cursor. Every time they implement, I always see them forget to check for a decode error or a cursor error, or setting the error not found. Usually, day to day, this is what our developers are doing. We’ve needed to actually use the cursor to do something interesting three times across literally hundreds of functions. It’s useful to have, but man it makes for verbose and repeated code.

I want a standard mongo library to have functions to make unpacking and timeouts easy. Because of the current mongo-go-driver design though, introducing this friendly syntax there isn’t possible because of backwards breaking changes. This is why I suggest a different package to wrap mongo-go-driver.

Do you follow my logic?

I can use .All() as well, but my point is the number of errors you have to check when you’re doing it is still higher. It would be nice to bundle that up a little.

All is definitely a more concise way of doing this. Changing how we do timeouts isn’t really on the table. Parts of the Go language that were built before context.Context has been updated to accept contexts (e.g. net/http). I would be open to a PR to add FindAll/AggregateAll to condense two error checks into one, but adding a completely separate fluent API isn’t really possible, especially given the fact that Go doesn’t have method overloading.

As a heavy user of this library, I, a user, am providing feedback that the current interface isn’t user friendly in general compared to literally every other mongo golang package out there. I don’t think we need method overloading, so that works out - Ideally, I want a different interface on top of mongo-go-driver - wrapping the existing code in mongo-go-driver, not the API.
Let’s put how to do timeouts aside and realize that usually we aren’t using a context, we pass nil, background or todo, so why force the user to specify it on every call? Why not just add it as an option instead? We can’t make Distinct easier without breaking the library, We can’t make these kinds of changes without breaking the library. That’s why I want to work together to design some wrappers in a separate library that I’m willing to maintain myself.
There are just design decisions in this library that have been made with Mongo, not the community, in mind. And that’s fine, we have different target audiences for our libraries.

In the Google thread I think that you had mentioned trying to do an mgo shim. We could use that as an opportunity to team up, extend that further and create something awesome. I envision things like, rather than having to manipulate a cursor, you could instead read from a channel.

I think what we could do is find libraries that currently consume mongo-go-driver and see what kind of patterns are being used to help drive what users are usually doing.

The approach taken for the PHP driver is interesting here: while the original PHP extension was designed to be used in userland, the second-generation extension was reduced in scope, to provide a small and inconvenient lower-layer, on top of which the userland driver was written, and meant to be used by most developers.
We have a similar situation here, with the new driver being aligned in terms of API with other MongoDB drivers, but being arguably less convenient than mgo. It could be used to provide the basis for a more user-friendly and more idiomatic driver written on top of it.

1 Like

Hi,

Sorry I haven’t updated this thread in a while. Our product manager is looking into this and conducting some field research regarding the API differences between the Go driver and mgo. As you can imagine, we want to make sure we don’t speed through this as we want this to be an official API that we can recommend to our users.We’ll post updates here once we have a concrete plan.

– Divjot

1 Like