I am trying to update multiple different documents within different collections using a transaction from the Swift Driver for a Vapor application. I have looked at the example transactions docs and it doesn’t seem to provide a concrete example of how to actually use them.
I am currently stuck on firstly being able to implement a transaction and secondly how to get data back from one or more of the transactions such as the objectID for a newly inserted document.
I have created example code expanding on the ComplexVaporExample to hopefully help demonstrate the problem I am having.
func kittenTransactionExample(_ req: Request) throws -> EventLoopFuture<Response> {
let newKitten = try req.content.decode(Kitten.self)
let neighbourhoodIncDocument: BSONDocument = ["$inc": ["totalCats": 1]]
let neighbourhoodFilter: BSONDocument = ["name": "London"]
let session = req.application.mongoClient.startSession()
session.startTransaction().flatMap { _ in
req.neighbourhoodCollection.updateOne(filter: neighbourhoodFilter, update: neighbourhoodIncDocument)
}.flatMap { _ in
req.kittenCollection.insertOne(newKitten)
}.flatMap { _ in
session.commitTransaction()
}.whenFailure { error in
req.eventLoop.makeFailedFuture(error)
}
// this doesn't compile
// how do I pass back a Response future ????
// how do I access insertedID for the new kitten ???
}
struct Neighbourhood: Content {
let name: String
var totalCats: Int
}
/// Possible cat food choices.
enum CatFood: String, Codable {
case salmon,
tuna,
chicken,
turkey,
beef
}
struct Kitten: Content {
var _id: BSONObjectID?
let name: String
let color: String
let favoriteFood: CatFood
}
This is what I have come up with so far, which at least compiles but I am not sure if it is the correct approach. The example code in the documentation uses the whenFailure call back but I can’t get that to work. I also haven’t implemented session.abortTransaction() or session.end() and I am not sure if they are required and if so, where.
Edit - BELOW DOESN’T WORK - results in a crash
func kittenTransactionExample(_ req: Request) throws -> EventLoopFuture<Response> {
let newKitten = try req.content.decode(Kitten.self)
let neighbourhoodIncDocument: BSONDocument = ["$inc": ["totalCats": 1]]
let neighbourhoodFilter: BSONDocument = ["name": "London"]
let session = req.application.mongoClient.startSession()
return session.startTransaction().flatMap { _ in
req.neighbourhoodCollection.updateOne(filter: neighbourhoodFilter, update: neighbourhoodIncDocument)
}.flatMap { _ -> EventLoopFuture<BSONObjectID> in
req.kittenCollection.insertOne(newKitten)
.flatMapThrowing { insertOneResult in
guard let insertedID = insertOneResult?.insertedID.objectIDValue else {
throw Abort(.notFound)
}
return insertedID
}
}.flatMap { objectID -> EventLoopFuture<BSONObjectID> in
session.commitTransaction()
.map { objectID }
}.map { objectID in
return Response(status: .ok)
}
}
Is there any intention of expanding upon the ComplexVaporExample app? It is a solid foundation and incredibly useful but I think most projects would soon go beyond the example functionality shown in it.
Ok - I’m unable to move up to 12 at the moment… and on 11.6 I can’t even get a basic vapor project to compile due to a swift-nio-transport-services bug.
/Users/mlynn/code/.../.build/checkouts/swift-nio-transport-services/Sources/NIOTransportServices/NIOTSConnectionChannel.swift:425:41: error: value of type 'NWConnection' has no member 'startDataTransferReport'
Are you able to get 11.7? (latest public release).
I can go on my work Macbook and give 11.7 a go and see if it works. It seems strange that you are unable to compile a basic Vapor project though. Probably worth doing the standard project clean stuff like deleting derived data, cleaning folder etc and trying again if you haven’t already.
Are you able to speak to the Mongo Swift Driver team if this nio problem persists?
I’ve got no issue building with Xcode 11.7 using the Mongo Swift Driver and my personal Vapor project. If that bug persists then the Vapor Discord would be a good place to ask about it as I don’t imagine it being a permanent issue that can’t be resolved.
Hi @Piers_Ebdon, thanks for reaching out and for your patience while our team was out of office.
To address a few points of confusion that have come up here thus far -
In order to have any operation you perform be considered part of a transaction, you must pass in the session you used to start the transaction as the session parameter. For example your updateOne call should have a final session: session parameter. This is how the driver determines whether an operation is part of a particular transaction or not. This is necessary because MongoClients are intended to be global, thread-safe objects, and there could be requests occurring in parallel using the same client that are not intended to be part of the transaction.
You must always call end() when you are done using a session. Failure to do so will result in a assertionFailure (only evaluated in debug mode) when the ClientSession goes out of scope. That said, you may find it easier to use the MongoClient.withSession helper as that will handle the cleanup for you automatically by calling end() once the passed-in closure has finished executing. (Shown in example below.)
abortTransaction() will effectively “throw away” the transaction a session has in progress if it exists. A session can only have one transaction in progress at a time, so you would need to call this if you wanted to perform multiple transactions with the same session. Calling session.end() will automatically abort an in-progress transaction if one exists, so if you are only ever doing a single transaction per session it’s not strictly necessary to also call abort.
I think the source of the crash in your example method is that you are not hopping back to the request’s eventLoop before the end of the route closure. Futures returned from the driver may fire on any of the event loops in the ELG you pass in when creating the MongoClient so you need to ensure you hop back to the correct one for the request before returning. I’ve shown how to do this below. (We discussed before upcoming work to make the driver easier to use with Vapor - one goal of that project is to remove the need for you to do this kind of hopping yourself.)
I did see that you commented asking about our plans to implement the “convenient API” for transactions, which adds support for automatically retrying transactions upon certain types of failures. We do not have a timeline for completing that work at the moment but we will certainly take into account your request as we plan upcoming work.
I am glad you have found the complex example project useful. Our developer advocacy team including @Michael_Lynn is working on creating more examples now, and we could consider augmenting the existing example app as well. We very much appreciate hearing from you on what your pain points have been thus far (such as you have described in this post) as it helps us figure out what examples we should prioritize creating.
Ok, all that said, onto the code sample you’ve provided! I’m able to get it working as I would expect by doing the following:
func kittenTransactionExample(_ req: Request) throws -> EventLoopFuture<Response> {
let newKitten = try req.content.decode(Kitten.self)
let neighbourhoodIncDocument: BSONDocument = ["$inc": ["totalCats": 1]]
let neighbourhoodFilter: BSONDocument = ["name": "London"]
return req.application.mongoClient.withSession { session in
session.startTransaction().flatMap { _ -> EventLoopFuture<UpdateResult?> in
req.neighbourhoodCollection.updateOne(filter: neighbourhoodFilter, update: neighbourhoodIncDocument, session: session)
}
.flatMap { _ -> EventLoopFuture<BSONObjectID> in
req.kittenCollection.insertOne(newKitten, session: session)
.flatMapThrowing { insertOneResult in
guard let insertedID = insertOneResult?.insertedID.objectIDValue else {
throw Abort(.notFound)
}
return insertedID
}
}
.flatMap { objectID -> EventLoopFuture<BSONObjectID> in
session.commitTransaction()
.map { objectID }
}
.map { objectID in
Response(status: .ok)
}
}.hop(to: req.eventLoop)
}
Basically what I’ve changed is switched to using withSession, added a hop(to:) at the very end of the returned closure, and passed the session parameter to both updateOne and insertOne.
Let me know if that works for you or if you have any further questions or issues.
I am running out of superlatives to say thank you for your responses!
Apologies for the slow reply. Currently in the process of moving cities in the UK and so I have been unable to really look into your amazing detailed explaination and answer.
Following your response I have been able to get transactions working with the Swift driver .
I only commented on the conventient transactions api ticket because I was desperate to get the transactions working in my app , so please ignore.
In the example code you provided, I assume if updating the neighbourhood document fails then the whole transaction fails and there is no need to add a check on the updateResult response to ensure it has succeeded. Is that correct?
I will let you know if I have any other questions but I think everything else is covered
I am glad you were able to get your application working!
Yes, that’s correct. You can test that behavior by, for example, modifying your neighbourhoodIncDocument to contain an invalid update operator the MongoDB server will reject (for ex. let neighbourhoodIncDocument: BSONDocument = ["$blah": ["totalCats": 1]]).
In that case, the response I get to a POST request to the corresponding endpoint is
{
"error": true,
"reason": "WriteError(writeFailure: Optional(MongoSwift.MongoError.WriteFailure(code: 9, codeName: \"\", message: \"Unknown modifier: $blah. Expected a valid update modifier or pipeline-style update specified as an array\")), writeConcernFailure: nil, errorLabels: nil)"
}
And both the kitten and neighbourhood collections remain unchanged.
.
, so please ignore.