Outbox Pattern using mongo-kafka connector

I want to implement Outbox Pattern in our microservices using Mongo-Kafka connector, in my outbox(a MongoDB collection) I store topic data using these fields:
kafka_topic, kafka_key, kafka_value, kafka_header_keys, kafka_header_values.

But how should I config mongo-kafka connector to dynamically choose the topic from the kafka_topic field and also its values and headers from other outbox fields?
I can not find settings to do this from its config reference.

I’m also thinking about getting a fork from mongo-kafka connector repository and extend it with my custom implementation of source class or any other way.

I would greatly appreciate it if you could help me.

Hi @Mehran_prs,

Sounds interesting and perhaps should be something we add to the roadmap.

Could I ask you to add a ticket to the Kafka connector jira project outlining your requirements?

I’m still a little unclear if MongoDB is the Sink or the Source in this scenario? So could you also add some more detail, so I can ensure I have the correct mental model of the outbox pattern.

Ross

1 Like