I’m using the mongodb source and sink connectors. It’s very easy to produce the change streams to a kafka topic per each collection on a single mongodb source connector (which has no collection field). But, I couldn’t find out how to consume the change streams to a mongodb collection per each kafka topic. Is it not possible? or am I missing something?
You need to specify the MongoDB CDC Event handler at the sink. ( com.mongodb.kafka.connect.sink.cdc.mongodb.ChangeStreamHandler)
See the " Selective Replication in MongoDB" section in this blog post MongoDB Connector for Apache Kafka 1.4 Available Now | MongoDB Blog
I’m in the process or writing tutorials for the kafka connector, there is one that is already done that covers this:
https://github.com/mongodb-university/kafka-edu/tree/main/tutorials/3-SelectiveReplication
1 Like
URL seems to be broken, can you please share the updated URL for Kafka-education topic
The updated link to the tutorial is -
https://github.com/mongodb-university/kafka-edu/tree/main/docs-examples/examples/v1.7/cdc-tutorial
2 Likes