I am getting an issue when trying to go from mongodb -> kafka topic -> sink connector -> mongodb. When the mongodb source connector detects a change it posts the full document as well as the “updateDescription”. If I go to sink that topic to another mongodb and the “updated field” is a nested value it appears as “value1.value2” which violates field name constraints in this connector. Is there a config that can be changed in the sink to deal with this? I can only make changes on the consumer side not the producer, my only other option would be to write a consumer off of that topic, remove that block, and then send it to a new topic which would then be sunk to the database though I would prefer not doing that.
2 Likes