Transform timestamp in mongo-kafka connector

Working with Data Connectors & Integrations
1

When using mongo-kafka source connector, I receive

Caused by: org.apache.kafka.con │ │ nect.errors.DataException: Only Struct objects supported for [converting timestamp formats], found: java.lang.String\n\tat org.apache.kafka.connect.transforms.util.Requirements.requireStruct(Requireme │ │ nts.java:52)\n\tat org.apache.kafka.connect.transforms.util.Requirements.requireStructOrNull(Requirements.java:61)

with the following config for transforms:

"transforms": "revecied_at_unix_to_iso",
"transforms.revecied_at_unix_to_iso.type": "org.apache.kafka.connect.transforms.TimestampConverter$Value",
"transforms.revecied_at_unix_to_iso.field": "fullDocument.received.$date",
"transforms.revecied_at_unix_to_iso.format": "yyyy-MM-dd'T'HH:mm:ssZ",
"transforms.revecied_at_unix_to_iso.target.type": "string"

How to make SMT work with mongo-kafka connect?

2

Hi @Sendoh_Daten,

The messages from mongo-kafka source connector are Extended JSON strings. What does the document look like in the collection ? Especially for fullDocument.received field.

Looking at Kafka Connect TimestampConverter SMT, the format property can used to generate the output or parse the input. Could you clarify whether you’re trying to generate a string date in the format of yyyy-MM-dd'T'HH:mm:ssZ ?

Regards,
Wan.

3

The next version of the connector will support formats other than string so you will be able to do SMTs

1 Like