The `Flatten` Single Message Transform (SMT) is useful when you need to collapse a nested message down to a flat structure.
This video explains using the Flatten SMT when streaming nested data to a database (MySQL in this case) with the JDBC Sink connector.
Kafka Connect is the primary method for doing integration with Apache Kafka and other systems. It includes Single Message Transform (SMT) which can be used to modify messages as they pass through the pipeline in or out of Kafka.
---
⏱️ Time codes:
00:00:00 Brief introduction
00:00:49 Creating nested data into Kafka using ksqlDB
00:03:14 Examining the nested data
00:04:14 What happens if you stream nested data to a database with the JDBC sink connector?
00:06:01 Using the Flatten SMT
00:07:59 Using the Kafka message key as the primary key with the JDBC sink connector
---
✍️Blog: rmoff.net/2020/12/10/twelve-d...
👾 Demo code: github.com/confluentinc/demo-...
🗒️ SMT reference: docs.confluent.io/platform/cu...
Learn more about Kafka Connect here:
🏃♂️ Quick: • Kafka Connect in 60 se...
🚶 More detail: rmoff.dev/kafka-connect-zero-...
---
☁️ Confluent Cloud: confluent.cloud/signup?...
💾Download Confluent Platform: www.confluent.io/download/?ut...
📺 Kafka Connect connector deep-dives: • Kafka Connect
✍️Kafka Connect documentation: docs.confluent.io/current/con...
🧩Confluent Hub: www.confluent.io/hub/?...
3 авг 2024