Тёмный

🎄Twelve Days of SMT 🎄 - Day 3: Flatten 

Robin Moffatt
Подписаться 4,2 тыс.
Просмотров 1,6 тыс.
50% 1

The `Flatten` Single Message Transform (SMT) is useful when you need to collapse a nested message down to a flat structure.
This video explains using the Flatten SMT when streaming nested data to a database (MySQL in this case) with the JDBC Sink connector.
Kafka Connect is the primary method for doing integration with Apache Kafka and other systems. It includes Single Message Transform (SMT) which can be used to modify messages as they pass through the pipeline in or out of Kafka.
---
⏱️ Time codes:
00:00:00 Brief introduction
00:00:49 Creating nested data into Kafka using ksqlDB
00:03:14 Examining the nested data
00:04:14 What happens if you stream nested data to a database with the JDBC sink connector?
00:06:01 Using the Flatten SMT
00:07:59 Using the Kafka message key as the primary key with the JDBC sink connector
---
✍️Blog: rmoff.net/2020/12/10/twelve-d...
👾 Demo code: github.com/confluentinc/demo-...
🗒️ SMT reference: docs.confluent.io/platform/cu...
Learn more about Kafka Connect here:
🏃‍♂️ Quick: • Kafka Connect in 60 se...
🚶 More detail: rmoff.dev/kafka-connect-zero-...
---
☁️ Confluent Cloud: confluent.cloud/signup?...
💾Download Confluent Platform: www.confluent.io/download/?ut...
📺 Kafka Connect connector deep-dives: • Kafka Connect
✍️Kafka Connect documentation: docs.confluent.io/current/con...
🧩Confluent Hub: www.confluent.io/hub/?...

Наука

Опубликовано:

 

3 авг 2024

Поделиться:

Ссылка:

Скачать:

Готовим ссылку...

Добавить в:

Мой плейлист
Посмотреть позже
Комментарии : 3   
@tomaszwegrzyn2636
@tomaszwegrzyn2636 3 года назад
those videos are really helpful, thanks!
@rmoff
@rmoff 3 года назад
Glad you like them!
@shibimukesh9369
@shibimukesh9369 3 месяца назад
Using mongodb sink connector tried to flatten message but iam getting error
Далее
🎄Twelve Days of SMT 🎄- Day 4: RegexRouter
10:33
Просмотров 1,3 тыс.
🎄Twelve Days of SMT 🎄 - Day 8: TimestampConverter
19:14
Советы на всё лето 4 @postworkllc
00:23
ksqlDB 101: Flatten Nested Records with ksqlDB
1:06
Просмотров 2,8 тыс.
ksqlDB HOWTO: Stateful Aggregates
13:56
Просмотров 2,8 тыс.
What is Database Sharding?
9:05
Просмотров 46 тыс.
Nokia 3310 top
0:20
Просмотров 3,5 млн