Using ksqlDB you can manipulate a stream of data in Apache Kafka and write it to a new topic with transformations including:
* Remove/drop fields
* CAST datatypes
* Reformat timestamps from BIGINT epoch to human-readable strings
* Flatten nested objects (STRUCT)
-----
💾 Run ksqlDB yourself: ksqldb.io?.devx_ch.rmoff_7pH5KEQiYYo&
☁️ Use ksqlDB as a managed service: www.confluent.io/confluent-cl...
👾 Demo code: github.com/confluentinc/demo-...
🤔 Questions? Join the Confluent Community at confluent.io/community/ask-th...
-----
⏱ Time codes
00:00:00 Introduction
00:00:34 Listing topics on Kafka cluster with ksqlDB
00:00:50 Creating a stream in ksqlDB on an existing Kafka topic
00:01:14 Examine an object's schema with DESCRIBE
00:01:19 Timestamps in ksqlDB
00:02:20 Querying a stream in ksqlDB
00:02:48 Converting a BIGINT timestamp to human-readable string in ksqlDB
00:04:02 CREATE STREAM … AS SELECT
00:04:30 Dropping fields from a Kafka topic with ksqlDB
00:05:22 Read ten rows from ksqlDB stream
00:05:29 Consume ten rows from Kafka topic
00:05:37 What's the difference between a SELECT and a PRINT in ksqlDB?
00:06:06 Recap - reformating timestamps and dropping fields with ksqlDB
00:07:03 Nested data structures in ksqlDB
00:07:44 CREATE STREAM…AS SELECT to flatten a Kafka topic's payload
00:08:09 Aliasing field names with AS in ksqlDB
00:08:13 Flattening fields in ksqlDB
00:09:15 Inspecting stream processing applications with DESCRIBE EXTENDED in ksqlDB
00:09:51 Recap - flattening data structures with ksqlDB
3 авг 2024
