Using ksqlDB you can reserialise data in Apache Kafka topics. For example, you can take a stream of CSV data and write it to a new topic in Avro. ksqlDB supports many serialisation formats including Avro, Protobuf, JSON Schema, JSON, and Delimited (CSV, TSV, etc).
ksqlDB uses SQL to describe the stream processing that you want to do. For example:
CREATE STREAM ORDERS_CSV WITH (VALUE_FORMAT='DELIMITED',
KAFKA_TOPIC='orders_csv') AS
SELECT * FROM ORDERS;
-----
💾 Run ksqlDB yourself: ksqldb.io?.devx_ch.rmoff_youtube_sLAztA-rt74&
☁️ Use ksqlDB as a managed service: www.confluent.io/confluent-cl...
👾 Demo code: github.com/confluentinc/demo-...
🤔 Questions? Join the Confluent Community at confluent.io/community/ask-th...
-----
⏱ Time codes
00:00:00 Introduction
00:00:35 Printing the contents of Avro messages using ksqlDB
00:01:35 Listing streams in ksqlDB
00:02:13 Creating a new stream in ksqlDB from existing Kafka data
00:02:56 Converting Avro data in Kafka to CSV in ksqlDB
00:04:10 List of streams showing the different serialisation formats
00:04:19 Examining the reserialised topic
00:04:41 Converting Avro messages in Apache Kafka to JSON format with ksqlDB
00:04:59 Examining JSON topic data
00:05:15 Why you should think twice before using CSV in your output topics
00:05:49 The benefits of reserialising CSV data in Kafka to Avro (adding a schema to the data)
00:06:24 How to apply a schema to CSV data in Kafka and serialising it to Avro
00:06:48 Printing CSV messages from Kafka
00:07:25 Declaring a schema on CSV data in a Kafka topic in ksqlDB
00:09:37 Viewing a stream's schema
00:10:16 Why does my ksqlDB SELECT query not return data
00:10:47 Inspecting the ksqlDB server log
00:12:54 Changing the schema definition of a ksqlDB stream
00:13:42 A little rant about CSV
00:14:52 Converting CSV messages to Avro - recap
3 авг 2024
