Тёмный

ksqlDB HOWTO: Joins 

Robin Moffatt
Подписаться 4,2 тыс.
Просмотров 2,8 тыс.
50% 1

Using ksqlDB you can enrich messages on a Kafka topic with reference data held in another topic. This could come from a database, message queue, producer API, etc.
ksqlDB uses SQL to describe the stream processing that you want to do.
With JOIN clause you can define relationships between streams and/or tables in ksqlDB (which are built on topics in Kafka)
For example:
CREATE STREAM ORDERS_ENRICHED AS
SELECT O.ORDERTIME AS ORDER_TIMESTAMP,
O.ORDERID,
I.MAKE,
O.ORDERUNITS,
O.ORDERUNITS * I.UNIT_COST AS TOTAL_ORDER_VALUE,
FROM ORDERS O
INNER JOIN ITEM_REFERENCE_01 I
ON O.ITEMID = I.ITEM_ID
PARTITION BY ORDERID;
-----
💾 Run ksqlDB yourself: ksqldb.io?.devx_ch.rmoff_youtube__0Ktp2eB-as&
☁️ Use ksqlDB as a managed service: www.confluent.io/confluent-cl...
👾 Demo code: github.com/confluentinc/demo-...
🤔 Questions? Join the Confluent Community at confluent.io/community/ask-th...
-----
⏱ Time codes
00:00:00 Introduction
00:00:29 Listing streams in ksqlDB
00:00:38 Querying the orders stream
00:00:58 Foreign keys in a stream
00:01:25 Storing reference data in a Kafka topic
00:02:12 The importance of message keys in topics used as tables
00:03:20 Declaring a table in ksqlDB over a topic holding reference data
00:03:40 Streams and tables in ksqlDB
00:04:21 Defining the primary key on a ksqlDB table
00:04:47 Examining a table in ksqlDB
00:05:14 Stream-table joins in ksqlDB
00:05:46 INNER JOIN vs LEFT OUTER JOIN
00:06:45 Sampling the results of the join
00:07:25 Writing the results of the join to a new Kafka topic
00:08:14 Inspecting the derived stream
00:08:30 Viewing the enriched data stream
00:08:49 NULLs in join results
00:09:18 Viewing the enriched topic
00:09:43 Summary

Наука

Опубликовано:

 

3 авг 2024

Поделиться:

Ссылка:

Скачать:

Готовим ссылку...

Добавить в:

Мой плейлист
Посмотреть позже
Комментарии : 4   
@mehrdadmasoumi2443
@mehrdadmasoumi2443 3 года назад
I want to join two tables or stream/table, but I got this error. Cannot repartition a TABLE source. If this is a join, make sure that the criteria uses the TABLE's key column MARKETER_ID instead of [ID]
@rmoff
@rmoff 3 года назад
Hi Mehrdad, the best place to ask this question is the ksqlDB category over at forum.confluent.io/
@mehrdadmasoumi2443
@mehrdadmasoumi2443 3 года назад
@@rmoff Okay, I put it there .tanks
@paulaan7035
@paulaan7035 3 года назад
its so bad when you guys acquired pipelinedB and stop maintenance afterward!
Далее
ksqlDB HOWTO: Schema Manipulation
10:56
Просмотров 1,6 тыс.
ksqlDB HOWTO: Stateful Aggregates
13:56
Просмотров 2,8 тыс.
ksqlDB HOWTO: Reserialising data in Apache Kafka
17:42
Просмотров 1,9 тыс.
ksqlDB HOWTO: Handling Time
9:08
Просмотров 2,4 тыс.
Data Pipelines: Join Data Streams with ksqlDB
9:39
Просмотров 6 тыс.
Why The Windows Phone Failed
24:08
Просмотров 128 тыс.
ksqlDB HOWTO: Split and Merge Kafka Topics
10:18
Просмотров 3,5 тыс.
joining streams and tables in ksqlDB
9:54
Просмотров 2,3 тыс.
ksqlDB HOWTO: Filtering
9:25
Просмотров 1,7 тыс.
iPhone socket cleaning #Fixit
0:30
Просмотров 18 млн