Building a Telegram bot with Apache Kafka, ksqlDB, and Go 

Imagine you’ve got a stream of data; it’s not “big data,” but it’s certainly a lot. Within the data, you’ve got some bits you’re interested in, and of those bits, you’d like to be able to query information about them at any point. Sounds fun, right? Since I mentioned “querying,” I’d hazard a guess that you’ve got in mind an additional datastore of some sort, whether relational or NoSQL.
But what if I told you...that you didn’t need any datastore other than Kafka itself? What if you could ingest, filter, enrich, aggregate, and query data with just Kafka? With ksqlDB we can do just this, and I want to show you exactly how.
In this hands-on talk we'll walk through an example of building a Telegram bot in which ksqlDB provides the key/value lookups driven by a materialised view on the stream of events in Kafka. We'll take a look at what ksqlDB is and its capabilities for processing data and driving applications, as well as integrating with other systems.
👾 Try it out for yourself: rmoff.dev/carparks
--
ℹ️ Table of contents
00:02:21 Telegram bot sample Golang code
00:05:07 Getting data from REST endpoint into Kafka
00:07:46 Processing CSV data with ksqlDB
00:09:34 Transforming data with ksqlDB
00:13:03 Building event-driven notifications on Kafka with ksqlDB
00:14:10 Creating a materialised view in ksqlDB
00:16:22 Using ksqlDB in a Telegram bot
00:18:51 Walking through the Telegram/ksqlDB bot Go code
00:20:41 Streaming data from ksqlDB to Elasticsearch
00:21:42 Analysing data in Elasticsearch with Kibana
00:24:35 Recap
00:25:09 Resources
--
🎓 Resources
👾 Try it out for yourself: rmoff.dev/carparks
🎥 Introduction to ksqlDB: rmoff.dev/ksqldb-introduction
🎥 Streaming data from Kafka to Elasticsearch: rmoff.dev/kafka-elasticsearch...
⚡ Golang ksqlDB client: github.com/rmoff/ksqldb-go
✍️ www.confluent.io/blog/buildin...
--
☁️ Confluent Cloud ☁️
Confluent Cloud is a managed Apache Kafka and Confluent Platform service. It scales to zero and lets you get started with Apache Kafka at the click of a mouse. You can signup at confluent.cloud/signup?... and use code 60DEVADV for $60 towards your bill (small print: www.confluent.io/confluent-cl...)

Наука