Stream processing: the easy way (with NATS and Benthos)

Просмотров 4,3 тыс.

% 197

In this video, Jeremy shares how to create operationally simple, highly scalable streaming and data processing pipelines using Benthos.dev and NATS.io, and why they can be a simpler alternative to frameworks like Apache Spark, Kafka Streams, Apache Flink or Kafka Connect.
Source code for this video:
github.com/synadia-io/rethink_connectivity/tree/main/18-benthos-and-nats
00:00 - Intro
01:23 - Why NATS and Benthos?
05:53 - 4 Patterns for NATS and Benthos
06:52 - Bridging
14:02 - Enrichment
23:39 - Analytics Pipeline
33:01 - Backups/Tiered Storage
36:16 - Closing Remarks
Don't want to host NATS yourself? Sign up for Synadia Cloud for free: cloud.synadia.com/
NATS is a connective technology powering modern distributed systems, unifying Cloud, On-Premise, Edge, and IoT.
Join the NATS Community on Slack: slack.nats.io
Learn More about NATS at docs.nats.io/
#nats #kafka #benthos #flink #aws #streamprocessing

Наука

Опубликовано:

11 апр 2024

Ссылка:

Скачать:

Готовим ссылку...

Добавить в:

Мой плейлист

Посмотреть позже

Комментарии : 28

@SynadiaCommunications 3 месяца назад

What other kinds of examples would you like to see with Benthos and NATS? Sound off in the comments!

@dandogamer 3 месяца назад

Sliding windows e.g. calculate the average watch count of a video in the last 24 hours

@holgerwinkelmann6219 2 месяца назад

Maybe not directly a use case, but what about having a bit of performance expectation in given environment, just to manage Expectations vs Resource Consummations and scalability patterns?

@mikkaruru 2 месяца назад

Subscribe to websocket data, transform and pipe it to NATS

@chivesltd 2 месяца назад

usecases for Kafka with Benthos and NATS

@SureshBala66 18 дней назад

To set the context, Redpanda acquired Benthos and launched Redpanda Connect.

@GabrielPozo 11 дней назад

Great video, thank you for sharing!

@chris-malek 3 месяца назад

Great explanation of Benthos. I have been looking at it for a while but I was having issues understanding the concepts. thanks for the video.😀

@bvhdv 2 месяца назад

Love how you explain everything, well structured and clear. Love NATS !

@SynadiaCommunications 2 месяца назад

Glad you are a fan and much more to come!

@alejandrosuarez5298 3 месяца назад

Loving this kind of content. Thank you so much for it!

@Ekosz 3 месяца назад

This was great! Love Benthos and love love NATS!

@dandogamer 3 месяца назад

Love nats and I've used benthos in the past and want to love it but struggled to get my head round some of the yaml options so this tutorial really helped thanks. Would love to see more open source libraries shown with nats

@garybonner7499 2 месяца назад

Love Benthos such a powerful project

@joeblue2492 Месяц назад

Love these videos !!

@meny054 2 месяца назад

Great, I think that one of the best features of Benthos is the Benthos studio with the `benthos studio` command You can create the flow with web UI and it will updated on the spot Thank you for the video

@SynadiaCommunications 2 месяца назад

Yeah be this studio is awesome!

@nathanprobst2164 2 месяца назад

Amazing stuff! Is that Excalidraw Plus that you’re using for the presentation mode with laser pointer?

@SynadiaCommunications 2 месяца назад

Yeah I switched to excalidraw plus earlier this year and it's been awesome!

@appology9102 2 месяца назад

I just binge watched the playlist, and I'm feeling sad that there isn't another video...

@SynadiaCommunications 2 месяца назад

That’s a ton of content to watch! We will be posting more soon so be sure to subscribe

@garybonner7499 2 месяца назад

I would love an example of using multiple benthos instances with nats. Benthos does not seem to have clustering capabilities

@SynadiaCommunications 2 месяца назад

Since Benthos is stateless it doesn’t need to be clustered. If you are using NATS for your state then you can simply scale Benthos horizontally by adding more instances/nodes

@joeblue2492 Месяц назад

Would like to respond to a web page via benthos and htmx

@trejohnson7677 3 месяца назад

YAML gives u gitops ez but I would like to write it out in imperative coad and ship it in the same big ass binary I’m shipping the rest of my go code in.

@SynadiaCommunications 3 месяца назад

You can do that too! Check out Benthos’s Streams API. You can construct all these same things using Go

@trejohnson7677 3 месяца назад

@@SynadiaCommunications Oh wow, interesting, didn't uncover in my (very) cursory glance at the documentation. This will save a lot of time.