Getting Started with AWS Managed Streaming for Kafka with in-depth service setup

Подписаться 27 тыс.

Просмотров 19 тыс.

50% 1

Amazon Managed Streaming for Apache Kafka (Amazon MSK) allows you to operate Apache Kafka-based applications on AWS.
We’ll look at the Amazon MSK service in this vidoe, which can capture, store, and analyze real-time streaming data.
Prerequisite:
---------------------------
Learn AWS VPC , Subnet , Route Table, Internet Gateway with Hands-on Demo
• Learn AWS VPC , Subnet...
VPC with Public and Private Subnets in-depth intuition with Hands On
• VPC with Public and Pr...
AWS VPC NAT Gateway In-depth intuition
• AWS VPC NAT Gateway In...
AWS VPC ENDPOINT in depth intuition & hands on
• AWS VPC ENDPOINT in de...
Using Amazon VPC Endpoints to Access DynamoDB in-depth
• Using Amazon VPC Endpo...
Steps for the Lab:
-------------------------------
github.com/SatadruMukherjee/D...
Govern how your clients interact with Apache Kafka using API Gateway
aws.amazon.com/blogs/big-data...
Check this playlist for more Data Engineering related videos:
• Demystifying Data Engi...
Apache Kafka form scratch
• Apache Kafka for Pytho...
Snowflake Complete Course from scratch with End-to-End Project with in-depth explanation--
doc.clickup.com/37466271/d/h/...
🙏🙏🙏🙏🙏🙏🙏🙏
YOU JUST NEED TO DO
3 THINGS to support my channel
LIKE
SHARE
&
SUBSCRIBE
TO MY RU-vid CHANNEL

Опубликовано:

9 янв 2023

Ссылка:

Скачать:

Готовим ссылку...

Добавить в:

Мой плейлист

Посмотреть позже

Комментарии : 33

@adityanjsg99 21 день назад

Was struggling with confluent. This video solved most of my issues.

@showmethemoney824

this is cool, let's say cluster broke and nothing is working after consumer consumed some amount of message. Now my question is how do the consumer will consume the message from the left message and not from the beginning after restart.

@mavensept

I keep getting confused in step 9 - is it MSK EC2 cluster or EC2 created for producer/consumer ? As in the beginning of step 9 - you mentioned its EC2 cluster where you install java and kafka. Then again you mention of creating topic in MSK EC2. Now moving to step 10 - why do start the producer from MSK EC2 ? If some one can clear the mist it will be useful for me to understand. Apologise my ignorance ..

@SadaMandal-hd3nf Год назад

how can How i can access msk kafka in my nodejs application ?I want to send message to msk from my nodejs application

@sreesaradhi5709 Год назад

Great video on MSK service setup, appreciate you sharing the knowledge with us.

@vipinsharma-jt3qt Год назад

Please Create Video on MSK with SASL/Scram authentication from public internet.Without using EC2 instance in AWS.

@shrikantpandey6401 Год назад

Your videos is great. Please do end to end kafka spark project on aws

@PoojaGupta-bi1kw

I'm new to MSK, I've a requirement that I should use MSK and my springboot producer app should be able to produce messages on MSK, what configurations do i need in my springboot app to connect with MSK?

@chenguo8767 Год назад

Thank you for the great lecture. Would you mind comparing the pros and cons between this approach and the other one in your another video using lambda to publish to MSK? which one is more scalable to handle streaming requests. Thank you!

@gersonmoralesmarin1666 Год назад

Thank you for such amazing video!! I have a question is it possible to propagate authentication from API Gateway to MSK via the REST proxy. Example by using Authorization HTTP header (e.g. --header 'Authorization: Basic <user:pwd>') and avoid keeping credentials in properties files? Is there a way to configure REST Proxy server with Authorization HTTP header maybe using Iam. I appreciate any help you can provide :)

@amitk6701 Год назад

Great video, thank you for posting!

@felipeozoski Год назад

Great video, thank you so much for thorough explanation :)

@alfbaez Год назад

Brilliant! 3 Things done! Brilliant knowledge sharing!

@jagdishbelapure7521

awesome content, thank you so much for such a great explanation!

@user-me2ed5jt4g

Thanks for the video. i have a question. to consume the data, i just need to use Get method?

@user-gy5wc2oz6y

thank bro🥰

@akshaychampavat6795

Its really great explanation .

@jeffdeng1652 Год назад

Great Video - Can you comment on or make a video the differences of when to use MSK vs Eventbridge/SNS/SQS?

@JesusJimenez-xs5fe Год назад

Excellent video, but now I have one question left, if I wanted to activate public access to the cluster it would have to be from IAM?, the idea is to be able to connect through the brokers

@Data_Avenger

Thanks a lot again Satadru for this nice explanation, however i could able to implement upto publishing and consuming messages from ssh, but when i am trying to run