Check out my fundamentals of Database Engineering udemy course with over 20 hours of deep database systems lectures. Head to database.husseinnasser.com for a discount coupon. Link redirects to udemy with coupon applied.
I always got demotivated by the complexity I experienced when I tried to learn this on my own but this one is a gem. I know it might have taken a lot of learning and experience for you to explains this in a simple manner Nevertheless thank you for sharing your knowledge!
Great tutorial as always. Literally a crash course fit into about an hour. Has everything to get started with kafka, especially because of the hands on.
Sweeeeet... did not feel like a tutorial, but like a friendly chat between deeply connected friends, every word makes clear sense. Kept me engaged every second till the end, yes I made to the end. Well done Sire... 👏👏👏
i had to deal with Kafka in one of my project, so searched for some tutorials, As soon as I saw that you have made a video on this, i had a sense of relief that i dont have to worry about the resource atleast. Thanks for all your efforts man.
I just wanted to express my heartfelt appreciation for the amazing Kafka crash course, Your dedication to simplifying complex topics shines through, and I can't thank you enough for making Kafka so much easier to understand.
I didn't know you before this video, but after this video? I'm gonna watch everything you upload man. Every tutorial you're selling? I'm buying! Obviously I liked & subscribed.
Thanks, I feel like I learned everything I need to know about Apache Kafka from this video. Sure, there are more advanced concepts. But this is the essence.
I just watched your RabbitMQ crash course and at the end of the video you mention how Kafka better at some point, then I bumped into this great video. Very clear explanation, love your video!
Luqman S thanks Luqman! Each software its use cases, I think RabbitMQ Is trying to be more than it really is (a message queue) where as kafka was designed to be a pub/sub and a message queue. If your app needs a queue and a pub/sub (or maybe streaming) than Kafka . If you just need a queue RabbitMQ is the way to go and kafka will be an overkill I think.. I would even consider zeromq as a lightweight mq.. Good luck!
this is the only tutorial i found that go in the depth and details of a pro should know , and i can compare Kafka with other messaging system like JMS and MQ. Thanks so much for sharing👍👏🙏
Course Table of Content time codes: (EDIT you should also see youtube chapters) Download slides here payhip.com/b/Y0Cq Kafka Broker: 2:32 Kafka Producer: 5:39 Kafka Consumer : 7:11 Kafka Partitions : 8:34 Queue vs Pub/Sub 12:50 Consumer Group 16:47 ZooKeeper 23:06 Example 29:45 Kafka Pros & Cons 1:02:48 Summary: 1:17:00 Example time codes Spin up Zookeeper 31:37 SPin up Kafka 34:00 Connect to Kafka (40:30) Create Kafka Topic (45:30) Create a Producer (49:44) Create a Consumer (55:24) Final test (1:00:00)
bro your tone when explaining these videos are so good. Didn't even feel like someone was teacahing me. It was as if a friend was talking to me lol. Good job man. Your excitement is intoxicating. :)
I made it to the end. Excellent. I don’t remember which video previously I saw was so long MD not boring at all. In fact you made it so interesting. May Allah (God) guide and bless you.
Not only the content, but the way you say it, and your enthusiasm, is something else. You sir deserve all the best, because you are, with no doubt, in the better's shelf..
thank you, this video was a great intro to understand what kafka is and how it works for a complete noob :) here is a docker-compose.yml that starts your two ontainers in case anybody wants to use that to go through your examples. basically i just took your two lines and converted them to compose. version: '3' services: zookeeper: image: zookeeper restart: always ports: - "2181:2181" kafka: image: confluentinc/cp-kafka restart: always ports: - "9092:9092" environment: KAFKA_ZOOKEEPER_CONNECT: 'zookeeper:2181' KAFKA_ADVERTISED_LISTENERS: 'PLAINTEXT://localhost:9092' KAFKA_OFFSETS_TOPIC_REPLICATION_FACTOR: '1' another addon: create a copy of consumer.js and edit the group name.. start that in parallel with the two consumer.js processes which then demonstrates pub sub behavior. i write all this one and a half hour after not even knowing what the difference between a queue and pub sub is and what kafka and zookeeper are :) .. this just shows how good this tutorial was :)
Hussein, You are a legend!!! Any one can be a master if they follow to your conversations. I like the way yo have a full conversation with yourself - such a good dialogue asking and answering yourself
Made it to the enddddd ... I know how kafka works “at least before witching this tutorial “ but after this tutorial many things become clear to me ... Thanks a lot
Appreciate your efforts...like the way you clearly say what you know/understand & what you dont...else I see most people in Industry are faking the intelligence & understanding about any technology & they pretend it so perfectly like they themselves have developed that tech...while those who actually know/understand the tech properly are very few & rare... So always good to watch your videos to understand something in simple words rather mugging those funky & techie words.
@@hnasr One question though.. Shouldn`t the consumer allways get all messages in the topic (within its partition)? I can see that the property "fromBeginning" i set to true, but when i restart the consumer, it still waits for new mesages. It does not replay the old ones?
27:28 answer: Every broker on a Kafka cluster is also named a Bootstrap Server. All brokers have the metadata required for the clients (producer or consumer) to discover brokers. When a client connects to one of the brokers (which are already configured as bootstrap servers in the Kafka configuration) it makes a "metadata request". The response includes information about topics, partitions, leader brokers, etc. Once the client gets this info, then - in the case of a producer- it makes the write request directly to the leader broker for that specific partition.
Hi Hussein, I love your way of explaining complex topics, kudos for good work. For some time, I was also confused about how the producer sends to the leader only, then I found out this in Kafka docs "The producer sends data directly to the broker that is the leader for the partition without any intervening routing tier. To help the producer do this all Kafka nodes can answer a request for metadata about which servers are alive and where the leaders for the partitions of a topic are at any given time to allow the producer to appropriately direct its requests." So I think the producer query the info before writing.
@KuldeepYadava - Thanks for answering the question. On top of this I have question does the producer request for "give me the leader" every time before sending message or it keeps doing on certain interval.
Hey ! Good video that I enjoyed it a lot. I have heard of kafka and wondering what is for and how to use, and now everything is clear for me thanks to your simple explication. and I hope you will make more videos in the future about full stack developing.
Amazing explanation Hussein. Every single bit of your lecture is fully understandable. You made difficult thing easy with practical example. You rocked! Good Luck.
Answer to - 27:25 - The producer sends a Metadata request with a list of topics to one of the brokers in the broker-list you supplied when configuring the producer. The broker responds with a list of partitions in those topics and the leader for each partition. The producer caches this information and knows where to redirect its produce messages. In case of failure while producing, failed broker's data (topics and its partitions) dynamically linked to existing replica which is present on another broker via topic's replication and new leader's information is communicated to the client (producer).
Amazing tutorial! Really find the topic interesting and easy to understand broker, producer, consumer etc.. Thanks for coming up with a great video. Hope we will get to see more on this.
