Amazing! It would be really cool if you could make a video on Apache Kafka streams in NodeJS with demo examples, I’ve noticed that there are no proper resources on this subject.
R. Rod. Aha you asked about the hardest piece in the puzzle that I cut from this video, checkout the full kafka video (end screen) I cover that in details
I have a question "Can the number of partitions changed during execution?" I did some research and share the answer here: 1. Number of partitions can be increased, but can not reduced. 2. When increasing number of partitions, Kafka servers are kept running. No code shut down time. 3. Rebalancing traffic is handled by Kafka, producer will send messages by the new partition and consumer will consume messages from new partitions too. Client does not need to handle the rebalancing.
Yeah. The idiot jobs that REQUIRE a degree... they're idiots and you don't want to work for fools. Smart people learn from any/every thing possible, and better than what schools offer.
Just want to clarify: at 12:27, you said: "One partition'd better be consumed by 1 consumer" This only applies to consumers within the same consumer group, correct? Let's say we have: - 2 consumer groups G1 and G2. - G1 has consumer C1 and G2 has consumer C2 Can a partition P be consumed by both C1 and C2 since those consumers are from different consumer groups?
8:05 Absolutely love that Jordan Peterson reference about how life is suffering. That was awesome! And likely Jordan would add, the antidote to that is voluntarily confronting our own suffering will begin to cure it. Thank you for this Kafka educational video! I very much appreciate your teaching style! =)
man , u are awesome. i just wanted to know what Kafka is as it was mentioned in some topics related to my job and found ur video, loved ur explanation and simplicity. well explained bro.... keep it up man.
"Kafka guarantees that messages from a single partition are delivered to a consumer in order. However, there is no guarantee on the ordering of messages coming from different partitions." - from the kafka's published paper
at 8:05 why do subscriber need to know which partition it is publishing the message to? The broker can itself figure out based on the rule of partitioning. And due to some case if it is not possible then do I need to change all my publish requests if add a partitions to an existing topic? That is a very bad design i think.
I think they made this flexible by allowing the publishing decide which partition, otherwise how would they configure the broker to decide which partition? But i agree with you it has its cost
When a simple request/response system isn’t cutting it. If the cost of calculating response is high OR if the same response is required by multiple requester services. having a pub/sub system helps in those cases (kafka is just one example) I give some use cases in RU-vid here ru-vid.com/video/%D0%B2%D0%B8%D0%B4%D0%B5%D0%BE-O1PgqUqZKTA.html Thanks for being long time subscriber and asking great questions over the years Vrunda.
@@hnasr I have built a game. It uses subscription provided by Firestore. game.cvfy.in - Will Kafka be useful in replacing Firestore? The game is if Bingo. So, random numbers between 1 and 90 are shown to players and they claim dividends. So, it's all players are on subscription to one document where updates happen.
