If anybody is looking for performant solution with caching max value (not applying it every time we encounter "max counter" operation - it's very time consuming) there you have it: vector<int> solution(int N, vector<int> &A) { std::vector<int> counters(N, 0); int max_val = 0; bool apply_max = false; for (int operation : A) { if (operation <= N) { int& cur_val = counters[operation - 1]; cur_val++; if (cur_val > max_val) { max_val = cur_val; apply_max = true; } } else { if (apply_max) { std::fill(counters.begin(), counters.end(), max_val); apply_max = false; } } } return counters; }
Idk how many videos I went over but not a single one would say that partition in Kafka is JUST A QUEUE. They would unnecessarily make it complicated with more technical jargon. Thank You good Sir🧔♂👋🎩
I love this animation style and want to use one similar for a course's extra credit project! What software did you use to create these if you don't mind me asking?
Yes. But the same is true with a "real life" shuffle. If you shuffle a deck of cards, any cards could be in the same position after the shuffle. Indeed, to be a true random shuffle, this needs to be the case. If items were guaranteed *not* to be in the same position, that provides information about the previous state, which means it can't be truly random.
sub: Apache Kafka is the answer to the problems faced by the distribution and the scaling of messaging systems. let me try to illustrate this by an example imagine we were to design a system that listens to various basketball game updates from various sources such updates might include game scoring participants and timing information it then displays the games status on various channels such as mobile devices and computer browsers in our architecture we have a process that reads these updates and writes them in a queue we call this process a producer since it's producing these updates onto the queue at the head of this queue a number of downstream processes consume these updates to display them on the various channels we call these processes consumers over time we decide to expand and start following more and more games the problem is that our servers are now struggling to handle the load this is mainly because the queue is hosted on one server which is running out of memory and processing capacity our consumers are also struggling in a similar fashion so now we start thinking of how we can add more computing power by distributing our architecture but how do we go about distributing AQ data structure by its nature the items in a queue follow a specific ordering we could try to randomly distribute the contents of the queue onto multiple ones if we do this our consumers might consume the updates in the wrong order this would result in inconsistencies for example the wrong scoring being displayed across the channels one solution is to let the application specify the way to distribute the items in the queue in our example we could distribute the items using the match name meaning that the updates coming from the same match would be on the same queue this strategy would maintain an ordering per basketball match this is the basic fundamental difference of Kafka from other messaging systems that is item sent and received Kafka require a distribution strategy let's have a look at some more detail and terminology used in Kafka each one of these queues is called the partition and the total number of partitions is called a partition count each server holding one or more of these partitions is called a broker and each item in a partition is called a record the field used decide which partition the record should be stored in it's called the partition key it's up to the application to decide which field to use as the partition key if no key is specified Kafka simply assigns a random partition a grouping of partitions handling the same type of data is called a topic in order to identify each record uniquely Kafka provides a sequential number to each record this is called an offset essentially a recording that topic is identified by a partition number and an offset in our application since we have now distributed our data in the topic using the name as the partition key we can now also parallelize our consumer applications having one consumer per partition guarantees ordering per game consumers can live on one machine or distributed amongst multiple ones one important concept in Kafka is that consumers are very lightweight and we can create many of them without affecting performance this is mainly because Kafka only needs to maintain the latest offsets read by each consumer typically consumers read one record after the other and resume from where they left after a restart however in Kafka it's up to the consumer implementation to decide on how to consume records it's quite common to have consumers to read all the records from the beginning on startup or to read the record in different orders such as reading back to front for example in Kafka each consumer belonging to the same consumer group do not share partitions this means that each consumer would read different records from the other consumers multiple consumer groups are useful when you have different applications reading the same contents in our example we could have a consumer group called mobile and another consumer group called computer these groups will read the same records but update different channels each consumer in these groups will have separate offset pointers to keep track which latest record was read if consumers can read using custom ordering how can Kafka determine that the record has been consumed and it can safely delete that record so it can free up space the answer is that tough comp provides various policies that allow it to do a record cleanup for example using irritation policy you can provide a record age limits say 24 hours after which the records are automatically deleted using this policy if your consumer application is never down for more than this age limit no messages are lost another capability of Kafka is to store records in a fault tolerant and durable way each record is stored on persistent storage so that if a broker goes down it can recover when it comes back up additionally Kafka replicates partitions so that when a broker goes down a backup partition takes over and processing can resume this replication is configured using a replication factor for example a replication factor of three leads to three copies of a partition one leader and two backups this means that we can tolerate up to two brokers going down at the same time Kafka can be a solution to your scalability and redundancy problems if the problem is well stated and the technologies are well understood there are of course a lot more technical and implementation details which can be found on kafkas documentation I hope that this short video has been helpful at providing an introduction but the fundamental concepts in Kafka if you like it please give it a thumbs up and subscribe.
