Absolutely premium content - as always. Thank you! This fragment: "They will be fighting among them...... Emmm... Not fighing because they all are gentlemen. They will be struggling for..." This one was funny ;-)
Can you shed some light on how we can make parallel for each to execute sequentially like for each . I have been seeing some of those claims in the community with transaction scope or something. Thank you
If I have a payload that is an array of 10.000 positions. The ideal collection should be 1 to 100 , 1 to 1000 ? I'm working on a flow that the developer set the collection as a payload. So I guess because of that they are having problems of performance. ( 10.000 roots in parallel ??? ) Making some tests I realize that setting the collection with 1 to 1000 or 1 to 100 is faster than using payload. Can you imagine why ? Thanks!
Hello sir, for me your videos are very helpful. I'm trying to know more about vcore and worker. But I'm not able to find good explanation, could you please do a video on vcore and worker.
Thank you very much for detailed explanation. I have 2 questions. 1. In all the formula you used number of cores for calculation, I have application on Cloudhub with 0.2 vCore with 1 worker node. In the logs its showing me following config. * Pooling strategy: UBER * * gracefulShutdownTimeout: 15000 ms * * uber.threadPool.maxSize: 117 * * uber.threadPool.threadKeepAlive: 30000 ms Can I use Max Concurrency for Parallel for each as 10 (more than number of vCore) ? 2. Lets say I have 100 records in payload for "Parallel for each pushing synchronous HTTP request" with max concurrency as 5, Can I assume that any moment of time, there will be maximum 5 connection ?
Sorry for the delayed reply, its a good question, hope the answer would helps you or someone having similar doubt. 1. You can use 10 which is more than the number of core, but in that case the excess number of threads (more than the core count) will be waiting for their turn to get the CPU time, which might result into degraded performance. The optimum concurreny as explained in the video can be attained by having a value same or close to the number of available cores. Coming to the available threads, it represents that we have 117 threads at our disposal which means we can have 117 tasks ready to be handled by thread but they won't get the CPU time together as the number of cores are less, the threads will be waiting for their turn. 2. Yes, you can assume a maximum of 5 connections.
Reliability, state management and fine grained error handling is the key difference between Batch and For loop (Sequential and Parallel). In case of For loops the only difference is of processing, one does sequential other is parallel. Use batch for heavy lifting process like data synchronizations, or schedules that involve huge records and complex processing, the batch steps give you fine grained control on errors and happy path and also makes code more readable.
Hello Vishwas, 4:51 i see the console output in sequential manner but i got in output as random(parallel processing) order, could you please clarify this.
0.1v and 0.2v cores have only 2 available cores which means you can attain real parallelism of 2, if you exceed this limit there will be context switching. Multiple worker will have no effect as both will be two different VM and there's no resource sharing. Regarding calculation of max thread, please watch the video again I've explained the formula.
How can I determine the number of Cores of an environment? If I debug local pointing to the dev environment is it sufficient to determine or should I evaluate that in the cloudHub? ( in the cloud I dont know how to determine that ) Thank you!
To find the number of available cores in a Cloudhub worker, you can write a java code in Mule app that would return the number of cores and deploy it to Cloudhub by selecting the worker size of your choice. You can use the code mentioned here: stackoverflow.com/questions/4759570/finding-number-of-cores-in-java#:~:text=Finding%20the%20number%20of%20cores%20the%20Java%20program%20can%20use,concern%20to%20keep%20in%20mind.
Does thread switching occurs in CPU_Intense?/ because if CPU_intense requires 2mins of time to exceute, y again thread switching occurs for next thread?? i mean, on what basis other thread comes in to use this intense thread while other is in processing state
First lets understand the orders of threads BLOCKING_IO > CPU_INTENSIVE > CPU_LITE, the BLOCKING_IO thread pool is the most expensive, now if the flow starts with Logger which uses CPU_LITE and the next processor is Transform then obviously a thread switch has to be done because CPU_LITE < CPU_INTENSIVE, so a thread from CPU_INTENSIVE will be used now again after Transform if you have logger so here CPU_INTENSIVE thread can easily to that task there is no point in switiching back to CPU_LITE, I hope this explains the idea of thread switching and the order of thread pools Now coming to your question of switching from CPU_INTENSIVE, obviously there can be situations where after a Transform component that uses CPU_INTENSIVE, there is another processor like DB_INSERT so this is an IO operation which is a blocking task so here CPU_INTENSIVE cannot do that task so it moves to BLOCKING_IO thread pool. All the processors belong to some thread pool by default but that doesn't mean they'll use that default thread pool, CPU_LIT tasks can be done by CPU_INTENSIVE/ BLOCKING_IO pools as well, that depends on the situation. I've explained all of these concepts in detail in thread pool video, please consider watching it. ru-vid.com/video/%D0%B2%D0%B8%D0%B4%D0%B5%D0%BE-YRTLPxRnC9w.html
