Closures & Accumulators in Apache Spark | Session-7

Подписаться 14 тыс.

Просмотров 5 тыс.

50% 1

Hi Friends
As promised, Today our Apache Spark Series from A-Z, begins, and here is the first video, where I have explored the entire documentation, and discussed the basic foundation topics of Apache spark , and also many FAQs mentioned in spark documentation, which can be very helpful for spark interview perspective.
In this video I have discussed, about Closures and Accumulators in apache spark, I have explained the scenarios of where to use accumulators and where to use closures in spark, also where not to use closures. Accumulators are explained on Spark User interface for better understanding.
-Arpit
GKCodelabs

Опубликовано:

11 сен 2024

Ссылка:

Скачать:

Готовим ссылку...

Добавить в:

Мой плейлист

Посмотреть позже

Комментарии : 25

@chandramouliramakrishnan2810 4 года назад

Very nicely framed video. Easy to understand. Even with no knowledge on Scala was able to follow. Great going. The video of you presenting along with the content added was very nice to have a connect. One clarification though, please share diff between scala object and class.

@DineshKumar-ui3lk 4 года назад

This video really helps for me to understand of closures and accumulators in scala.You are explaining in a depth way that is very good .I am expecting from you to make more videos related to scala and python in spark

@ankurdas6021 4 года назад

Nice way of walking through the documentation in details.helps a lot..keep going

@mangeshchikane7430 4 года назад

Great...keep doing

@ronitdhingra4395 Год назад

good example and explanation !!

@sgrhr024 4 года назад

bro..i watched and liked all ur videos so far... it's a genuine and humble request to make a video on end to end real time project... any domain will do...it'll be very helpful for interview...and hearty thank you for making this videos... keep up the good work.. we need such teachers....gbu

@GKCodelabs 4 года назад

Thanks Sagar 😊. Yes i am planning to make e2e project video. i will start this series in parallel to spark sessions, in some time.

@sgrhr024 4 года назад

@@GKCodelabs Thanks for the reply bro... i'll wait for the video... feeling humbled that u considered my suggestion...frankly speaking app jitni bhi videos banoge utna accha hai hamare liye..very few are there who use such simple and to the point approach...

@naganathavanthianchellam3669 4 года назад

Very good detailed explanation of closures and accumulators

@vitty6656 4 года назад

awesome brother..can able to grasp the concepts quickly...explanations are point to point..superb.. please make videos on scala programming in depth covering all the concepts

@sagarsinghrajpoot6788 4 года назад

finished this one :) As always, very nice video

@Dyslexic_Neuron 4 года назад

why were there 8 tasks ? How do you know the number of tasks? Why did some tasks didnt contribute anything to the accumulator ?

@Dyslexic_Neuron 4 года назад

excellent sir :)

@dpatro7245 4 года назад

Super video sir

@oscargong9438 4 года назад

Thanks for your explanation.

@architsoni3669 3 года назад

Hi, very nice explanation, could you please let me know, if an accumulator is val, then how its value is getting changed by the Drivers ?

@GKCodelabs 3 года назад

Yes its a val, but what we get in this "val" is not a concrete value that we are defining by ourselves, rather we are actually calling a method "longAccumulator" (its a method, not a concrete value), which returns an object of LongAccumulator class, and this class finally has the .add or .value methods which update/increment the values and using "var" hence its is editable.. And once the increment is done from all executors, that final concrete value can not be changed, hope that answers your question.!

@architsoni3669 3 года назад

@@GKCodelabs yes that makes sense, thanks again :)