10 frequently asked questions on spark | Spark FAQ | 10 things to know about Spark

Подписаться 10 тыс.

Просмотров 21 тыс.

50% 1

This video talks about the most frequently asked question on spark. The terms that should be known by each developer and Architect who works on Spark
Spark architecture in 3 mins
• Spark Architecture in ...
Spark Memory Management
• Spark Memory Management
Spark performance optimization Part1
• Spark performance opti...
Spark performance optimization Part2
• Spark performance opti...
Spark Date Timestamp datatypes
• Spark Date Timestamp d...

Наука

Опубликовано:

15 июл 2024

Ссылка:

Скачать:

Готовим ссылку...

Добавить в:

Мой плейлист

Посмотреть позже

Комментарии : 35

@vaibhavjoshi6853 19 дней назад

Getting confidence in spark because of you only. Thanks so so much!

@BigDataThoughts 14 дней назад

Thanks

@nikhilgupta110 2 года назад

I'm impressed , no copy paste , Covered whole architecture and done full justice to the title. Great work ,you are heading in the right direction, keep going :).

@BigDataThoughts 2 года назад

Thanks Nikhil

@iwonazwierzynska4056 10 месяцев назад

The next great video

@angelangeli2089 Год назад

good explanation and easy!! :)

@sachinragde Год назад

Thank you ma'am u explain it very nicely due to u I understand it And ur voice is very beautiful Thank you so much

@samirharris1 Год назад

Thankyou 😊

@harshmohan8419 2 года назад

Mam you are awesome.. teaching talent with example is like god gifted us..... Mam, can u briefly tell me if we use yarn, then what would be the role of the cluster manager and its application master in spark.. because driver directly communicating to executor

@rajeshd3940 2 года назад

quite formative for beginners and Appreciate the efforts made

@BigDataThoughts 2 года назад

Thanks rajesh

@dhavaldalasaniya Год назад

Very nice !! Simple & very knowledgeable 👍🏻

@BigDataThoughts Год назад

Thanks dhaval

@sathwikemmadi4428 2 года назад

Great content shreya ,I really like all the videos and wanted to learn more about datalakes and best practices designing datalakes.If you have some time and bandwidth please keep a post on the above. Thanks for all your hardwork.

@BigDataThoughts 2 года назад

Thanks sathwik. Will surely do. Have posted one on what are data lakes ru-vid.com/video/%D0%B2%D0%B8%D0%B4%D0%B5%D0%BE-wHG0ljN3plg.html Will post on best practices too.

@BigDataThoughts 2 года назад

This week have posted on data lake best practices

@ramkumarananthapalli7151 2 года назад

Thanks for making this video. Kindly make more videos on big data engineering.

@BigDataThoughts 2 года назад

Thanks Ramkumar surely

@geetakadam6728 Год назад

Hello Mam I had gone through your video the way you have delivered is very nice. I have one dought though silly but I wants to know in local system that means on our laptop when we execute pyspark code who is executor and who is worker and on cloud please explain me executor and worker? Thanks in advance

@SagarSingh-ie8tx Год назад

Nice

@BigDataThoughts Год назад

Thanks sagar

@khajaabdulmujeebbasha6327 2 года назад

mam there is shuffle in coalese even it is considering as narrow transformation why?

@guptaashok121 2 года назад

Is it driver which reads all data and divide in partition and sends to executor?

@BigDataThoughts 2 года назад

Yes executors job is to just execute.

@guptaashok121 2 года назад

@@BigDataThoughts will it not create bottleneck..

@BigDataThoughts 2 года назад

@@guptaashok121 no it won't be cos actual data resides on executors, driver is just the coordinator. while assigning task principal of data locality is kept in mind.

@guptaashok121 2 года назад

@@BigDataThoughts that's what my doubt is, if driver is not reading the data then how it sends command to executor on which data to work on