Catalyst Optimizer in Spark SQL| Logical Plan Vs Physical Plan

Подписаться 5 тыс.

Просмотров 6 тыс.

50% 1

Hi Friends,
In this video, I have explained Spark Catalyst Optimizer with some sample code. Explained the stages in Catalyst optimizer , unresolved logical plan, resolved logical plan , physical plan etc components with some sample code execution.
Please subscribe to my channel for more interesting learnings.

Наука

Опубликовано:

27 дек 2021

Ссылка:

Скачать:

Готовим ссылку...

Добавить в:

Мой плейлист

Посмотреть позже

Комментарии : 21

@venukumargadiparthy8233 3 месяца назад

Insightful, thanks for sharing..)

@gauravmishra7591 2 года назад

You are explaining it very beautifully. Really liked it.

@sravanalakshmipisupati6533 2 года назад

Thanks a lot 😊 👍

@dhanaraj5638 9 месяцев назад

Thank you for your video. Nice explanation and clearly understand the concepts. Much appreciated

@sravanalakshmipisupati6533 9 месяцев назад

Thanks a ton for the feedback.

@antonyjesu7698 2 года назад

Nice explanation with important topic.

@sravanalakshmipisupati6533 2 года назад

Thank you very much Antony.

@sravankumar1767 2 года назад

Simply Superb

@sravanalakshmipisupati6533 2 года назад

Thanks a lot Sravan.

@nagendraprasadreddy353 Год назад

Good explanation

@sravanalakshmipisupati6533 Год назад

Thanks a lot

@Khader_views 2 года назад

Good explanation sister.. In eclipse how can we see the catalyst optimizer works.. Resultdf. Explain()

@sravanalakshmipisupati6533 2 года назад

@Rocker - Thank you very much. Please checkout the video - studio.ru-vid.comDKLhGMAx0bg/edit

@gauravmishra7591 2 года назад

Please prepare videos for spark interview questions and answers

@sravanalakshmipisupati6533 2 года назад

There are some videos on this already, Gaurav. Please watch, I'll get back with more videos.

@gauravmishra7591 2 года назад

I am searching for that but not able to find

@sravanalakshmipisupati6533 2 года назад

@@gauravmishra7591 Could you please search here github.com/sravanapisupati/SampleDataSet/blob/main/RU-vid_videos_list

@antonyjesu7698 2 года назад

Had an interview question..what will happen if my spark code didn't have action command. According to me, spark will design upto codengen and will not take into job list. Because spark is lazy evaluate. But DAG is not lazy evaluate. Am I correct or not?

@sravanalakshmipisupati6533 2 года назад

For every action it will create a job, if no action then no spark job and no stage and apparently no DAG. You can try with a simple databricks notepad (by taking some example code like printschema()), you can clearly see that no jobs will be created if no action.

@antonyjesu7698 2 года назад

@@sravanalakshmipisupati6533 thanks for answering.