No video :(

20. Runtime Architecture of Spark In Databricks

Подписаться 19 тыс.

Просмотров 12 тыс.

50% 1

Follow me on Linkedin
/ bhawna-bedi-540398102
Instagram
www.instagram....
Data-bricks hands-on tutorials
• Databricks hands on tu...
Azure Event Hubs
• Azure Event Hubs
Azure Data Factory Interview Question
• Azure Data Factory Int...
SQL leet code Questions
• SQL Interview Question...
Azure Synapse tutorials
• Azure Synapse Analytic...
Azure Event Grid
• Event Grid
Azure Data Factory CI-CD
• CI-CD in Azure Data Fa...
Azure Basics
• Azure Basics
Data Bricks interview questions
• DataBricks Interview Q...

Опубликовано:

22 авг 2024

Ссылка:

Скачать:

Готовим ссылку...

Добавить в:

Мой плейлист

Посмотреть позже

Комментарии : 21

@Dinesh-g1o Месяц назад

Great explanation, Thank you

@shrutikansal9831 5 месяцев назад

You are doing Amazing, really appreciate your teaching skills and knowledge. Keep it up.

@ashutoshdeshpande3525 Год назад

Very nice explanation. Was getting confused between stages and task part but now cleared. Thanks for this 😊.

@Prashanth-yj6qx 7 месяцев назад

your teaching skills are amazing.

@ramaraju3273 2 года назад

Very informative video , please continue to upload more videos on Databricks... Thank you .

@Ravi_Teja_Padala_tAlKs 10 месяцев назад

Super after lot of confusions Thanks 😊

@lifewithtarun Год назад

Thanks Ma'am for explaining.

@MohammedKhan-np7dn 2 года назад

Nice Explanation Bhawana. Thank you!!

@nagabadsha 7 месяцев назад

well explained. Thanks

@deepikasoni7423 2 года назад

Thanks a lot... Very well explained.. please upload videos on optimization techniques in Databricks.

@cloudfitness 2 года назад

ru-vid.com/video/%D0%B2%D0%B8%D0%B4%D0%B5%D0%BE-a2ehHq3DJrw.html here i the link that might help you

@Dinesh-g1o Месяц назад

I have one question, I am using AWS EMR, in that cluster one worker node can have more than one executors,... in databricks is it a hard rule, that one worker node = one executor

@deepjyotimitra1340 2 года назад

very nice explanation

@ayushsrivastava6494 2 месяца назад

say I've a heavy parquet file lying in S3 and I want to bring that file (COPY INTO command) into databricks as a delta table. What would be the ideal worker and driver type in that case if I have no transformations at all while moving the data but the dataset is very huge.

@nagamanickam6604 9 месяцев назад

Great

@nagendraprasadreddy353 2 года назад

Super

@abhishek310195 Год назад

What happens if... 1. In a databricks cluster a worker node get's down,what happens to the data which resides on that worker node??? 2. Meanwhile in continuation to above scenario,if databricks spins a new worker node...what happens if a select query goes to that new node..which doesn't have data(as its newly added in place of other node which went down and had data previously), will this cause data inconsistency???

@billcates4048 Год назад

we use metastore for that purpose which contains every information about the storage of data like which partition is in which node ,so if it fails ,it automatically recovers as data would be replicated across nodes .

@shivamjha9720 2 года назад

Very in-depth explanation. Keep up the good work. But I have one doubt - where we are defining the partitions?? No of tasks = no. of partitions. From where does the number of partitions will come. Are we defining it somewhere?

@cloudfitness 2 года назад

We can define number of partitions in code and then choose the cluster configuration as per number of partitions you have set up in code(other factors are also taken into consideration while choosing cluster)...if you donot specify partitions in code spark in dbx will default create partitions for you, usually it's 200partitions with around 128mb size

@shivamjha9720 2 года назад

@@cloudfitness Thanks a ton !! You've gained a new subscriber. You can upload more videos pertaining to databricks, PySpark, SQL. It would be helpful