22. How to select Worker/Driver type in Databricks? 

1. Usually, drivers can be much smaller than the worker nodes.
2. More cores for your DBUs, is more parallelism per DBU (but on smaller partitions because of less memory/CPU)
3. Allows for data to be split into bigger == fewer partitions to allow less shuffling.
4. Storage Optimized is especially relevant for queries with many joins/groupBy’s.
5. Choose Memory Optimized for less intensive ETL tasks.
6. Use Compute Optimized for more Parallelism and iterative tasks.
7. Do not choose HDD Clusters as they are less performant.
Connect with me on Linkedin
/ bhawna-bedi-540398102
Instagram
www.instagram....
Data-bricks hands on tutorials
• Databricks hands on tu...
Azure Event Hubs
• Azure Event Hubs
Azure Data Factory Interview Question
• Azure Data Factory Int...
SQL leet code Questions
• SQL Interview Question...
Azure Synapse tutorials
• Azure Synapse Analytic...
Azure Event Grid
• Event Grid
Azure Data factory CI-CD
• CI-CD in Azure Data Fa...
Azure Basics
• Azure Basics
Data Bricks interview questions
• DataBricks Interview Q...