Тёмный
No video :(

Delta Lake: Optimizing Merge 

Databricks
Подписаться 114 тыс.
Просмотров 14 тыс.
50% 1

Опубликовано:

 

21 авг 2024

Поделиться:

Ссылка:

Скачать:

Готовим ссылку...

Добавить в:

Мой плейлист
Посмотреть позже
Комментарии : 12   
@jacek_laskowski
@jacek_laskowski Год назад
The pace, tone and fairly detailed slides made this talk so pleasant to tune in. Thanks Justin! 👏👏👏
@guambomber448
@guambomber448 7 месяцев назад
I don't understand how pruning on the left has any effect because the left is the source of the distinct dates in the first place
@schallereqo
@schallereqo 3 года назад
Great talk Justin! Learned a lot of new things
@Universal_MisiQ
@Universal_MisiQ 2 года назад
Very informative Justin
3 года назад
Gracias Justin , esta explicación está excelente
@titowoche
@titowoche 3 года назад
Great talk
@harshikamahesh9459
@harshikamahesh9459 10 месяцев назад
Just because u get a separate bucket for atable doesn’t mean that the bucket will be in its own partition. S3 will scale
@alessiocesaretti3614
@alessiocesaretti3614 Год назад
Hello, I was trying to apply your suggestion about partition pruning on the existing table, by using the distinct values coming from the partitioning column of the incoming table to be merged, I wanted to do this dinamically but I found a corner case: in case I have an historization table that I'd like to update, if one new record has a new date (i.e. 2024), and the old version of that record was created in 2023, my merge condition "hist.BK_id == incoming.BK_id AND partition_year in (2024)" wouldn't allow me to update (flag is_current = False) the old record... I end up having duplicates and I cannot figure out an efficient way to include the partition without doing another expensive lookup. Do you have any suggestion for this use case?
@sushantpachipulusu8646
@sushantpachipulusu8646 2 года назад
Can you please share the deck or the KB articles shared in the slides?
@vdsg
@vdsg Год назад
Is the slide deck available as a PDF somewhere?
@ritjarijari5822
@ritjarijari5822 3 года назад
2:32 I like that😍💋 💝💖❤️
Далее
Eliminating Shuffles in Delete Update, and Merge
32:01
Просмотров 4,4 тыс.
AI-Accelerated Delta Tables: Faster, Easier, Cheaper
39:13
МЕГА МЕЛКОВЫЙ СЕКРЕТ
00:46
Просмотров 174 тыс.
Simple Flower Syrup @SpicyMoustache
00:32
Просмотров 642 тыс.
Accelerating Data Ingestion with Databricks Autoloader
59:25
Diving into Delta Lake 2.0
29:37
Просмотров 4,5 тыс.
Advancing Spark - Understanding Low Shuffle Merge
18:51
Delta Lake 2.0 Overview
37:56
Просмотров 11 тыс.
Advancing Spark - Databricks Delta Change Feed
17:01
Просмотров 14 тыс.
МЕГА МЕЛКОВЫЙ СЕКРЕТ
00:46
Просмотров 174 тыс.