Data Warehousing With BigQuery: Best Practices (Cloud Next '19)

Подписаться 1,1 млн

Просмотров 145 тыс.

50% 1

Take an in-depth look at modern data warehousing using BigQuery and how to operate your data warehouse in the cloud. During this session, we'll give lessons learned and best practices from prior implementations to give you the playbook for implementing your own modern data warehouse.
"BigQuery ML → bit.ly/2Khkqoq
BigQuery for data warehouse practioners → bit.ly/2TWh6P9"
Watch more:
Next '19 Data Analytics Sessions here → bit.ly/Next19DataAnalytics
Next ‘19 All Sessions playlist → bit.ly/Next19AllSessions
Subscribe to the GCP Channel → bit.ly/GCloudPlatform
Speaker(s): Ryan McDowell, Alban Perillat-Merceroz
Session ID: DA307
product:BigQuery; fullname:Ryan McDowell; event: Google Cloud Next 2019; re_ty: Publish; product: Cloud - Data Analytics - BigQuery; fullname: Ryan McDowell;

Наука

Опубликовано:

26 июл 2024

Ссылка:

Скачать:

Готовим ссылку...

Добавить в:

Мой плейлист

Посмотреть позже

Комментарии : 21

@anandakumarsanthinathan4740 2 года назад

Beautifully explained. Many thanks. Now, why wouldn't anybody want to move to the sunny side of France and land a job at Teads !! Both of the presenters did an excellent job.

@kunfang2457 3 года назад

the best 'best practice' sharing session i ve ever had

@3rdinnings326 Год назад

Excellent session!

@ThisIsAli_Off Год назад

Very useful!

@jennwng 9 месяцев назад

At 13:36, why do we need to load into GCS first for batch loads? Can't we use Dafaflow/BigQuery BatchLoad / Data Fusion directly from GoldenGate?

@TheMitali_ 4 года назад

How to perform sum of columns value in big query. And no of columns are notfixed , at runtime we need to decide no of columns need to consider for sum. Depends on user inputs

@LuckyHongTJ 8 месяцев назад

At 13:35, what is the benefit of loading data from Oracle into GCS first and then BigQuery (i.e. won't directly loading into BigQuery from Oracle be faster)? I know GCS can serve as a staging area and we get another copy of the data for fault tolerance, but is there any other benefit? Thanks :)

@jennwng 9 месяцев назад

At 11:58 why DataFlow / Data Fusion can speed up pipelines? Like, the essence is either to do the transformation in BigQuery or in DataFusion. Even if the query is complex, won't it still be faster to do the transformation directly in BigQuery, rather than connecting to DataFusion and transforming data there?

@vishnureddys4801 4 месяца назад

It is because they can bill you in both Services, for using BigQuery and also dataflow.

@amieewright4417 3 года назад

H thanks u so much love you amiee Wright and Bella daws Xxxxx

@krishnabg1350 4 года назад

Can you share slides?

@mzamanmintu3694 Год назад

Congratulations

@Thiago280690 Год назад

6:30 do nada aparece uma nota de dez conto hahaha adorei

@Vinch157 5 лет назад

Hello, will you also share the slides? Thank you

@vishnureddys4801 4 месяца назад

Has he yet?

@LouisChiaki 3 года назад

How is this different from Spark?

@jean4j_ 2 года назад

Well it's auto-managed and much simpler to work with. That's just SQL. You don't need to fine-tune the configuration like you would for a Spark job I think.

@JOPINC 3 года назад

Dedicado a JMB!!!! :)

@sunilpipara 2 года назад

Kafka to S3 and S3 to Cassandra via Spark is wrong approach.

@anandakumarsanthinathan4740 2 года назад

Yes, @Sunil Jain. I feel the same too. The source application would directly write it out to S3 for batch processing. Kafka would be required only for the streaming jobs.