Тёмный

How Instacart Optimized Snowflake Costs by 50% 

Snowflake Developers
Подписаться 20 тыс.
Просмотров 9 тыс.
50% 1

Опубликовано:

 

11 сен 2024

Поделиться:

Ссылка:

Скачать:

Готовим ссылку...

Добавить в:

Мой плейлист
Посмотреть позже
Комментарии : 13   
@olgashevchenko8439
@olgashevchenko8439 День назад
Cool insights, thanks
@MattRomano88
@MattRomano88 Год назад
Are you able to post the link to the open source DBT package for monitoring costs at the query level? Specifically the slide that appears at 8:42
@snowflakedevelopers
@snowflakedevelopers Год назад
Hi @MattRomano88, thanks for your question! Yes, here's the link to the package: github.com/get-select/dbt-snowflake-monitoring
@MattRomano88
@MattRomano88 Год назад
@@snowflakedevelopers Awesome thanks for the quick response!
@colouredlaundry1165
@colouredlaundry1165 10 месяцев назад
where can we find the slide deck?
@cheapdbs
@cheapdbs 9 месяцев назад
Can you please provide some context on the 56% savings in 2022 as mentioned on the slide at 03:44? According to the Instacart S-1 filed with the SEC in August 2023, on page 280 it states that Instacart paid "approximately $51 million" to Snowflake in 2022. Does the 56% savings applied to this $51M overall cost of Snowflake in 2022, meaning otherwise it would have been $116M, or does the 56% savings applied to only a single or few warehouses, or somewhere in between? If the latter how many of the 200+ warehouses was the saving applied?
@nickakincilar3757
@nickakincilar3757 9 месяцев назад
I believe $51M was a multi-year contract where they could burn that amount in any yearly volume as they wanted (for example 17M, 17M, 17M OR 30M, 11M, 10M OR 25M, 15M, 11M & etc.). Fairly certain 56% savings is compared to their initial yearly run rate when they were just building pipelines & products as fast as they to meet the fast growing market due to covid with little time to optimize. Once they were able to scale fast, meet the market demand , capture marketshare & finally stabilize their workloads, they went back to existing pipelines and started optimizing them. Once they were done, they were able to achive 56% savings over their initial data products/pipelines which were deployed very quickly & w/o optimizations to meet the fast growing demand.
@snowflakedevelopers
@snowflakedevelopers 9 месяцев назад
Some additional context may be found in the blog as well: www.snowflake.com/blog/snowflake-and-instacart-the-facts/
@cheapdbs
@cheapdbs 9 месяцев назад
@@snowflakedevelopers Thank you! So then I must adjust this question with the updated information that the actual usage spend was $28 million in the 2022 calendar year. Does the 56% savings of this overall cost in 2022 mean that otherwise the spend would have been $63.6M without the optimization? Therefore the optimization has a huge impact in saving unnecessary costs and we should all have extra resources dedicated to tuning and workload management on every warehouse to keep on top of costs as a priority. Additionally, does the 227% cost impact without optimization represent a common issue that is reflected by the comment made at 00:15 that Snowflake "has a reputation of getting really expensive really fast"?
@cheapdbs
@cheapdbs 9 месяцев назад
@@snowflakedevelopers Actually the context you provided in the referenced blog provides the answer. According to the blog "the actual usage of Snowflake (Instacart’s expenses) was $28 million in the 2021 calendar year, $28 million in the 2022 calendar year" which means that the usage expense was flat from 2021 to 2022, therefore there was no reduction in overall usage costs from 2021 to 2022. So to have reduced the "overall Snowflake cost by 56% in 2022" would mean that the optimization prevented a potential cost of $63.6M (28 / (1 - 0.56)) or cost avoidance of $35.6M (63.6 * 0.56) in 2022 to arrive at an overall usage cost of $28M (63.6 - 35.6) in 2022 to keep actual usage costs flat from 2021 even though there was an expansion of use cases and workloads. Conversely it prevented a 227% cost increase in 2022 over 2021. Can you please confirm that this is how the reduction of overall Snowflake cost by 56% was measured or provide the accounting method on how the 56% reduction was measured?
@nini-vd3gy
@nini-vd3gy 10 месяцев назад
5 million active tables? how many of them contain duplicated data?
@kunalplaysgames
@kunalplaysgames Год назад
Can you post the links to "Useful References For the Query Profile" on 33:48?
@snowflakedevelopers
@snowflakedevelopers Год назад
Yes! Here they are: teej.ghost.io/understanding-the-snowflake-query-optimizer/ and medium.com/snowflake/introducing-the-snowflake-visual-table-clustering-explorer-6fbb66a15bd5. Thanks for your comment!
Далее
ФОКУС -СВЕТОФОР
00:32
Просмотров 211 тыс.
Новый уровень твоей сосиски
00:33
Как подписать? 😂 #shorts
00:10
Просмотров 536 тыс.
Вопрос Ребром - Булкин
59:32
Просмотров 998 тыс.
Cursor Is Beating VS Code (...by forking it)
18:00
Просмотров 65 тыс.
Snowflake Query Optimization
35:04
Просмотров 8 тыс.
How to Work with Snowflake Cloud Data Model
57:05
Просмотров 28 тыс.
Load Data Into Snowflake Faster
38:13
Просмотров 4,8 тыс.
ФОКУС -СВЕТОФОР
00:32
Просмотров 211 тыс.