Тёмный

5 Ways to Improve Your SQL Queries 

Kahan Data Solutions
Подписаться 38 тыс.
Просмотров 88 тыс.
50% 1

Learn a few tips to improve performance and overall design of your SQL queries.
►► The Starter Guide for The Modern Data Stack (Free PDF)
Simplify the “modern” data stack + better understand common tools & components → bit.ly/starter-mds
Timestamps:
0:00 - Intro
0:27 - Tip #1
2:20 - Tip #2
3:52 - Tip #3
4:28 - Tip #4
8:19- Tip #5
Title & Tags:
5 Ways to Improve your SQL Queries | Performance Tuning and Design Approach
#kahandatasolutions #SQL #dataengineering

Опубликовано:

 

1 июн 2024

Поделиться:

Ссылка:

Скачать:

Готовим ссылку...

Добавить в:

Мой плейлист
Посмотреть позже
Комментарии : 46   
@KahanDataSolutions
@KahanDataSolutions 2 года назад
►► The Starter Guide for Modern Data → bit.ly/starter-mds Simplify “modern” architectures + better understand common tools & components --- After spending more time working with modern data stacks, I wanted to add an updated note to this video. Some of these tips are focused on writing SQL in the context of traditional row-based databases such as SQL Server, MySQL, etc. as opposed to columnar platforms such as Snowflake. The main update is in regards to point #1: Filtering a data set -- This rule is not _necessarily_ always still the case with some of the modern databases, such as Snowflake as the query plans have become really efficient and a "select *" is not always a no-no. The query processors can handle an insane amount of data without much of a performance hit (if written responsibly). Research around topic this is covered really well in this article - discourse.getdbt.com/t/ctes-are-passthroughs-some-research/155. I still suggest the filtered approach when working with any of the row-based databases mentioned above, and the other tips still hold true regardless of data stack. I mainly just wanted to add this caveat to point #1 as my viewpoint has evolved a bit as I've learned more!
@adnanghafoor3226
@adnanghafoor3226 2 года назад
#1 Use filter Data Set #2 Use temp tables/ CTEs #3 Don't repeat yourself (Dry) #4 Use indexes #5 Be consistent
@cdaugherty1870
@cdaugherty1870 2 года назад
Something that would add value here for me would be showing examples. I'm a little newer to SQL, so actually looking at examples would be helpful! This video may have been aimed at more advanced users, so that makes sense, but this is just my input! Thanks for taking the time to make this and listen to our feedback!
@KahanDataSolutions
@KahanDataSolutions 2 года назад
Great suggestion and thanks for watching! I will make sure to add examples in future videos. You may find this other video helpful as it does have some example SQL - ru-vid.com/video/%D0%B2%D0%B8%D0%B4%D0%B5%D0%BE-asCQj1ybnds.html
@SAURABHKUMAR-uk5gg
@SAURABHKUMAR-uk5gg Год назад
This is the best indexing explanation I have seen anywhere on the internet. Period!
@KahanDataSolutions
@KahanDataSolutions Год назад
Thanks!
@stopthink9000
@stopthink9000 Год назад
Sounds like great tips but as a beginner / intermediate it would be great to see some actual before / after code examples. It's easy to get lost without visuals.
@KahanDataSolutions
@KahanDataSolutions Год назад
I agree, check out this other video that adds some examples. ru-vid.com/video/%D0%B2%D0%B8%D0%B4%D0%B5%D0%BE-asCQj1ybnds.html Hope that helps!
@sheshkumarbhombore391
@sheshkumarbhombore391 2 года назад
This is just a theory! Wanted to see some examples :(
@KahanDataSolutions
@KahanDataSolutions 2 года назад
Check out this video - it has specific examples: ru-vid.com/video/%D0%B2%D0%B8%D0%B4%D0%B5%D0%BE-asCQj1ybnds.html
@baloney_sandwich
@baloney_sandwich 11 месяцев назад
He's reading from Google, he don't even know what sql is
@jonspell88
@jonspell88 3 года назад
Thanks for this! I also like to put indexes on my temp tables if they get a little bigger (combining tips 2 and 4) And lots of comments!
@KahanDataSolutions
@KahanDataSolutions 3 года назад
I agree on both of those as well!
@marcinbadtke
@marcinbadtke 4 месяца назад
Than you for the video. I would like to add one thing: use partitions. Better than indexes for sets of identical values.
@christheone248
@christheone248 Год назад
#1, So valid, it's not even funny! I work on a data warehouse and let me tell you, this is rule number one for a reason!
@Cog-pd3qn
@Cog-pd3qn 2 года назад
I am newer to SQL, but for #1, wouldn't the query optimizer first pare down your data set by any WHERE clause found, regardless of whether that clause appears at the top to filter rows into a temp table, or down a ways in a longer query? Or -- is the optimizer really that useless?
@KahanDataSolutions
@KahanDataSolutions 2 года назад
This is likely accurate when working w/ CTEs (common table expressions) vs temp tables in the more modern cloud databases (Snowflake, Redshift, etc). CTEs will function as a pass-through and the optimizer would likely function in the way you describe. This is a common strategy that you see in "best practices" on tools such as dbt that rely heavily on CTEs. But again, this is mainly based on modern analytical cloud databases vs traditional row-based ones (sql server, mysql, postgres, etc.). Here is a great write-up on this topic - discourse.getdbt.com/t/ctes-are-passthroughs-some-research/155 Temp tables on the other hand are built as separate table objects in your database and may not necessarily function as a pass-through in the same way. Truthfully I haven't personally tested that statement but at a first thought I don't think a WHERE clause outside of the temp table would be able to impact the result set that becomes the temp table object. But not to say it isn't possible - worth testing out.Try running some queries and take a look at the optimizer to see what results you get when you try different approaches. Thanks for your comment!
@emmanuelaolaiya
@emmanuelaolaiya 2 месяца назад
Many thanks for this great content
@krupaneshkrishnan9284
@krupaneshkrishnan9284 2 года назад
Informative.
@KahanDataSolutions
@KahanDataSolutions 2 года назад
Thanks!
@sqlservertutorialin3minute485
@sqlservertutorialin3minute485 2 года назад
Useful advice
@KahanDataSolutions
@KahanDataSolutions 2 года назад
Thank you
@adamscott8831
@adamscott8831 2 года назад
Excellent.
@KahanDataSolutions
@KahanDataSolutions 2 года назад
Thanks!
@rickhehe
@rickhehe Год назад
Thanks, very well said.
@KahanDataSolutions
@KahanDataSolutions Год назад
Thanks for watching
@hananmostafa2311
@hananmostafa2311 Год назад
Could you please explain how can I make a loop in Presto sql?
@alteshaus3149
@alteshaus3149 2 года назад
Nice, thx
@KahanDataSolutions
@KahanDataSolutions 2 года назад
Thanks for watching
@miguelvictoria4514
@miguelvictoria4514 2 года назад
Thanks :) went from more than 2 minutes query down to 750 ms average :D
@KahanDataSolutions
@KahanDataSolutions 2 года назад
Nice! What changes did you make that sped it up the most?
@miguelvictoria4514
@miguelvictoria4514 2 года назад
@@KahanDataSolutions We have a many subqueries for updating a day capacity. Having 7 days we ran it for each day and for each object (we have more than 1000) so we were doing the same complex subquery over 7 thousand times. I use a CTE and group all the results of that query by the day and the objectID and now I just need to select from that CTE where the day = myDay AND object= myObjectID :) that pretty much did it.
@notanenglishperson9865
@notanenglishperson9865 2 года назад
#3 Let's say I have a query with 2 exact same subqueries, is it possible to write a subquery once and use it again somehow?
@KahanDataSolutions
@KahanDataSolutions 2 года назад
Definitely! You could pull it out and put it in a CTE (common-table-expression) or a temporary table. Then join the CTE/Temp Table in the various places where it was being used as a sub-select. But now you have the actual query logic in just one place, not multiple.
@notanenglishperson9865
@notanenglishperson9865 2 года назад
@@KahanDataSolutions Thanks, but unfortunately I use Jet SQL so no WITH statement thing for me, haha
@bpruss2
@bpruss2 Год назад
Appendix or index in a book?
@baloney_sandwich
@baloney_sandwich 11 месяцев назад
So u read from someone else blog and made a video outta it
@onethousandyearsofcultivat78
I dont understand the point 2# use temp tables. Other advice i read says avoid temp tables like the plague. So what gives?
@KahanDataSolutions
@KahanDataSolutions Год назад
In my experience, properly used CTEs/Temp Tables can massively help a query and I recommend them. But the key here is "properly" using them. When used strategically, they help you stay more organized and can boost performance. When used blindly and without understanding what's happening you could end up creating an unnecessarily complex execution plan.
@tsunamio7750
@tsunamio7750 Год назад
no examples
@jlbciriaco3142
@jlbciriaco3142 4 месяца назад
excellent information but boring as hell. Add some music or images man
@nonamespls3468
@nonamespls3468 3 месяца назад
you should have given samples man
@user-zy6lm8wm9w
@user-zy6lm8wm9w 4 месяца назад
This video tells you to do things but it doesn't tell you how to do the things.
@DailyShorts1689
@DailyShorts1689 2 года назад
If you used examples when it would be better
@KahanDataSolutions
@KahanDataSolutions 2 года назад
Check out this video (it has some examples) - ru-vid.com/video/%D0%B2%D0%B8%D0%B4%D0%B5%D0%BE-asCQj1ybnds.html Thanks for watching!
Далее
5 Tips for Better SQL
6:37
Просмотров 7 тыс.
Improve SQL Performance w/ Indexes
5:49
Просмотров 10 тыс.
Прилетели в Дубай
00:17
Просмотров 75 тыс.
Expert Level SQL Tutorial
23:27
Просмотров 151 тыс.
How to Use Jinja w/ dbt Macros (3 Examples)
11:34
Case Study - Past Paper v Live Course
3:02
Faster database indexes (straight from the docs)
13:28
Просмотров 124 тыс.
Modern Data Engineering Workflows, Explained
6:38
Просмотров 4,5 тыс.
lookup & Reference | Xlookup | دالة
3:21
Прилетели в Дубай
00:17
Просмотров 75 тыс.