How SQLite and Python are getting better

Подписаться 248 тыс.

Просмотров 13 тыс.

50% 1

Comparing performance of Python and SQLite and what 10 years can do.
*Correction: thanks to the awesome commenters for pointing this out. In the video at 5:42 there are zero rows selected. This was because I was selecting the value 'YES' instead of 'Yes'. When all rows are selected, the time is 3.4 seconds.
▶️ Are you using the SLOWER Python? - • Are you using the SLOW...
▶️ M1 Max vs Intel Core i9 Python Race | XPS 15 2022 - • M1 Max vs Intel Core i...
▶️ M1 Ultra vs Intel Core i9 Python Test DESKTOPs - • M1 Ultra vs Intel Core...
▶️ M1 MacBook vs Intel i9 MacBook Python Test - • M1 MacBook vs Intel i9...
▶️ C++ Sorting 1000000 Items - • M1 MacBook Air vs Inte...
▶️ Open CV - • OpenCV Build SLOW? Mak...
Sebastian Raschka's blog post: sebastianrasch...
#python #sqlite #programming
💻NativeScript training courses - nativescriptin...
(Take 15% off any premium NativeScript course by using the coupon code YT2020)
👕👚iScriptNative Gear - nuvio.us/isn
- - - - - - - - -
❤️ SUBSCRIBE TO MY RU-vid CHANNEL 📺
Click here to subscribe: / alexanderziskind
- - - - - - - - -
🏫 FREE COURSES
NativeScript Core Getting Started Guide (Free Course) - nativescriptin...
NativeScript with Angular Getting Started Guide (Free Course) - nativescriptin...
Upgrading Cordova Applications to NativeScript (Free Course) - nativescriptin...
- - - - - - - - -
📱LET'S CONNECT ON SOCIAL MEDIA
ALEX ON TWITTER: / digitalix
NATIVESCRIPTING ON TWITTER: / nativescripting

Опубликовано:

17 сен 2024

Ссылка:

Скачать:

Готовим ссылку...

Добавить в:

Мой плейлист

Посмотреть позже

Комментарии : 38

@smko Год назад

I think mostly it's because of powerful m1's SSD. 10y ago, he was probably using hdd.

@chetana9802 Год назад

was thinking the same, the hardware considerations matter too

@salmanahmed-qn2ui Год назад

SSD's do matter

@AZisk Год назад

*Correction: thanks to the awesome commenters for pointing this out. In the video at 5:42 there are zero rows selected. This was because I was selecting the value 'YES' instead of 'Yes'. When all rows are selected, the time is 3.4 seconds, which means SQLite got a bit slower. But to get the real deal numbers, I would have to use my M1 Max MacBook to run the 2013 versions of Python and SQLite.

@shoarectube Год назад

Sqlite will keep the data in memory whenever possible, which in this case is probably always since the entire dataset is 75MB. In this case, the query will measure the RAM advancements where as, just reading the file from disk or populating the DB is mainly I/O bound. The conclusion is that disk has become considerably faster. Hard to say how much python and SQLite improved since it is not running on comparable hardware.

@phuocle 11 месяцев назад

Not good at math but also not good at realizing that the difference is all in the SSD vs. a mechanical HDD. Hardly a surprising result.

@jonathanbereyziat4063 Год назад

5:42 -> 0 lines read 😅 I think that explains the speed bump. Could be interesting to take those functions and compare python 3.3 (most recent stable version of python in 2013) on M1 max with completely rewritten code to take advantage of python 3.11 code on M1 max. The hardware gap here makes the comparison kind of not relevant in your tests.

@AZisk Год назад

Thanks! I've pinned a comment addressing this.

@PankajNikam Год назад

Power consumption should also be a consideration for Python in large application.

@heroe1486 Год назад

I guess most of the time it's neglectable compared to engineering time and thus cost

@techheaven8132 Год назад

Your read of the SQLite file read 0 lines of the 6 million lines. That is why it was so fast. It didn't read any of the lines.

@AZisk Год назад

Thanks! I've pinned a comment addressing this.

@Dygear Год назад

SQLite is a really good database, I've been using it in production for the last 10 years as my website's database for my company. It flies on an Optane drive if you need to squeak out some extra performance.

@vectoralphaSec 3 месяца назад

Im new to all of this. How do set up sqlite to be used in production when deploying your code to be hosted online? Im having trouble understanding this.

@MrDowntemp0 Год назад

Really appreciate you starting the video with "Hey Google" That certainly didn't cause any issues in my listening environment :P

@Nithis-kumar Год назад

As someone commented, it's mostly because of nvme SSDs and hardware upgrades too.

@primarchlogarius Год назад

I was curious that the first test was fast because it was the same line 6 million times. Would the speed change if the text was different? Is there any optimization due to the content of the file being the same?

@soham7510 Год назад

if anyone is more intrested in speed ups in python can see this video > Why Python 3.11 is so fast | Guido van Rossum and Lex Fridman < (*didnt provide link as youtube might censor my comment)

@AZisk Год назад

Thanks! Here is the link (I'm immune to my own channel's filter 😀) ru-vid.com/video/%D0%B2%D0%B8%D0%B4%D0%B5%D0%BE-TLhRuZ9cJWc.html

@FreakyDudeEx Год назад

that's obviously ssd vs hdd.... tried something similar size for reading before and it read it in 10s of seconds on a hdd in a dual socket xeon server from 2019... got upgraded to ssd and the time went way down....

@UrnestHemingouey Год назад

04:16 Why it doesn't matter what the line is? Especially when it comes to writing, wouldn't the length of the strings be crucial?

@anantmishra6783 3 месяца назад

length of strings is crucial very obviously, and there are a 100 other factors, this video is shit-on-steroids.

@kenvererer Год назад

We also call it monkey patching in Python. 🐒🐍

@victorutomo7244 Год назад

What I see is a huge improvement in text file reading. After years of .NET coding, I must have said that python read text file (and CSV) very fast.

@yiyinyu9679 Год назад

Why don't you recreate the software stack (as of the test was conducted, didn't look into it, but there could be some old docker images :) ), test both on the same hardware, which eliminates the hardware difference :)