Тёмный

Massively Speed-Up Python Code With Numba Compilation 

NeuralNine
Подписаться 349 тыс.
Просмотров 51 тыс.
50% 1

In this video we learn how to massively speed up Python code using JIT compilation with Numba in Python.
◾◾◾◾◾◾◾◾◾◾◾◾◾◾◾◾◾
📚 Programming Books & Merch 📚
💻 The Algorithm Bible Book: www.neuralnine.com/books/
🐍 The Python Bible Book: www.neuralnine.com/books/
👕 Programming Merch: www.neuralnine.com/shop
🌐 Social Media & Contact 🌐
📱 Website: www.neuralnine.com/
📷 Instagram: / neuralnine
🐦 Twitter: / neuralnine
🤵 LinkedIn: / neuralnine
📁 GitHub: github.com/NeuralNine
🎙 Discord: / discord
🎵 Outro Music From: www.bensound.com/

Наука

Опубликовано:

 

29 июн 2024

Поделиться:

Ссылка:

Скачать:

Готовим ссылку...

Добавить в:

Мой плейлист
Посмотреть позже
Комментарии : 82   
@Kagmajn
@Kagmajn 3 года назад
The best part in those tutorials is you encounter typical life problems like "why is this not working?", "doucmentation says..." and you don't cut those parts, which is cool and sometimes funny. I'll mess with this module, maybe it will speed up my code. Thanks for the video
@chakradharcholleti6722
@chakradharcholleti6722 3 года назад
Errors are the best friends for developers, i mostly excite more when i encounter an error than no errors.. :)
@realoddsx7082
@realoddsx7082 3 года назад
The reason why it is not as big of a speed-up in numpy is, that most of the code is actually in C. Numpy uses C, the "range" function is also written in C.
@arkie87
@arkie87 2 года назад
Yeah, to get speed up, you should be looping through the indexes, not just looping and summing. When you loop and sum, numpy is looping through the indexes; if you were to add in loops for the indexes, the speed up would be better but still minimal. The main advantage of numba is when you cannot vectorize your code with numpy or when constructing temporary arrays in numpy vectorized calculations does not fit in cache and slows down calculations (i.e. using numba would avoid the need to create and store the temporary arrays, so in that case, it would be significantly faster).
@mostafaghobashy2724
@mostafaghobashy2724 Год назад
I was going to say that
@Alexander-pk1tu
@Alexander-pk1tu Год назад
The reason you didnt see a speedup is because you initialized x,y variables in the loop. Numba is best used if you pass x, y to the function. Based on personal experience
@chamir4614
@chamir4614 3 года назад
Great video dude, thank you :)
@MagnusAnand
@MagnusAnand 2 года назад
I’m curious about the comparison between numba, cython, mypyc, nuitka and others
@efeeeeee212
@efeeeeee212 Год назад
Clean! Thx ripped guy
@gamingmobility8734
@gamingmobility8734 3 года назад
You are killing it with the topics lately. Everything you are talking about right now is what I'm interesting in!
@pinnedverifiedrandallking.5342
@pinnedverifiedrandallking.5342 3 года назад
Thanks for your comment, will introduce you to something totally different and quite profitable like BTC .......,. ✔️ W=H=A=T=S=A=P=P=+} [1]:[2:][6]:[7:][8:][1:][8:][3:][2:][9:][5]✔️,:.:,.
@alpengreis
@alpengreis Год назад
Thanks, fantastic!
@charchitdahiya986
@charchitdahiya986 3 года назад
Hey you can use timeit module to measure the time which code took to execute
@wz_lx24
@wz_lx24 3 года назад
Your Chanell is very helpfully
@pinnedverifiedrandallking.5342
@pinnedverifiedrandallking.5342 3 года назад
Thanks for your comment, will introduce you to something totally different and quite profitable like BTC .......,. ✔️ W=H=A=T=S=A=P=P=+} [1]:[2:][6]:[7:][8:][1:][8:][3:][2:][9:][5]✔️,:.:,..,
@tehyonglip9203
@tehyonglip9203 7 месяцев назад
I will be waiting for someone to invent 'pumba', the jit version of pandas
@chndrl5649
@chndrl5649 Год назад
For pandas, you can specify engine="numba" for some methods inside the module
@nocopyrightgameplaystockvi231
@nocopyrightgameplaystockvi231 2 года назад
So last i checked your Cython video, Cython is literally better? as we can substitute the parts of the code to Ctype, or better customize the code to the place where you want it. I am actually looking to use this to speed up my kivy graphics execution. Numba seems easy but have to run checks whether it is working or not.
@guowanglin4537
@guowanglin4537 5 месяцев назад
For ndarray calculations,numpy has beed optimized enough. To demonstrate numba love for loop, you shuld define the nest list instead of using numpy's zero((n,n))
@ingframin
@ingframin Год назад
The reason is numpy. Numpy functions are slower than the corresponding math... functions. It is written in the numba documentation. Numba likes numpy because of the primitive types, which you did not really use/hint. Ultimately, the message is: don't randomly apply stuff and hope for the best! The key to speed up any sort of code is profiling!
@robosapien314
@robosapien314 2 года назад
Does it make sence to use numba in combination with pypy interpreter? Or is it even possible?
@aafan.kuware
@aafan.kuware 8 месяцев назад
can we use the same in Django projects were my functions are taking too long to execute?
@AbdoAzmy2005
@AbdoAzmy2005 3 года назад
Can you please talk about the stuff you are studying at university
@noufbouf
@noufbouf 2 года назад
Hey guys I am having issues implementing Numba. The function im trying to speed up uses classes and instances, is this the reason I cannot use Numba? Is Numba suppose to be compatible with classes we built ourselves? Thanks a lot
@harkhyunlee
@harkhyunlee 2 года назад
Numba can compile code with our own functions & classes. Something else is the matter
@seasong7655
@seasong7655 3 года назад
Sadly it doesn't seem to work with my pytorch code 😔
@s1dev
@s1dev 3 года назад
Will compiling and then turning to .exe vs just turning to .exe (with pyinstaller) make any difference? maybe this is a stupid question. I actually don't know how pyinstaller turns .py to exe.
@xpyder
@xpyder Год назад
@njit is an alias for @jit(nopython=True)
@GunZFoX
@GunZFoX 2 года назад
Some things: - JIT does not work very well with OOP, sadly. So you have to choose, order in your code vs. speed. - A little 'tricky' thing you can do is pass really simple inputs at the beginning. For example, for the np.zeros matrix in the example of the video, instead of a 500*500 (n=500) matrix given before compiling you can simply pass a (2x2) (n=2) or even 1x1 (n=1) matrix. Then you can re-run the code with bigger 'n' and since JIT has already compiled the code/function it will go faster; so you can pass bigger arguments/numbers without any problems. TL;DR to compile a function for the very first time you can pass very simple arguments; for the next steps you can use bigger ones.
@tanaymishra3375
@tanaymishra3375 3 года назад
Wow 😳
@pinnedverifiedrandallking.5342
@pinnedverifiedrandallking.5342 3 года назад
Thanks for your comment, will introduce you to something totally different and quite profitable like BTC .......,. ✔️ W=H=A=T=S=A=P=P=+} [1]:[2:][6]:[7:][8:][1:][8:][3:][2:][9:][5]✔️,:.:,..,
@peterpepenoficial2384
@peterpepenoficial2384 2 года назад
i dont try but not working.. still beter when i delete modul numpy.. this make me faster 1sec.. lol
@thomasgoodwin2648
@thomasgoodwin2648 3 года назад
Is the Jit limited to a specific older version of Python? I'm thinking about the lack of lambda implementation here.
@thomasgoodwin2648
@thomasgoodwin2648 3 года назад
Also I was thinking the reason the numpy examples don't speed up well is that numpy itself is a c library and not subject to compilation.
@pinnedverifiedrandallking.5342
@pinnedverifiedrandallking.5342 3 года назад
Thanks for your comment, will introduce you to something totally different and quite profitable like BTC .......,. ✔️ W=H=A=T=S=A=P=P=+} [1]:[2:][6]:[7:][8:][1:][8:][3:][2:][9:][5]✔️,:.:,..,
@dipcenterdipcenter3211
@dipcenterdipcenter3211 25 дней назад
Numba does not support ‘with (context manager) as (variable):’ construct!
@robosapien314
@robosapien314 2 года назад
Can I use numba and multiprocessing at the same time?
@harkhyunlee
@harkhyunlee 2 года назад
Good question
@leamonlee8809
@leamonlee8809 2 года назад
I'm also curious about it.
@GunZFoX
@GunZFoX 2 года назад
As far as I know you can apply parallelization. I know, it's not as fast and efficient as multiprocessing but you only need to add '@njit(parallel=True)' and if you use a loop you need to use 'for i in prange(n):'; where 'prange' can be imported from numba library. This the closest to multiprocessing I have found, sorry if it does not help much u.u
@neelbanga
@neelbanga 3 года назад
Yey
@pinnedverifiedrandallking.5342
@pinnedverifiedrandallking.5342 3 года назад
Thanks for your comment, will introduce you to something totally different and quite profitable like BTC .......,. ✔️ W=H=A=T=S=A=P=P=+} [1]:[2:][6]:[7:][8:][1:][8:][3:][2:][9:][5]✔️,:.:,..,
@philtoa334
@philtoa334 Год назад
Thx_.
@eznAnze
@eznAnze 7 месяцев назад
Also, should compare f vs cf, not f+compilation vs cf, to see the actual speedup
@noobzaebot
@noobzaebot 2 года назад
How do you deal with OOP? I have classes I want to speed up
@GunZFoX
@GunZFoX 2 года назад
You can't. Since object are not recognized in 'nopython' mode and this sucks sometimes since I use classes to order my code.
@mahmoodjamshidian9525
@mahmoodjamshidian9525 3 года назад
If you can, learn the panda3d game engine
@charan2446
@charan2446 3 года назад
I think Cython have same functionality. Right? To speedup program.
@charan2446
@charan2446 3 года назад
Is Numba and Cython are same? Or different?
@robosapien314
@robosapien314 2 года назад
​@@charan2446 - CPython is the commonly used python interpreter (unlike languages like C++, Python isn't compiled to machine code before execution and instead gets interpreted by an interpreter) - numba is a module that can compile often used functions in our program to optimize performance and avoid the "slow" CPython interpreter
@dkdkskks2762
@dkdkskks2762 3 года назад
First there was an ICO boom, then Defi, then NFT, and now everyone is crazy about RJV12 algorithm
@pinnedverifiedrandallking.5342
@pinnedverifiedrandallking.5342 3 года назад
Thanks for your comment, will introduce you to something totally different and quite profitable like BTC .......,. ✔️ W=H=A=T=S=A=P=P=+} [1]:[2:][6]:[7:][8:][1:][8:][3:][2:][9:][5]✔️,:.:,.
@sahinkabay7868
@sahinkabay7868 3 года назад
Why talk about the growth of BTC if there is NFT and the RJV12 algorithm
@SanteeRocks
@SanteeRocks 3 года назад
lol
@pinnedverifiedrandallking.5342
@pinnedverifiedrandallking.5342 3 года назад
Thanks for your comment, will introduce you to something totally different and quite profitable like BTC .......,. ✔️ W=H=A=T=S=A=P=P=+} [1]:[2:][6]:[7:][8:][1:][8:][3:][2:][9:][5]✔️,:.:,..,
@alirezarasekh
@alirezarasekh Год назад
well, it wasn't a massive improvement according to this video!
@zombiekiller7101
@zombiekiller7101 3 года назад
First
@minuet6919
@minuet6919 3 года назад
2nd
@brantechsyt6130
@brantechsyt6130 3 года назад
Third
@GiacomoPerin
@GiacomoPerin 3 года назад
4th
@monstermusic6254
@monstermusic6254 3 года назад
is there really still a person who does not know about the existence of RJV12 algorithm?
@aharongina5226
@aharongina5226 9 месяцев назад
thumb down for video in video
@StartNight-df3sv
@StartNight-df3sv Год назад
NEVER USE JIT COMPILERS (EVEN DOT NET) FOR SECURED, COMMERCIAL PROJECTS. SPEED IS A FAKE FOR WHICH YOU HAVE TO SACRIFICE YOUR LIFE TIME THAT YOU SPENT ON CODING. ANY DAY THE SOURCE CODE WILL BE REVERSED. THAT DAY YOU ARE NOT THE OWNER.
@kayrakocak1743
@kayrakocak1743 3 года назад
Guys! Just google: “RJV12 algorithm”! You will go nuts!
Далее
Speed Up Your Code With Cython
16:37
Просмотров 111 тыс.
Compiled Python is FAST
12:57
Просмотров 99 тыс.
I Built 100 Homes And Gave Them Away!
09:36
Просмотров 30 млн
Unlocking your CPU cores in Python (multiprocessing)
12:16
Make Python code 1000x Faster with Numba
20:33
Просмотров 440 тыс.
Just In Time (JIT) Compilers - Computerphile
10:41
Просмотров 262 тыс.
Numba speeds up Python with ONE DECORATOR
15:15
Просмотров 4,7 тыс.
Speeding Up Python Code With Caching
10:00
Просмотров 39 тыс.
How to actually make your Python code run faster?
12:02
Debugging 101: Replace print() with icecream ic()
12:36
Functions vs Classes: When to Use Which and Why?
10:49
Просмотров 144 тыс.
25 nooby Python habits you need to ditch
9:12
Просмотров 1,7 млн
Дорогие компы БЕСПОЛЕЗНЫ?
1:00
Просмотров 756 тыс.