Compiled Python is FAST 

You're very underrated

Mojo and Codon(Exaloop)?

Absolutely!

@@dougmercer taichi doesn't look very boiler-platy either with just the use of a decorator.

Absolutely. C/C++ and gcc -O3 is basically magic.

​ @dougmercer I am pretty convinced that taichi under the hood creates 1D array and not 2D. Doing vector hits the performance quite a bit (while not the most reliable test, changing vector to normal vector gave ~10% boost. Although both C++ versions where faster than taichi for me. (compiled with MSVC release). There are still some minor things, but they shouldn't influence anything since in my case it was ~40-50% in std::max and 20-30% in creating the vector. All in all, nice video showcasing the tools.

Ah, also... just out of curiosity: @numba.njit def lcs2(a, b): m, n = len(a), len(b) dp = [0] * (n + 1) prev_row = [0] * (n + 1) # Temporary storage for the previous row for i in range(1, m + 1): for j in range(1, n + 1): if a[i - 1] == b[j - 1]: dp[j] = prev_row[j - 1] + 1 else: dp[j] = max(prev_row[j], dp[j - 1]) for j in range(1, n + 1): prev_row[j] = dp[j] return dp[n] Less memory allocation / 2D array. Testing this against C++ / taichi would be a nice one :) [and you have some vectorisation you can throw there]

​@@BartekLeon-jx5jv it's not a 1D array, but a homogeneous ND array. It's somewhere between vector and int[A][B]. It is represented as a flat array in memory, but unlike int[A][B], the data type, number of dimensions, sizes of these dimensions and the iteration strides are dynamic. Also, it's not just taichi that's using ndarrays, numpy and numba are also using ndarrays here.

​@@ruroruro That's what I meant in a sense. Although all is still boiling down to: are you allocating once or are you allocating N times (in case of vector).

Thanks for the support Maks! =]

I'll check it out! Also, I have looked at Nim in the past. It seems nice. Eventually I may do another video on this topic, and branch out to other languages (Nim, Julia, and now Mojo). Thanks for the idea, the video rec, and thoughtful comment =]

Enjoy! And good luck =]

Thanks so much =]

That's so nice! thanks =] 🤞

Thank you so much!

After reading the other comments while thinking up my own, I feel compelled to echo this sentiment first. Fantastic job, @dougmercer - both technically and visually - I loved it all.

Thanks @stereoplegic! That means a lot =]

500x is great! And good point on prange-- I should have covered the parallel aspect more of all the solutions (numba, Taichi, and cython) but I glossed over it due to the serial nature of the example problem. Thanks for the comment =]

Taichi is great for fractals! I like that it has good built in infrastructure for plotting to a canvas. That said, I'm sure you'll find a use for your new-found C++ knowledge =]

My favourite was numba as we were able to achieve our goal with very little code, there are certain shortcut algorithms that can be applied to makeup for its non applicable functions

That's an awesome compliment-- I'm gonna put "Code Documentarian" on my resume. Thanks for watching and commenting =]

Glad it was helpful =]

Thanks so much =]

Thanks! =] Yes, you should be accelerate a pygame-based game with these tools. You can't speed up pygame functions and methods, but you can speed up your code between those calls. It'll be most well suited for larger, number crunchy parts between methods rather than quick little one-off operations. Let me know if you end up tweaking something and seeing a boost in performance!

Thanks! Glad you found it helpful =]

Thanks so much! I should have another performance related video out in mid April so see ya then =]

Thanks =]

Thanks Finnnicus! Much appreciated =]

Thanks! Glad it was helpful!

Yay! Success =]

Numba should work great! You may just need to tweak your implementation slightly to use the subset of numpy features supported by Numba.

@@dougmercer have you tried anything that helps with matplotlib?

Hmm. Hard to say. Could try mypyc-- maybe it'll just magically work. Alternatively, though this might be a bit disruptive, you could swap out CPython with PyPy (a JIT compiled replacement for the CPython interpreter). In the video I'm working on now, PyPy was shockingly convenient and fast.

What are you plotting, out of curiosity? Maybe do a quick sanity check to make sure the amount of data your plotting has exceeded the usefulness of matplotlib. If it's a scatter plot with millions of points, maybe you should use something like datashader or similar

@@dougmercer I'm using it to do histograms for images that have been turned black and white and then converted to 8 bit png files to convert them to stippling.

PyPy is fantastic -- I'm actually going to cover it in my next video!

That's awesome! I love numba-- super convenient and fast

Thanks for watching and commenting!

Thanks Manuel! Glad to have you =]

=]

Glad it was helpful!

Thanks Russell =]

Thanks so much!

I did not count compilation time for the c++ times, but did include JIT time for the first run of Numba. However, it doesn't play a big impact, because we are typically doing 100s or thousands of runs and adding up their times (so the first run being slow only accounts for a small part of the overall time)

Thanks =]

Thanks for watching!

Yup! You can check out the C++ code/compile command here, gist.github.com/dougmercer/1a0fab15abf45d836c2290b98e6c1cd3

Thanks!

> "look what they need to mimic a fraction of our power" Haha, true! In another comment, I said I loved that even if I write terrible C++ it still turns out pretty fast. That said, the same argument could be reversed, if we consider productivity and third party library access. If an application is 95% high level glue and one hot spot, I'd rather write the majority in Python and the hot spot in an AOT or JIT compiled variant of Python than write my entire app in a low level language. The overhead would be worthwhile from a productivity perspective. > Proper compilation flags Do you have flags you want me to try in particular? I did -std=c++11 -O3, but maybe I'm missing something. > SIMD Since this is all sequential, can SIMD help? I thought SIMD was for packing multiple of the same operations in a single instruction (but again, I'm not a C++ dev) > the Python just provides an interface to a lower level language. True! And I'm OK with that! I def agree that well written, native code in a lower level will out-perform generated code from Python. That said, for all but the most trivial algorithms, I can't write well-written C++. So, if I can get even a 95% solution for free from these high level LLVM interfaces, then I'm stoked!

@@dougmercer ( : That remind me a benchmark done by Microsoft, Debug C++ /NoSIMD vs Release C# SIMD, and they notice faster C# :D Yeah sure... The point of Python is not to be faster, it's mostly to be gentle with non-engineer-long-beard programmer, the user are mostly scientist and data-analysts. > Productivity For this example I see no productivity differences between C++ and Python. But personally I'm more productive in C++ with Eigen and few other lib Like an experimented Python will be faster with numpy and his other favorite libs. > Proper compilation flags I don't know what is your compiler, but for Visual Studio: /Ot {favorize speed} /Oi {Inable Intrinsic} To increase the STL speed, Disable C++ expcetion, "Basic Runtime Checks", /GS-, /GR- ... To help intrinsic generation /Zp8 or /Zp16 (here you're processing int), but we can process And based on your hardware /arch:AVX, ... > SIMD You have gather and scatter instruction that could help, need to profile ( : > Improve On both side I'll bet we can performance by using only type you need. If your number cannot go higher than 100 just use a byte/uint8_t, etc. As I said the video was funny, the point is not to say Python is faster than C++, but more "if you're careful you can have performance higher or close to baseline C++"

I'm using g++, I'll try to find the analogs for the compiler flags you recommended. And true, a uint8 is enough. I'll mess around with that too. In any case, thanks for the comments! I'd def like to learn more about C++ but I don't get the opportunity very often

@@chkone007 Ok, I get the point but theres a lot of production code written in python, most code writing does not require performance and the few bits that do you can write a C extension or simply use C++ and python together

​@@user-zi2zv1jo7g I kind strongly disagree. Did you ever experienced slow UI, stuttering App, lagging game, ... If yes, you already met a programmer who said "most code writing does not require performance". If you said a code does not require performance that just mean you consider your time more valuable than the user time. As a developper we don't own time, the time is not ours, it's the user time. That's what make the difference between a smooth app, slow and memory heavy software, like everything web based, slack, etc. And all chromium stuff. Most of the devs said It's just a chat app, I don't need C++, just a chromium based. Consequences... My Mac/PC uses 8 GiB for doing nothing, just running a VM. And in a industrial point of view, you can release your startup with python code and saying "how I don't care it's CUDA underthehood". You just expose yourself to have a competitor who implement his stuff on C++/CUDA directly and this competitor will explode his profitability because his AWS bill will be much cheaper. We always require memory efficient and fast code. If none of those argument convience you, consider the CO2 argument, it's more eco-friendly for you PC or your server or your N-instances of your programmer running on AWS. I love python to prototype idea, and accelerate my exploration of ideas, but I cannot be serious with that to my clients. I know lot of "AI startup" are like that, download the model from the researcher, create a docker, build a website => step 2 => profit. Most of them rely on Python, but any competitor with cheaper infrastructure can scale more and be more efficient. I had in mind Facebook developed on PHP fine, cool, but at the beginning each new user cost more than the previous one, ... FB wasn't able to scale. They create "HipHop" compiler from PHP to C++, and now the company became profitable each new user became cheaper than the previous one. Conclusion => Performance always mater. Don't read me wrong, that doesn't mean I over-engineer everything to save 1 byte or 1 pico second in median. But keep in mind the quote "early optimization is the root of evil" was written from a time when everybody was written C and assembly code... The code is different, today with python, javascript, ... "early non-optimization is the root of evil".

Oh interesting, I didn't realize taichi wasn't on conda-forge. I wonder if they'd accept a PR 🤔. For what it's worth, you can pip install it (and that's possible even if you're using an environment.yml). I did not try torch, but I suspect it would very slow. Reason being-- the main use case for torch is parallel computing via tensors. Since this problem is inherently not parallelizable, my guess is it'd be super slow in torch.

@@dougmercer Thx for insinghts

Thanks =] I appreciate it. It's been a slow grind, but the past few days the algorithm has blessed me with some impressions, so I hope it keeps going 🤞

Thanks! I was surprised too

Oh, that's awesome! I think that's the fastest anyone has gotten it so far! Someone else in the comments encouraged me to try a 1D vector of size (m+1)(n+1) and index into it with arithmetic -- that gave me a roughly 1.1-1.2ish x speedup over the original C++ . So, I guess much of the remaining speedup came from data locality-- very cool that it was another 0.3x-ish boost. I'm glad you found the video interesting =]

Thanks Roshan! Means a ton to hear that =]

Thanks so much! Fingers crossed the channel does grow 100x 🤞. At that point I prob could make videos full time 🤯

Thanks! And welcome =]

I used -O3. Another commenter recommended using a 1D array and handling indexing through arithmetic, and that does speed up the C++ by about 1.1-1.2x. (still pretty similar to the ndarray approach from Taichi) Here's the c++ code and build script if you want to play around with it yourself =] gist.github.com/dougmercer/1a0fab15abf45d836c2290b98e6c1cd3

I do plan to do a project in Go sometime soon

Thanks so much! Be sure to share with friends/coworkers you think might enjoy this, and hopefully the channel will grow over time 🤞

@@dougmercer keep up the good work, it sure will 🙌

Thanks for watching! =] I'd try mypyc first. The others are way more disruptive and would probably require changes to your code

​@@dougmercer Thanks for your answer! I was thinking of something. Nowadays we almost always use type hints because they are great. But only for clarity/type-checkers like mypy. So we are not getting any performance benefit out of it, although I think we could have! Cython translates python to C and forces us to write statically-typed python for that. Which type hints could also be used for... Turns out that Cython supports type hints as well! Then we have stuff like MonkeyType that allows us to automatically type-hint code based on runtime behavior. Nice for annotating legacy code. 1) we write python code with type hints 2) if needed apply MonkeyType to apply them everywhere 3) compile with Cython 4) get a C-like performance I wonder why it's not actually practiced. Do you have any idea?

Mmm, for using type hints to achieve better performance through compilation, I think there's a high level design question: "should your code (1) look/feel like vanilla Python, or (2) are you OK with using non-standard Python features, or (3) are you willing to use syntax that only works in your special language, as long as it still vaguely resembles Python and interoperates with it"? I think mypyc is the closest to achieving the goal of speeding up vanilla Python. cython's python mode is pretty OK, but you need to add extra metadata to make it be performant (e.g., the locals decorator). Cython also has its own type system rather than using Pythons built-in types (e.g., cython.int vs int). Cython as a language (in non-python mode) isn't really Python any more, but interpolates with it well. Some other languages (e.g., Mojo) claim to have a "python-like" syntax and support interacting with Python, but the code isn't really Python.

​@@dougmercer Yeah it would be amazing if we could just write vanilla python with standard type hints and compile it with Cython. Apparenly Cython somewhat supports it. RU-vid blocks my commend if I paste a link but you can search this on google: Can Cython use Python type hints? Because todays type hints are everywhere and we don't get any performance benefit out of it at all, which feels weird.

It's hard to say-- when I was experimenting with this problem I remember not observing any speed up when adding vanilla Python typehints, and it wasn't until I started adding things like the @locals decorator that I really noticed any improvement. Let me know if you do any testing that shows a meaningful speed up!

Good point, I def could have made the metrics interpretation clearer. As for SIMD, it's hard to parallelize this because it's an inherently serial problem (everything requires previous solutions)

Thanks 💎 =]

Thanks! I primarily used Davinci Resolve, but used the Python library `manim` (community edition) for the code animations.

Thanks! @@dougmercer

I'm gonna try! Hahaha Thanks for subbing =]

The LCS dynamic program was on CPU. The visualization I showed at the beginning of the section of a kind of warping fractal was on GPU.

@@dougmercer Thanks. Taichi certainly looks promising, but I still prefer Numba for its simplicity, i.e. adding a couple of decorators, without altering the code too much. Have you tried Spark and Dask ? They're both parallel programming libraries.

Yup, both are great! Since this problem couldn't be easily parallelized, I didn't mention them. And I agree, in general Numba will be easier than Taichi by a long shot. I just thought Taichi was kind of neat so I included it in the video ¯\_(ツ)_/¯

If this video ends up getting some more views, maybe I'll do another pass at adding other options. I have a *guess* though... PyPy would speed this up significantly, probably on par with numba. I've heard good things about it *but* it didn't install first try when using conda on my M1 Mac, so I skipped it ¯\_(ツ)_/¯ Nuitka would only speed things up a little bit. From what I've read, nuitka is more so about compatibility (supports *all* python language constructs) and for making standalone, portable builds. For nuitka, speed is secondary to those concerns

True! Through C/C++ extension libraries, you can directly write/link C/C++ libraries and write your own Python interface to it. Cppyy, ctypes, cffi, pybind11, and Cython are all fair game for this.

Hah, *very carefully* in Davinci Resolve (Fusion Page) =P I manually drew the graph using rectangles, then applied (noise + displace) to make it more irregular + (fade it out with noise + the "painterly" effect from Krokodove) to give it the water color appearance + paper texture + adding lens blur One of my favorite animations I've made =]. Thanks for commenting on it

That's like the best compliment =] thanks!

That is nice! I haven't played with Julia since undergrad, but it's definitely a cool language! I'll have to find an excuse to dive back into it!

I agree! there is usually a lot of room to make your algorithm/implementation better

Python is super slow by default. The only skill issue is actually the choice of Python when performance matters, because it was never designed for speed, or power draw, and optimizing it goes against its fundamentals. If it does not fit, as is, then use another language instead of a shoehorn.

No prob! Glad it was helpful

Thanks =]. What's your preferred language if Python is against your will?

@@dougmercer Haskell is my love and I like lambda calculus so I am writing a interpreter and compiler for my own lc implementation for fun. (in haskell)

@@user-up8fm3vb1r very cool. I haven't touched Haskell much, but I'm learning ocaml for fun recently and enjoying it

@@dougmercer glad to see you join the functional land.. enjoy!!

Probably means that I left some performance on the table in the C++, or the JIT pulled some tricks that most people wouldn't pull when writing it natively. Someone else in the comments found that using a flat 1D array gave the C++ a 1.1-1.2x speedup. That probably puts it on par with the Numba/Taichi ndarray approaches That said, the point of the video still stands-- for at least this particular problem, there are several approaches for getting performance on par with native C++

Absolutely! Fantastically well done. I'm really quiet impressed by what you did.

Thanks =]

Maybe one day! Thanks Max!

You can definitely use Cython or Numba to help speed some things up with pygame. I found a few old reddit threads that included demos and discussions by searching "Numba pygame reddit".

Glad it was helpful! And definitely, I'm a big fan of "good enough" speed, and generally I can get that with Python

What about Nim?

Yup! gist.github.com/dougmercer/1a0fab15abf45d836c2290b98e6c1cd3

@@dougmercer thanks! i just managed to get it 169% faster (see fork). still, the speed improvements offered by numba, pyx, and taichi are really impressive :)

Very cool! Yesterday I implemented the 1D index approach (not nearly as cleverly-- just hand jammed the indexing arithmetic in line) and I got about 1.1-1.2x speed up. Does the noexcept make a difference in performance? Or is there something else causing the extra 0.4ish speed up 🤔

Thanks =]

Thanks! I had fun choosing music for this one =]

I did compile with -O3 for my C++ test

gist.github.com/dougmercer/1a0fab15abf45d836c2290b98e6c1cd3

@@dougmercer Oh, you did. This is surprising how Python can be faster than C++!

Definitely surprising! That said, I'm sure someone could write faster C++! But, it did beat my first attempt at translating the code into C++ ¯\_(ツ)_/¯

You're right! It's not easy to get C++ performance in Python. I think these tools are appropriate when there are a few "hot spots" in your code, but the majority of your application benefits from Python's ecosystem. It's possible to directly build C extensions and call them from python, but I think these tools are way easier. For some (new) projects, it might make sense to write the whole thing in Rust from the start. In practice, most of my projects use a lot of Python libraries, and my team is not very flexible (they mostly only know Python), so it'd be pretty disruptive if I wrote a critical component in a different language and with different tooling. Good question! (Sorry I don't have a good answer =P)

@@dougmercer that is indeed a good answer, thanks. Since I am working in the Data analysis field (geospatial) I love Python for its possibilities. I was wondering if it makes sense to learn another language for intensive calculations like C++. But think I will try your tools 😊 Many thanks!

Good point! I may have to re-run this experiment at some point-- I wonder how Numba/cython would perform with that more memory efficient approach 🤔

Taichi was super fun. I did use GPU (well, metal) for rendering the fractal animation. Was pleasantly surprised at how easy it was.

@@dougmercerSorry that possibly came out the wrong way - I meant that you did a great job demonstrating it *even without* using the GPU!

Oh, I see now-- hah! Thanks =] I definitely would like to try using Taichi for an ML project. Taichi + Torch seems like a great fit. Do you have any open source projects you've done with it? (I have skimmed through the docs section involving torch, but haven't looked at real projects). I also thought it might be fun to make a "shader" to process video (but I can't for the life of me figure out how to extend Davinci Resolve with Python code, so that's kind of an unrelated blocker).

@rcer Yep! A few now - most of them are for bits and pieces I do at work, and largely undocumented e.g. for an HDR Camera ISP pipeline or a spatial subdivision grid for distance queries. By far the biggest one so far is a Taichi library for Gaussian Splatting rasterization, I called it taichi-splatting (distinct from original taichi_3d_gaussian_splatting, which it originally derived from but is very different now!). It has a few rough edges but I think it has enabled quite a clean yet performant implementation. I replied a yesterday but I see my comment is nowhere to be seen I think because I put a link in here, so I haven't this time! I must admit that before watching this video I did not realise that the CPU implementation in taichi performed so well, especially with the outer loop serialised!

Oh right! I saw your posts in the discord-- I read through the readme a bit. It looks very interesting-- I'll take another look at the source code sometime tomorrow. And sorry about the link issue! For some reason it's not showing up in comments held for review on the mobile app. I'll check in a browser tomorrow and hopefully approve it (if not, RU-vid totally ate it-- sorry)

I would say Cython and Numba are definitely "production ready"

Yeah, pypy is a great option

I do not believe any of these options will compile or JIT third party libraries. If I'm wrong, hopefully someone will correct me. That said, you can try using a different a Python interpreter altogether. PyPy would JIT whatever code it runs (but you need to use the PyPy interpreter instead of CPython)

@@dougmercer Thanks a lot : )

I've been meaning to give it a shot... It definitely seems very approachable

Hmm, I don't have any sure-fire recommendations. Could potentially try using multiprocessing if your plotting function is easy to map over an iterable of inputs? That way you can maybe speed up by the number of cores your CPU has.

Are you showing the plot? There is a way to not show the plot windows but still save to a file, it is still slow but much less than two seconds.

I've never tried it! Does it work well? I'll have to mess with it sometime 🤔 That said, I am working on a video where I cover one library that I wanted to include in this video (PyPy).

Hmm, what specifically did you have in mind? As an aside, I typically use FastAPI for Python web projects, but have used Flask in the past

Either is fine. Maybe an api that calls a couple other APIs and reads from a database. That would be your typical business api.

Thanks =]

Storyblocks

This is my first time hearing about either of those. Very interesting 🤔

Glad it was helpful =]

Definitely agree

More like 90%... I am trying to find an excuse to use c and its actually very hard finding something thst dosent have optimized code for you

@@nevokrien95 same-- I need to carve out time to learn Go this year but have literally no reason to do so ¯\_(ツ)_/¯

@dougmercer i am using it to write a proxy server that I switches between vpn connections. Go let's u do networking stuff python. Just can't.

Hmm, I might be wrong, but I don't believe Taichi has any filesystem support. I believe the simple thing to do would be to read data in Python and pass it to Taichi for processing. That said, I love Dask and Pandas! They rock!

Polars is faster than Pandas with almost identical API, right?

Yes, it is. I'm actually working on a video that talks about trying to read a very large CSV file and do some basic number crunching with it. (The one billion rows challenge, 1brc, but in Python) Spoiler alert, Polars and Duckdb are great choices.

Polars is by far the fastest to load CSV. It might even be faster when you load in polars and convert to pandas.

Hey @arta6183 - Sure! Here's a link to the code and compile command in a gist - gist.github.com/dougmercer/1a0fab15abf45d836c2290b98e6c1cd3 Note-- this algorithm is inherently *not* parallelizable unless you do some really wonky stuff (wave front optimization). So, I'm not sure if AVX will help. That said, I would love to see you squeeze 10x more performance out of it and share a gist back to me. Like I said in the video -- I only know the absolute basics of C++, so my C++ code is *bad*.

Thanks! Glad you liked it =]

I'm working on a video that uses it right now =]

Thanks! =]

=] I also used Nosferatu in my other video called "Your code is almost entirely untested"... I wonder what it means that I keep putting horror movie clips into my Python explainers 🤔

I didn't try Julia, but I've used it a bit in the past and it is quite fast. In a future video, I'd like to throw Julia and Nim into the mix

@mayankmaurya8631, I think you missed the point of the video. For a Python developer trying to accelerate a hot spot in their code numba, cython, and taichi were found to be just about as fast as C++. So, rather than hand writing C++ and complicating their build system, they can pip install numba and get just as good performance for very little work.

"Hand writing C++" Python is not "hand written"? Most of the fasy Python solutions presented in the video resulted in messy code and/or dependency/compatibility. C++ has none of those problems. I suppose that's the real point of the video. Can you make Python run fast? Yes, but it is easier and more maintainable to get the same result using C++.

Yup! gist.github.com/dougmercer/1a0fab15abf45d836c2290b98e6c1cd3 Some people in comments have gotten between 1.1-1.7x speed up through other improvements, but it doesn't really change the narrative much: these compiled Python tools frequently give good enough performance

@@dougmercer thank you! I think is really problem dependent. In some codebases I worked I had for example a 40x speed up over cython or numba by embedding very very small pure C functions using ctypes.

Oh definitely agree. Squeezing out performance is always "it depends" and "did you profile it?"

Yes, in my case there were two issues, the first was that cython for some things relies on the python interpreter if data and objects are not managed in the most cythonic way, the second was cache misses. I was working on a kd-tree implementation and a tiny detail on how nodes are managed let me cut out on cache misses during tree traversal. For that purpose I used perf to sample from the process but I know for sure that there are many other options for doing that.

​@@dougmercer Moreover, numba is a life saver if you need performance on the fly without many refactors.

=]