You are awesome Miguel, seldom have I seen such complex things explained and put into perspective this easily, take care and god bless you and your family.
Why can't I do web stuff with the multiprocessing library and what is the alternative? Let's say I computer that needs to run a python project that is both a Flask server and also does multiprocessing actions
1:51 simple definition of async programming 2:23 ways to do multiple things at once(concorrent programming) 3:57 async programming(example : play with a chess champion) 8:12 complete definition 8:42 how is async implemented( functions need the ability to suspend and resume 10:04 which function gets the CPU next(the idea of cooperative multitasking (event loop) ) ) 11:00 examples[this slide have like to more exampels]( 11:54 Asyncio example[use generator function VS async/await] 14:39 greenlet framework example[gevent VS eventlet] ) 16:03 Async pitfalls( 16:21 asyncio.sleep(0) 17:42 blocking!? ) 20:28 conclusion
Wonderful overview of not only async but the advantages over multiprocess/multithread. It is good point that asyncio is designed to make you write/think in an async way.
How the async program handle the token expiry for oauth. Eg: I have 30 requests running async but first request got the http error meanwhile before handling refresh token we I'll get the all responses.
This came up in the next recommended videos, and I wasn't particularly immediately interested in the topic as far as today was concerned (though I did have a todo interest for future review). But once I saw Miguel Grinberg was the speaker, I ponied up for the watch.
Threads arn't really any use if there are CPU bound tasks and hence GIL is an issue, but with async if GIL is not issue you can do concurrent CPU bound tasks, i think its an important he missed mentioning.
GIL is still there and notice he said that there is actually one thread andprocess involved so there is not much scope for high cpu bound tasks in async.
If asynchronous programming is just a style of concurrent programming, what are the other things that come under the umbrella of concurrent programming??? I thought this is the only way to do concurrent programming...
You've got to be careful though. Although async is called "concurrent", all the asynchronous tasks that run on the same thread do not actually run at the same time in parallel *at all*! Each async task gets to run for a little bit before it is paused and another task is then started or resumed. The async tasks (on the same thread) just simply take turn to run. Since CPU's are really fast, we get the illusion that async tasks run in parallel when in fact they do not. So, "concurrent" is somewhat a misleading term to describe asynchronous.
@@magno5157 That is actally the meaning of concurrent that you take up multiple tasks at one not neccesarily mutitasking them at the same time while what you are thinking about is called parallelism.
I have a question on the slide shown at 20:42. For async case how do we prevent the OS scheduler from interfering with async event loop. As per my understanding the async event loop is scheduled by OS?
the OS does not interfere with the event loop. The OS schedules processes and threads, while the event loop is run single-threadedly within one process.
A good talk. However, the gevent and eventlet examples were mismatched to the preceding "yield from" and "await" examples. Those used a loop to run the functions, which is the natural way to do it as it allows other functions to be also run asyncronously by that loop - hence the 10 call example with 10 hellos, followed by 10 worlds. The gevent and eventlet examples just called each single function in a non asynchronous way.. To be matching, they should have used the corresponding loop from those frameworks to similarly allow 10 versions of those functions to potentially be launched at the same time. The reason I mention this was the point that gevent and greenlet were harder to keep track of, as this omission served no purpose, other than to contrive this point a little.
No, you are actually incorrect. The gevent/eventlet examples are 100% asynchronous. They have a loop and all the function calls are truly async. As I mention in the presentation, these frameworks hide the asynchronous bits from the application, the loop is created implicitly by the framework, so that the application does not need to write the code in a different way. If you call the gevent or eventlet function 10 times in a loop, you'll get 10 hellos, 3 seconds and then 10 worlds, like with asyncio.
You are right, I judged the code based on the part of the video where you talked about the asyncio function being called 10x and printing 10x. The given gevent/greenlet code was not set up for a beginner easily doing this, where the asyncio version was. I think the gevent and greenlet examples are not representative of actual use, and that in practice most if not all production code based on that tech would use an explicit loop. Given this, the nebulous and otherwise correct warning about hidden complexity, is now extended to implying a loop is not even needed.
The gevent and eventlet examples are 100% correct, and totally representative of how production projects use these frameworks. The event loop in those frameworks exists, but it is implicitly handled by the framework.
Any projects that use gevent or eventlet would not explicitly define an event loop. Take any Flask application, run it on a gevent or eventlet based WSGI server and it'll be automatically async, w/o having to make any changes. The examples from my talk are here: gist.github.com/miguelgrinberg/f15bc03471f610cfebeba62438435508#file-eventlet-greenlets-py. Give them a try if you want.
In summary, asyncio is great when at minimum you need thousands of concurrent executions in a process. Threads work for when you need mere hundreds at maximum, albeit with possible race conditions. Also, threaded programming is different and reasoning about it is different.
No. Threads are something you only need for CPU performance (which is not what unaided Python is good at). The problem with threads is that they are notoriously prone to bugs. Coroutines are much easier to understand, so if your bottleneck is in the I/O (including network connections), then coroutines are preferable to threads, while offering performance just as good--the potential for race conditions essentially disappears.
