What the Heck Are Monads?! 

👷 Join the FREE Code Diagnosis Workshop to help you review code more effectively using my 3-Factor Diagnosis Framework: www.arjancodes.com/diagnosis

Whew, thank you for uploading this. I have a wedding to go to tonight and wasn't sure what to talk about with my fellow attendees. Now I do!

7:05 small correction: associativity means that x+(y+z) = (x+y)+z. What you've describe is commutativity (or the abelian property).

I got kicked in the Monads once, boy was that painful!

Finally someone who actually came close to explaining monads in a way that normal people can understand! I've only been waiting over a decade since Brian Beckman made the 'definitive' explanation that ended up confusing everybody.

Monads fill in the shortcomings of functional programming languages. They allow to sequence function calls while attaching a context, which is something that exists natively in python, so monads should stay in the FP world as python has better native ways to do the same things. But they're nice, the maths behind it is a bit harsh just to understand that it's merely a context passed down the chain of recursive calls to emulate a chain of sequential calls.

haha, this one was magical and I didn't see it coming! Following along I was like "feels like this weird concept which I haven't fully understood when i was trying to get into Rust" - turns out it was and I finally got it now! so I've watched a python video about a thing I shouldn't do in python, learnt about rust and ended up a better programmer than i was before. That's why you're once of a kind on youtube! have my thumps up :)

Happy to see some more advanced content! After being lost for about 18 minutes, it suddenly clicked how I could have used something like this instead of regular error handling on a project a couple of months back. Cool to be reminded that there are alternative approaches out there! Cool to be reminded in mind that there are more ways to do stuff!

This is a fantastic video, I have tried time and time again to understand monads, but this was the first video that really clicked. I like how you broke down the "monoid in the category of endofunctors" so thoroughly, that was a huge help.

In general, when thinking about programming languages in terms of category theory, functions in the PL are considered morphisms, and types are the objects. Functors map one category into another category. Therefore, unless you're somehow departing the language altogether, you're only ever going to touch endofunctors. To be more precise a functor, is any type that is: 1) defined in terms of another type, 2) preserves functions between types (this is what "map" gives you), and 3) preserves identity functions. The initializer is NOT required for it to be a functor. That is, any functor F does not automatically give you a function T -> F for any type T. The initializer is better thought of as a natural transformation from the identity functor to the functor in question.

Not necessarily a criticism, more of an observation but this seems leaps and bounds ahead in complexity with respect to most of the subjects of your other videos.

Thanks for picking up this important topic! Those new to functional programming might find the explanations involving monads, functors, and monoids within category theory, where their origins lie, to be intimidating. In my view, it is more crucial to first present the practical elegance and simplicity of these concepts, as demonstrated in your video. Just a small hint: i think you are sometimes confusing associativity with commutativity. commutativity: (a + b) == (b + a) => order of operands does not affect the result associativity: (f ・ (g ・ h))(x) == ((f ・ g) ・ h)(x) => grouping of operands does not affect the result

Thanks for showing this. Best video on monads by far and great to see better pattern matching usecases. Lately I’ve been writing tools that return tuples (eg (ok:bool, val: int, reason:str) returned from tryParseInt ). This is a great use case for pattern matching logic in the caller.

Thank so much Arjan! I tried to understand Monads each time I got hyped on functional programming, but only this time I think I understood the idea of a monad and usecases. And actually it seems that I have already been using something like that for error handling with ts-result library that brings in this railway approach that you mention in the video

Very nice to have you discussing an advanced topic again! Are you currently working in languages where monads do make sense?

Thanks, Arjan. As a Scala refugee, this was very useful to me. If someone in the Golang community can come up with an implementation, it'll be funny to see what they called it.

Any chance of a video on Generics? I have used them a bit in the past but i don't understand why you made the Monad depend on T at 8:49. I've never seen a good video on this topic.

It's easier to understand functors and monads using diagrams. A functor is a mapping between categories. A category is a collection of objects with arrows between the objects that can be composed. An endofunctor F in the category of Python types maps the type a to the type F[a] and a function from a to b to a function from F[a] to F[b]. The simplest functor in Python is the list constructor []. List comprehension is a clear example of mapping a function between types to a function between lists. In the category whose objects are functors, a monad is a triple, consisting of a functor and two natural transformations (that is the name of the mapping between two arrows from functor to functor). One acts as the identity under composition and the other composes two arrows. That is the monoidal structure. A list is also a monad, with double list comprehension acting as the composition. Since the result is a list and not a list of lists this operation is called flatMap in some functional languages. Scala and Haskell allow using comprehensions with any monad. Since this allows sequencing operations, it is commonly called a glorified semicolon. Monads allow reifying computations as data structures. Which means that instead of calling an API immediately one obtains instead a description of the API call. This is the reason why it is said that in pure functional programming there are no side effects. All functions return data. In Haskell only the main function can execute side effects. They are called side effects because in mathematics there is no function for sending emails. The idea of functional programming is to make writing software akin to writing a book in Word instead of with a typewriter. It is not for academics. It is what engineering always does: it gives us more powerful and sophisticated tools that allow us to go farther than our ancestors. The Joy of Abstraction is a quite accessible introduction to category theory. Also check out CQL and Ballerina. Programming with diagrams is not old-fashioned, it is only getting started. Thanks for making this video Arjan! 🎉

Very nice explanation about Functors and Monads. Seems like you have broken the curse of the Monad!

I think for a 20-ish minute video you're doing a good job in explaining what monads are. However there are a few things that are missing, unclear or slightly wrong IMO: * After describing monads at 11:13 I find the explanation of how they can capture side effects very unclear and hand-wavy (both literally and figuratively). I don't think it's obvious how from the structure of monads it follows that they can capture side effects. Adding an example of such monad would be helpful. * The Maybe monad is meant to be used as a replacement for None or null values. Using it to represent errors is akin to using None to represent errors: it could work in some simple cases, but is not the general solution. * In rust, the Maybe monad is called Opion. The Result monad is somewhat similarly looking to Maybe/Option, but instead of the None case, it has the error case, where it contains the error that happened. So it's a better mapping for exceptions. I agree that monads are most useful when they are part of the language from the get-go. In ML languages like Haskell, Ocaml and Rust you don't have the concept of null. Instead the standard library and basically the rest of the code is using the Maybe/Option monad. There are no exceptions either, so they are using the Either/Result monad. The compilers also check if the code handles all cases, which makes the use of monads there much better. Their usefulness in languages like Python or Java is limited. Even so, it's good to expand one's knowledge.

Hi Arjan, thanks for this video. I'm actually considering the railroad oriented programming approach in a data project. With a colleague we are about to migrate some processes running in an ETL tool that is actually already working in a functional way and which already has a way of chaining operations along happy/sorry paths. Besides functional programming makes parallelization easier, and in data processing this is important. So yes, in conjunction with the itertools, generator expressions and multiprocessing, I think we will give monads a chance under Python 😉

Awesome you back on advanced stuff! You should do metaprogramming topics

Thanks for this great demonstration! Very clear and easy to understand!

Thanks a bunch for your video, very informative. Regarding the usability of monad - I've spent quiet a lot of time getting rid of code that relies extensively on results library (specifically, it's error handling way). At some point of codebase growth it became virtually impossible to tell, what behavior and return types you should expect from methods (and code overall), therefore, we decided to move to Python's built-in exception handling and typing mechanisms

@ArjanCodes Very heady topic presented in an extraordinarily approachable way. I really liked seeing you turn maybe into a decorator. That gives me lots of ideas for my own home-grown functions. Thanks for another great tutorial.

Dang, missed opportunity to title the video "What the func are monads?!"

good and informative video, i have a small nitpick, Maybe isn't really the typical exception handling mechanism, its more the null or none handling mechanism, and to this extent the Result in rust would be the try except equivilant and Option would be Maybe (synonymous) monadic handling can be very useful even in languages its not implemented, it would come down to developer preference and style, i myself prefer monadic handling

Thanks Arjan, I tried to understand what monads were some time ago, but every video I started was just too complex for my understanding. You've explained it in a way that was very easy to understand!

This is great!!! Thank you so much for making this

I've watched like 5 videos on this really esoteric topic, and this is one that finally made sense to my programmer brain by typing up code examples rather than like Venn diagrams or whatever else functional thinking people do.

very nice, thank you, complex topics but I think I´ve learned something!

we need more videos like this!

I've known monads from using functional reactive programming paradigms like RxJava KotlinFlow and Swift Combine. They're very useful on multithreaded the front systems like mobile apps where you want to safely react to user input and do thread safe concurrent operations with standardised error handling. Functional reactive is a combination of monads and the observer pattern

Something isn't quite right with the example at 4:15. What do you exactly mean when you say it's not an endofunctor? I mean, in a programming languages we usually talk about only one category - category of types of the language. Which means all functors are endofunctors just by nature. And it's not even a functor, at least because map isn't polymorphic as it should, it only works with functions from str to str. Correct me if I'm wrong, maybe I just misunderstood.

thank you, that was very good explanation, it could be usefull if you have lot of possible types and validations, so you don't have to write special cases for everything, but define it in generic level

Great video, very good explanation! One thing, is that in Rust the equivalent of Maybe is actually the Option enum, the Result enum is another sum type that is specialized to represent an error.

I've been curious about funcotrs/monoids for so long and finally a video explaining this in practical terms! I also love that you explain why it wouldn't be useful yet in python. Personally, I see this as doing a bunch of unnecessary computation. If i divide by zero, there is no reason why I should continue down the Maybe(None) rail, why not just short circuit actually handle the error. On the other had I do see how this allows you to move error handelling to a high level and not worrry about it when there are other things going on

I really wish this video existed in 22 years ago when I had to sit through my first computer science class.

Great channel, subbed. Reddit sent me here. I am still working through a bunch of kaggle courses to learn python all the way up to AI coding. Even me having a barely basic understanding of the syntax, this was extremely easy to follow. 😊

I'm glad I completely agree with your conclusion. If it's in the language, sure, why not? But I won't step out of my way to use that when it's not the norm. It just makes reading the code more confusing if and when someone else has to dive in.

Thank you, an actual explanation of what the attributes of monad mean in practice

I'm just learning about monads and functors in python. Cool stuff. Might be wrong but looking at the example you gave with the `add_one` method been mapped to the Functor(5) which returns Functor(5 + 1). It think a functor is a programming implementation of functions in math. I feel that given that a functor returns its value when called, we can say: Functor(5) = f(5) -> 5 Say '->' means returns Hence; Functor(x) = f(x) -> x If add_one(x) -> x + 1 Then; f(x).map(add_one) -> f(x + 1) If we let; g(x) = f(x).map(add_one) = f(x + 1) g(x) = f(x + 1) -> x + 1 But f(x) -> x, Therefore g(x) -> f(x) + 1 Mehh. I'm not actually sure if this is right but if it is, then i think understand better using this analogy.

Great vid! First class functions - a good thing. Functional programming - meh not so much. And I write a lot of Rust... For real-world applications functional programming will give you poor performance, or great performance right up until COOOOOOOOWtf? - clunky error handling and readability problems past a certain scale. For raw data processing it's fine. Feels like unix pipes. For anything bigger, anything complicated nah. AAA game? Image editor? Web browser? Not a prayer. I've never seen a functional body of code that didn't come with a huge accompanying document trying to explain the big picture because the abstractions start to fall apart as things get bigger. Error handling is either ignored or becomes a nightmare that relies on a bunch of macros. Haskell is 30+ years old and still not widely used. Fun ideas, but 99.999% of all code is some form of imperative - even most real world Rust is often heavily imperative, unless you're playing code golf. Keep up the good work!

Excellent post, thank you.

Great video!! This explanation would be absolutely top-notch with a comparison against a functional language (Elixir?) or in a language with functional capabilities (R?, Julia?).

I love this kind of videos! Do you have any plan to create a video or a serie talking about rust? 🤔🤔

Is the method `map` in the class `Functor` the same as `>>=` in Haskell?

Nice video !! I took haskell class at collage but I never imagine I could bring this to python.

This is a big step forward in describing monads. I feel closer to understanding them but still not there yet. Perhaps need more examples, we kind a drifted into talking about maybe and railroad etc. Other examples of monad usage would be needed to help differentiate.

You are correct that operating on monads shouldn't require any boilerplate. In fact, it requires neither that nor close integration. A monad is not a runtime representation or syntax feature, but an abstract category: a relation formed by morphisms defined over structures, with coherence properties. You never arbitrarily define a composite class or modify the language to exploit the monoid formed by integer addition, the functor formed by a list, or the monad formed by async actions. Property based testing, perhaps a runtime protocol, would make more sense. Generally speaking, the separation of behaviors from representations is advised in the functional style, so rather than considering a monad as a tightly coupled object to be converted to, consider it as operations which may be defined over potentially existing objects, if they haven't already. It would be fairly trivial to define the bind operator as a regular function decorated with singledispatch, for instance. In fact, type classes in Haskell are implemented similarly with an implicit parameter gathered by the type system. To make this concrete, consider the list monad: let unit(x) = [x] bind(xs, f) = [y for x in xs for y in f(x)] where fail = [] Extensibility without modification is a powerful principle of software architecture closely related to the Expression Problem. Otherwise, excellent content, as always!

спасибо, Эржан

Typescript is also not a functional programming language. Even though with fp-ts you can do it very comfortably. It provides you all the monads you need - and I find the "exception handling" in fp superior to the "object oriented"

I understood... at least half of that. I can definitely see the use case for them, at the very least, though I admit some of the construction logic is a bit confusing to me(though I've never really understood generics so that might be part of it).

Looks like something that you can find in code that you inherit from developer who suddenly left the company and noone is familiar with. Great content. Thanks.

Love the t-shirt, where'd you get it?

You know, its been a long time since we had a code roast, and that kind of videos is my favorite, i would be happy for another episode, maybe even something that uses openai api or django...

Thanks. I am studying FP and Thinking how to apply some concepts in Python. You rocks.

How about adding an `exception` field to Maybe to save whatever got raised. It could allow better user feedback, and also better debugging because the programmer could see the actual exception.

So, would a dataframe be a monad? Since you can chain functions on a dataframe together and they return the DF

Best explanation of monads I've seen.

Lovely!

Good Stuff....... Thank you..

Small correction on your Maybe monad. Bind always returns `Maybe[U]`. The return type of `Maybe[T] | Maybe[U]` is incorrect.

I watched this and realised i will never be using this and dont want to hear about it again

Now I will be able to answer what a monad is when a relative asks me in a family funeral.

Thanks for the explanation. I agree. I love the predictability of modads, but at this time using them in most python projects is *overkill*.

I think Monads have a usecase when dealing with computationally expensive processes, e.g. fitting multivariate functions. It would avoid spending a lot of time on values that you know would throw an error in later parts of the algorithm.

Have to see this a few more times… 🤓

The problem with explaining monads is that all monads are different, and sometimes completely different from each other. A list and an Async task does not have much in common except that they are monads. It's sort of like a spaceship and a bicycle are both transportation devices, but that's where the similarities end.

Monads! Love the videos, Arjan

Finally a good explanation. I agree with your assessment: it doesn't seem to integrate well with existing "pythonic" code.

Fantastic video

Amazing!

thanks for the vid, just a small error: 6:35 talking about the binary operation of a monoid "it should be associative so the order in which you do the binary operation shouldn't matter..." That's not associativity but conmutativity "a·b" = "b·a" and the binary operation of a monoid does not need to be conmutative. What you meant to say was "the way in which you group operation doesn't matter" that is: " (a·b)·c " = "a · (b ·c) " I guess you wanted to say this. Just to clarify.

Great use of decorators!

Thanks!

I believe the concept of the monad can be very useful. I noticed a lot of people using the exception framework to communicate business validation errors in a DDD architecture. To me this has always been kind of weird, because it is expected behaviour, and not an exception. So switching to another control path is kind of off. I am using Result bags for this, a custom solution to communicate errors, warnings, etc. Monads could be nice for this as well as it doesn't break the main control flow and leaves the exception framework for what it is meant for.

What I'm missing in this video is an explanation of higher-order functions. I'm aware you explained it before, but that was a while ago. I already know the concepts in the video, but if I didn't I would wish I knew what a higher-order function was.

I understand classes, Objekts and functions/methods in Python. So was interested what a functor is. The definition of the map Returns an Objekt or instance of the class. Thus the definition reminds me of recursive functions.

21:07 where is this dunder match documented? I can't find it

Where is your shirt from? :)

in most languages it does not make sense to directly implement monads as a single entity. it can be beneficial only in languages that have function-based dispatching (not class-based) and that can reliably inline higher order functions. errors as values are totally superior to exceptions outside of scripting. i wish there was a way to switch between those on per-session/per-build basis (theoretically, this is possible).

I once try to understand monads years ago and found that Leibniz wrote a humongous book about that theory called Modanology

Libraries: returns, expression - any other ?; Railway programming - usefull for pipelines, etl process

I'm a mathematician who works with monads on a daily basis and yet I've never managed to figure out why programmers, of all people, care about monads. I have to admit, I still have to squint to see the connection to the arcane programming terminology :p A category is just a notion of stuff and stuff that goes from stuff to stuff. Functors are stuffs that go between categories, so it has to be able to do something both to the stuff and to the stuff that goes from stuff to stuff, in some consistent way. Endofunctors go from a category to itself. In particular that means you can do it multiple times to the same stuff. A monoid is a stuff that has a way to "multiply" different copies of itself. For endofunctors that multiplication is composition, and the "way" is to compute the composition to an instance of just a single use.

You said associativity but explained it like commutativity. Didn’t even notice until I was confused by your code that tests for associativity of the monad when I wondered „Huh, where is he switching the order?“. Would be nice if you corrected that. Otherwise nice video.

Oh these are GREAT

cool shirt, where do i get one?

Gods, this will be so useful for my work.

why it's a functional pattern if it contains a type and an instance of this type which comes from OOP, I would say it's a mix of OOP+functional.

90 seconds in and he said the thing!

What do you mean by same shape?

Interesting. I could see a potential use for this in data pipelines where you have succesive transformations and don't want to put error handlers everywhere. On the other hand, it would be nice to know exactly where the error occurred so maybe not 😂

I find the use of the `@maybe` decorator confusing, as it changes the function signature for `parse_int` in a way such that it obfuscates the real return type when the function is called. I like this type of content, though I find it a bit far from real-life coding efforts: I hope Python will provide built-in support for these patterns as e.g. Scala does.

I like Monad as they can be type checked. Regular exceptions are not. Python does not have a `throws` keyword like Java.

Monads the best thing OOP has to offer

Is it not possible to add monads as a concept into python using metaclasses?

Books there? Clean Code, Clean Architecture, Programmer's Brain. The last one?

A monad is just a monoid in the category of endofunctors, what's the problem?

'...functor - the one who functs'©