Тёмный

Carl Meyer - Type-checked Python in the real world - PyCon 2018 

PyCon 2018
Подписаться 21 тыс.
Просмотров 59 тыс.
50% 1

Speaker: Carl Meyer
You've heard about Python type annotations, but wondered if they're useful in the real world? Worried you've got too much code and can't afford to annotate it? Type-checked Python is here, it's for real, and it can help you catch bugs and make your code easier to understand. Come learn from our experience gradually typing a million-LOC production Python application!
Type checking solves real world problems in production Python systems. We'll cover the benefits, how type checking in Python works, how to introduce it gradually and sustainably in a production Python application, and how to measure success and avoid common pitfalls. We'll even demonstrate how modern Python typechecking goes hand-in-hand with duck-typing! Join us for a deep dive into type-checked Python in the real world.
Slides can be found at: speakerdeck.com/pycon2018 and github.com/PyCon/2018-slides

Опубликовано:

 

13 июл 2024

Поделиться:

Ссылка:

Скачать:

Готовим ссылку...

Добавить в:

Мой плейлист
Посмотреть позже
Комментарии : 74   
@polypus74
@polypus74 6 лет назад
5:17-6:38 should be required viewing for every programmer. well done
@lawrencedoliveiro9104
@lawrencedoliveiro9104 3 года назад
17:19 It became part of the standard “typing” module in Python 3.8.
@cr8onpetty
@cr8onpetty 5 лет назад
Very clear, concise, and well rounded intro to typing. Thank you!!!
@matszczygiel5530
@matszczygiel5530 6 лет назад
Great talk, excellent slides, very clean.
@luvelyvids
@luvelyvids 4 года назад
An almost perfect overview of Python type annotations and how to use them to type check Python code. Well done
@MartinThoma
@MartinThoma 4 года назад
Awesome talk! 12:10 typing.overload 17:10 typing.Protocol - docs.python.org/3/library/typing.html#typing.Protocol ... I wrote a complete list of what is said when, including when he starts talking about protocols. Then RU-vid autoplay deleted my draft -.-
@loic.bertrand
@loic.bertrand 3 года назад
Oh no.. that's sad :'(
@pyadaaah9866
@pyadaaah9866 3 года назад
Thanks for your excellent, comprehensive introduction to type annotations in Python! It let me become aware of how well the static type checking of functions / methods provided by said type annotations (resp. mypy / pyre and others) and the dynamic type and range checking of variables / attributes provided by Pyadaaah complement each other!
@kindjacket
@kindjacket 6 лет назад
how was this deck made?? looks awesome
@xucongzhan9151
@xucongzhan9151 5 лет назад
Very good talk and cool slide deck. Thank you~
@lazerbro
@lazerbro 6 лет назад
Excellent talk! Thank you!
@user-vv5lv4cy8k
@user-vv5lv4cy8k 6 лет назад
I am curious about how the animation implemented in his PPT.
@quentinsf
@quentinsf 6 лет назад
Very nice, clear, and practical intro
@kzorbadelos
@kzorbadelos 3 месяца назад
Amazing presentation!
@liron19
@liron19 6 лет назад
good talk, thank you!
@brambeer5591
@brambeer5591 4 года назад
Thanks, really useful!
@JackQuark
@JackQuark 5 лет назад
This is very helpful.
@SerapioSergiovich
@SerapioSergiovich Месяц назад
Nice video shows methods to create a business..
@hongleijiang
@hongleijiang 6 лет назад
"PEP 544 -- Protocols: Structural subtyping (static duck typing)" will appear in Python 3.8 ?
@BryanChance
@BryanChance 2 года назад
Oh yes!! Can we get rid of the indentation/white space significance too?
@clip6399
@clip6399 6 лет назад
The slides page is 404
@Zergosss
@Zergosss 5 лет назад
As someone else said, there are mental costs associated to writing and reading code with type annotations. Even watching this video is a cost. It's probably worth it only for a small percentage of the code bases, therefore know what type annotations are and why they're useful, then come back when you'll actually need them.
@TheShanks
@TheShanks 5 лет назад
Sure, reading a keyword that indicates a type takes mental effort. However, figuring out the typed from the doc or even inferring it from the code takes at least as much mental effort. And that doesn't even include the mental effort that comes from testing/debugging typeless code.
@aoeu256
@aoeu256 5 лет назад
They are great for when you'll be appending an object to a list.
@dlf_uk
@dlf_uk 2 года назад
Agreed. I don’t mind the overhead for straightforward functions and variables, but if I have to write multiple fake overload functions just to say what the argument types are, that’s horrendous for readability.
@ZuLuuuuuu
@ZuLuuuuuu 6 лет назад
So should we write Python always with type annotations now? For small scripts I would imagine there is no need, but can we say for medium/big sized code bases we should always write type annotations?
@cjm46543
@cjm46543 6 лет назад
The value is probably less for small scripts, but so is the investment. Even there I find it handy, mostly as a form of documentation. But certainly I think the benefits are more pronounced in bigger codebases and with larger development teams, where it's more likely that people will have to frequently touch code they aren't familiar with.
@Auslegung
@Auslegung 6 лет назад
I'm not being sarcastic but an honest answer is, "You add type annotations to every function that you want to be made safer." If you have an unimportant 6-line script you may feel ok without the added safety. However, I insist on doing something 100% until I know the rules, and therefore when to break them, so I suggest you add type annotations to your cURL scripts ;) until it's drilled into you.
@davenirline
@davenirline 6 лет назад
If I were to make the coding standards of a team, very much yes.
@yvrelna
@yvrelna 5 лет назад
No you shouldn't. Python isn't a static language, and type annotations aren't meant to convert python to a static language. There are mental costs associated to writing and reading code with type annotations. For smaller scripts, it's most likely quite redundant to add type annotations because you can see the code being called from just one place, and it's often quite obvious what those types are. Type annotations are meant for the situations where you need to clarify, in the a form of executable documentation, for when things aren't obvious from context. Even in larger code bases, the mental costs with trying to satisfy the type checker and with trying to read code that are full of junk annotations can be quite significant. You aren't paid when you get your types right, you get paid when you ship features. So use annotations sparingly, use it when it helps you reduce mental overhead, and don't fight the battle if you don't benefit from it. I often liken type annotations to comments. If you have to add comments/annotations to make your intent clear, then your code probably was not structured well enough to make it obvious. At the end of the day, sometimes it's just easier and more precise to annotate, and annotations have the advantage of being actually checkable compared to comments, so it's more useful in more cases. However, you should be wary about any code conventions that requires you to annotate everything. As a general rule of thumb, you should try to architect the system so there are clear boundaries different parts of the system. These parts is generally quite chunky (not one class/methods == one part), these parts are usually a constellation of many classes and modules that fits within a theme. Type annotations and checking are mostly useful in the boundaries of these chunks, as it specifies the contract between one part of the overall system and another, but it's usually just noise and distraction within a single part.
@aoeu256
@aoeu256 5 лет назад
Write type annotations if: your function or method is long, you don't have doctests, you don't have default arguments. Also nearly always use Sequence, Number, and str never use int, list for maximum code reuse.
@phudlow
@phudlow 4 года назад
12:55 What if get_foo is called with an invalid foo_id, e.g. one that is negative / out of range, or with an id of a non-existent foo? In that case the get_foo method would return None. Should the second @overload definition of get_foo have a return type of Optional[Foo]? Manual type checking before referencing the id property of the return value seems necessary.
@KanadaKid19
@KanadaKid19 4 года назад
It depends what your real-world scenario is. Your function might automatically instantiate a new Foo whenever one didn't exist with the given ID (like a defaultdict does, for example), but an Optional[Foo] solution is probably more typical.
@frankt9156
@frankt9156 3 года назад
Why type checking is not dash option on python.exe?
@AngelHdzMultimedia
@AngelHdzMultimedia 5 лет назад
Are Protocols interfaces?
@sadhlife
@sadhlife 2 года назад
go-like interfaces, yes. implicitly implemented.
@archerreilly4881
@archerreilly4881 6 лет назад
git repo don't have 2018 slides, and the upper link can't be opened
@simonmasters3295
@simonmasters3295 5 лет назад
Funny that: I got here via the slides!
@aoeu256
@aoeu256 5 лет назад
Instead of having to define a protocol/interface for everything can't you just it'd be nice to write the type as r: able[render] or r: _.render.
@skepticmoderate5790
@skepticmoderate5790 4 года назад
Well, then you don't know the argument types or return types from render.
@edwingarcia5043
@edwingarcia5043 3 года назад
Uff It feels overwhelming, but types look useful.
@Mahorir
@Mahorir 4 года назад
Instead of typing protocol, why not classes implement python abstract base class(ABC)?
@DrorF
@DrorF 3 года назад
I'd like to know too
@sadhlife
@sadhlife 2 года назад
It's for flexibility. Protocols are accepted implicitly, you don't have to extend every concrete class from the ABC. It's also less code to type.
@benhbr
@benhbr 3 года назад
28:42 Pause for (well-deserved!) applause, alas, in vain
@masakikozuki3214
@masakikozuki3214 5 лет назад
Wow, the slides are available on speakerdeck speakerdeck.com/pycon2018/carl-meyer-type-checked-python-in-the-real-world. I was looking for this in GitHub.
@doc0core
@doc0core 3 года назад
In the real world, eh? So here @23:05 we have a Complex class with int attributes instead of float LOL
@zeppelin0110
@zeppelin0110 6 лет назад
Now is it possible to make this work in real time? Can an IDE like PyCharm or VSCode or Visual Studio offer suggestions in real time for the types? If it would be possible to also force the declaration of data types from the get-go, Python development could suddenly become a language that continues to be enjoyable even after your initial script starts becoming a larger program.
@dachuanlin8625
@dachuanlin8625 5 лет назад
maybe pylint or something can do these jobs.
@chiefir969
@chiefir969 5 лет назад
PyCharm Pro offers type checking during code developing.
@aoeu256
@aoeu256 5 лет назад
Another idea is to code while your program is still running in the console by stopping at a breakpoint that way you get more information than just the type of your variables. You can also in some cases change classes and class methods at run time and reload(module).
@saltrocklamp199
@saltrocklamp199 4 года назад
Pycharm CE now has extensive type checking support.
@olexandrklymenko
@olexandrklymenko 6 лет назад
I’m curious what the font does he use in his presentation.
@cjm46543
@cjm46543 6 лет назад
Titles are Graphik, code is Trim Mono.
@Auslegung
@Auslegung 6 лет назад
whatfontis.com is a great service for this, just take a screenshot of the video and upload it to www.whatfontis.com.
@knpatel86
@knpatel86 5 лет назад
During video, i was curious about the exact same thing and though of searching comments for similar question, and you know what, you did had the same question. lol.. thanks anyway for asking so i get my answer without waiting. :)
@lawrencedoliveiro9104
@lawrencedoliveiro9104 3 года назад
4:01 “Item” and “Sequence[Item]” denote types, but “None” is not a type, it is an instance of the return type. Synecdoche, much?
@DrorF
@DrorF 3 года назад
But None is a type in python... 😕
@davenirline
@davenirline 6 лет назад
I use and prefer static languages a lot but I do dabble in Python at times. Why use Optional or Union!? That's just inviting bugs. Why not just use a single type? Makes code more readable and less error prone.
@bloodgain
@bloodgain 5 лет назад
Because a big benefit of Python is its generic programming, achieved through "duck typing." Many statically-typed languages offer generic programming through other means. Java approaches it by requiring explicit inheritance of interfaces, which must be fully implemented -- and at least throw UnsupportedOperationException, which unfortunately, won't fail at compile time. C++ does this through template substitution, which in practice, is like a (compile time) type-safe version of duck typing, and is a more pure version of generic programming than the interface inheritance model. So to avoid forcing your Python code into an interface inheritance-based, truly type-safe model, sometimes your type annotations need to be more generic. You still gain some benefits from using it, particularly communicating your intention to other programmers. EDIT: For what it's worth, the Protocol approach he demos from typing_extensions is basically the interface inheritance model, but more Pythonic. It shifts the burden to the API that requires the interface.
@user-ip4zu6dp8q
@user-ip4zu6dp8q 5 лет назад
Postmodern fact: with asyncio, multiprocessing and typing, Python is no more a simple language as it seemed before ten years.
@yvrelna
@yvrelna 5 лет назад
Those are libraries and if you don't want to care about them, you don't have to. In its core, Python is a simple language with a standard library that implements everything. In the core python language, type annotations is simply a syntax to add metadata to a variable declaration. async/await are just resumable functions/coroutine, which is simply a generator function under the hood. They are pretty simple concept at the language level. Most of the magic really happens in the libraries.
@RobertLugg
@RobertLugg 5 лет назад
You are exactly correct. Python written following best practices is now nothing too special.
@saltrocklamp199
@saltrocklamp199 4 года назад
These are optional tools that make it possible to scale Python to larger and more sophisticated code bases. You can still write simple scripts without them.
@stumblinzz
@stumblinzz 5 лет назад
The whole god damn point of python was KISS and and readability and to get away from this kind of thing. Im irate because a project I want to inherit has all this garbage in the project and I know its [Facebook PC]Instagram's engineers driving this. Yes your point is valid and I understand why you're doing it, but the implementation readability wise is absolute dogshit. Now I have pointers in python and my code is starting to look like swift and C++ had a baby and I can't tell you how aggravating it is to see this bullshit back in codebase. I feel like this was the kind of thing that Guido Van Rossum would have shot down because it moves away from the original core tenants of the zen of python. Its not complexity that I can't understand that I'm afraid of, its needless readless code that could have been done using the language's native constructs that would have kept this simple. Python is already capable of type checking at the function definition layer, and by implying the type through a default value in the parameters. Even if you are using kwargs or args that only moves your type interpretation down a few lines. This needs to be seriously checked and made into a readable format.
@thoperSought
@thoperSought 4 года назад
at 13:22, with the two get_foo defs that just pass, followed by the real get_foo, I scrolled down looking for exactly this take on the topic. I'm not a professional developer, but this seems very weird to me-kinda anti-python. why use get_foo in the first place, when you can parameterize a function later? why have three definitions for one function, just to define the type?
@saltrocklamp199
@saltrocklamp199 4 года назад
You can put type annotations into a separate "stub file" (extension .pyi).
@davidblewett5703
@davidblewett5703 4 года назад
You realize that Guido is helping lead this entire project? mypy-lang.org/about.html
@samdunggawaxy7314
@samdunggawaxy7314 4 года назад
@@thoperSought yep lost me at that point. I will not support this practice at my company
@user-fu6it4cm7x
@user-fu6it4cm7x 4 года назад
This is horrible compared to Scala.
@Kitulous
@Kitulous 3 года назад
scala was a static language since the ground up, tho, but python was, is and will remain a dynamic language, and this is not even a static type checker, it's type hints
@SerapioSergiovich
@SerapioSergiovich Месяц назад
Nice video shows methods to create a business..
Далее
Хотите поиграть в такую?😄
00:16
Просмотров 686 тыс.
Luiza Rasulova #luizarasulova
00:11
Просмотров 327 тыс.
Talk: Dustin Ingram - Static Typing in Python
21:50
Просмотров 16 тыс.
Type Hints  - Guido van Rossum - PyCon 2015
49:26
Просмотров 47 тыс.