18 ms·
Python's Type Checking Renaissance
- Majestic121 6y ago'Renaissance' implies a first version, was there any attempt to add typing in older versions of python that were dropped ? As a side note, for a related project linked to typinglike the TypedDict, I found that Pydantic is pretty amazing.
- davidfstr 6y ago> was there any attempt to add typing in older versions of python that were dropped ? Not that I know of. I used "renaissance" to connote an increased positive focus, invigoration, and activity.
- guywhocodes 6y agoIt makes sense if see it as Renaissance for Python, driven by type checking. Or at least the most sense I can find. For me this is true, I've come back to Python because of it for personal projects. I'm still trying to cargo cult myself into thinking that Python async makes any sense however. Why not just go with a genserver abstraction and expose the primitives over this mess we have?
- soledades 6y agoI think the way to parse it is as "a renaissance of type checking, in python," i.e. in the same sense that "The Renaissance" was "a renaissance of classical culture, in Italy."
- 4gotunameagain 6y agoOne could argue that the first version is Archaic, and Renaissance the second attempt
- Galanwe 6y agoI feel like type checking for Python is a plot by engineers bored by a language that is too pragmatic and efficient for their taste.
- guywhocodes 6y agoIt's optional. How about if you are better without it, outperform by yourself and show us the way?
- davidfstr 6y agoblink A plot to do what exactly? Python is amazing, both in its traditional typeless form and in its typed forms. :D Type checking mainly starts pulling its weight in large codebases, or otherwise in long-lived and large programs maintained by rotating groups of people over long periods.
- wodenokoto 6y agoI think it is a plot by engineers frustrated with python codebases in the 10s of thousands of lines of code.
- high_density 6y agowell there are some warts... 1. no multiline lambda function? - breaks flow of writing fp code 2. default-arguments are 'shared' by default? - have to 'break' this sharing by assigning 'None'... - very unintuitive / I don't know any language that does this... Anyway, not everything in python is 'pragmatic', and adding types is one of the first steps in going in the right direction.
- urxvtcd 6y agoNo multiline lambda is a feature. It is convenient enough to use it like in sort(items, key=lambda x: x.size) but really anything much bigger should have a name stamped on it dammit. I find JS largely unintelligible due to lambda overuse and nesting.
- high_density 6y agono it's not -- every other languages other than JS/TS has multiline lambdas... Java 8+, Kotlin, Rust, Scala, C#, ... seems you're confused between "limitation" and "feature"... it's a limitation due to python's "no-braces/indentations-only" policy -- can't think of a way to mark where a lambda starts and ends clearly and cleanly with indentation-only...
- nynx 6y agoPeople who use dynamically-typed languages are finally realizing that having to simulate a type-checker mentally is a bad idea. Also, many statically-typed languages do not take advantage of the flexibility a good type-system can provide. For example, C, C++, Java.
- rational_indian 6y agoBetter late than never.
- kashif 6y agoJava is not a good type system at all - Haskell is.
- nesarkvechnep 6y agoThat's what the OC said.
- kashif 6y agoAh my bad. I misread that.
- SAI_Peregrinus 6y agoThey listed those three as examples of languages that don't take good advantage of having static typing.
- BiteCode_dev 6y agoIt's a trade off. You don't want to write all your types for quick scripts, data analysis and small web sites. As a trainer, experience teaches me that it's also much easier for beginners. Now, types are nice to have for bigger code bases, but that doesn't mean there is no place for a language with optional typing.
- smlckz 6y agoThere is Hindley-Milner type inference. Don't need to type types in most cases. See, for example, OCaml.
- antipaul 6y agoWhat’s the big deal about type checking? I haven’t worked on large code bases, only run small ML analyses on tabular data. Can someone be concrete with the advantages and use cases?
- nynx 6y agoWhen you're writing code in a dynamic language, you're still using a type-checker; it's just in your head instead of the computer. This is a fantastic waste of mental energy and working memory.
- tzs 6y agoI'm not so sure about that. When I'm actually using variables in expressions or as arguments to functions, as opposed to when I'm declaring them, the code is essentially the same in a typed language and in an untyped language. Realizing when I'm writing that the expression I'm about to type is wrong because the types do not match still depends on my doing type checking in my head. It seems to me that the case where typed wins is when my in-head type checker fails and I write something that is invalid. The typed language will catch that failure at compile time. The untyped won't catch it until runtime (and maybe not even then). For the typed language, an IDE might be able to catch a mistake immediately so I don't have to wait until compilation to find it, and that immediate feedback might prevent me from writing dozens more lines with the same mistake, but even then I was using my in-head type checker to write that first mistake for the IDE to catch.
- valenterry 6y agoYou are only describing a small part. Here's another example. You are using a library and call a function that expects a List[Foo] as argument. Now you check in the same library if there is a function that returns a List[Foo] or at least a Foo. So you find it and are good. Without types, now you have to read the documentation which is much much slower than having your IDE help you out. And yeah, sometimes variable names and function names are sufficient, but in my experience they are very often not. Also, what happens when you update your library to the next version and it changed what it expects and returns? Now you either have a typechecker that checks _all_ the code, even the one you haven't written... or you have to do it yourself. For each version. And each library.
- brundolf 6y agoI sure hope things continue to progress, but right now, having come from TypeScript, Python is quite a ways behind when it comes to static types. A few major differences: - Library support for static types is not very good. This can be fixed, of course, but it's also very hard to fix in a concerted way. It'll just depend on the community getting on board. - The syntax is limited. There isn't proper support for declaring generics, you have to declare a separate TypeVar, as a Python variable, somewhere else in scope and it just... gets used to approximate a generic. It mostly works, but sometimes it doesn't, and it's very unintuitive and awkward. And then concepts like Callable, Union, TypedDict, and Optional don't have dedicated syntax for readability; they're generic types that you have to import and parameterize. Etc. - Support isn't great for highly "dynamic" data. TypeScript gives you powerful features for reasoning about dynamic property-sets of objects (dicts), combining and separating them, duck-typing, doing really complex inference, etc. These features in Python are usually some combination of unreliable, third-party, syntactically awkward, and so on. - Inconsistency between different type-checkers. You'd think the fact that Python has standardized type syntax would help with consistency, but what it actually means is that everyone gets to define their own semantics for the same syntax. Different checkers mostly orbit around the same semantics, but there are always gaps. So for example, MyPy does a pretty good job of being strict and smart, but it's really slow. So you'll end up using an IDE-optimized checker for development, like Pytorch, but Pytorch will allow some things that MyPy doesn't and not allow some things that MyPy does. So you can use your IDE to get most of the way there, but you always have to remember to run a "real" type-check before you commit, or you may break the build in CI. I should point out the one big advantage that Python has here: unlike TypeScript you don't need a build step, because Python interpreters can parse (and throw away) the type annotations. That's pretty nice, especially for gradual adoption/casual typing of scripts. All of the problems (except maybe the syntax) are solvable, and I genuinely hope they get solved. For now, if you stick to primitives and core or class-based data structures you'll have a great experience with Python types. If you do anything more complex, the results will be mixed. This is of course much better than nothing, but it could be a lot better still. If you're picking between typed Python and TypeScript for a new project, it's worth factoring in.
- davidfstr 6y ago> Library support for static types is not very good. Granted. I do think it's improving over time. For example the django-stubs project is a really nice addition to the regular Django distribution: https://pypi.org/project/django-stubs/ https://pypi.org/project/django-stubs/
- stereo 6y agoFor those who have never used type checking in Python, what's a good introduction?
- high_density 6y agoread mypy docs, but use google's pytype... pylance is much stricter, but it seems to have a lot of false-(+)s from what I've used, the implementation varies wildly...
- aodin 6y agoFor the actual type annotations themselves, the official docs are great: https://docs.python.org/3/library/typing.html https://docs.python.org/3/library/typing.html Know that the signatures vary with version, so select the correct one. To check these annotations, you'll need a third-party type checker somewhere in your build process. I use pylance with VS Code, as it can detect errors as I type: https://marketplace.visualstudio.com/items?itemName=ms-python.vscode-pylance https://marketplace.visualstudio.com/items?itemName=ms-pytho...
- tgb 6y agoThis doesn't seem like an ideal introduction. I don't think it actually covers type checking at all (besides the note at the top) since that's relegated to third parties.
- ThePadawan 6y agoTo learn how to use it, I would recommend setting up your IDE (I recommend VS Code) to use mypy. Then just open a small script you know well, let's say less than 200 lines, and check what "missing" type annotations mypy complains about. For most cases, the IDE will be able to tell you what it recommends, and from then you can start reading the docs for specific types for a deeper dive.
- dharbin 6y agoPylance is much faster and can index a large virtualenv without issue, IMO
- cjdell 6y agoWhen proponents of dynamic languages tell me "that's what the unit tests are for", I say good for you that you have unit tests. However a unit test may simply inform you that you have something wrong with your program, whereas a type checker will (more often that not) show you exactly where the error is, and usually as you were typing it. Reality is that unit tests and type checking both have their place even if there is overlap between the two. Type checkers can however remove the need for trivial and repetitive unit tests.
- david422 6y agoThe type checker may as well be one big automated unit test. For all the time a developer spends not writing out types, it then has to be made up in manually written unit tests.
- dehrmann 6y agoI'd rather my IDE flag the type error before going to the effort of running a unit test. That, and I never have to ask myself what type a parameter is.
- deleted 6y ago[deleted]
- avolcano 6y agoI'm excited about Python's typing potential. I recently rewrote an API from TypeScript to Kotlin since I am fairly unhappy with the server-side TS ecosystem, but ran across https://github.com/tiangolo/fastapi https://github.com/tiangolo/fastapi when exploring options and really dig it - seems to be by _far_ the lowest-ceremony way to make an HTTP API with static types that integrate with parsing/validation (something TypeScript is still really bad at, unless you bring your own runtime typing libraries...).
- manojlds 6y agoMore specifically, Pydantic. It's great!
- avolcano 6y agoAh, thanks, I hadn't realized that's the underlying library! Runtime validation of types is absolutely the largest missing feature from TypeScript, IMO, so Pydantic is really impressive to me.
- ensignavenger 6y agoYes, I have been using Pydantic with my Django projects, and it is great!
- mattanimation 6y agoFastAPI got me turned onto Pydantic, both are awesome!
- marliechiller 6y agoI use pydantic all over the place now. FastAPI was what introduced it to me. Its really nice to be able to take advantage of the autocompletion when utilizing pydantic models rather than a straight dictionary. Now, in a lot of instances I use the pydantic models instead of dictionaries as it feels more explicit
- giovannibonetti 6y agoHave you taken a look at https://rocket.rs https://rocket.rs?
- deleted 6y ago[deleted]
- toolslive 6y agohttps://i.redd.it/umtx1gc28eb41.jpg https://i.redd.it/umtx1gc28eb41.jpg
- iso8859-1 6y agoI am blind, what does the image show?
- toolslive 6y agotwo people, both solving a puzzle. The first person (static typing) has some pieces and looks at the second person (dynamic typing), who shouts "finished!" but the puzzle is completely wrong.
- true_religion 6y agoThe funny thing is that when you look at the pieces on the static typing side there is no way to solve the problem at all. For example, the piece which has two legs of the giraffe should to at the bottom most position but it doesn't have a single straight edge so it can't be a bottom most piece. A deeper message is.... Dynamic typing: Solve your problems but leave bugs because you moved too fast. Static typing: Discover that there is no sound solution, then have to manually cut and glue pieces together to satisfy your type checker.
- oblio 6y agoYou do know that almost every language with a static type system has a hatch, right? Object, Any, whatever.
- zappo2938 6y agoI recently started working through python tutorials in videos, books, and blogs on statistics and machine learning. As a front end developer who has over the last several years started to adhere to using immutable objects and functional programming in JavaScript and subsequently TypeScript, which doesn't matter as much from my point of view. All the bugs that arise in the python tutorials from data mutation and side effects in class methods (looking at you inplace=True in Pandas) is what disturbs me the most about my first exposure to Python. I really hope to start seeing some immutable architecture and structuring in Python more than type checking, as my first impression.
- booleandilemma 6y agoI'm glad that strong typing is getting popular and it's not just for weak minds anymore. But (for new projects anyway) what's the point of using Python with types instead of using a real typed language like Java or C#?
- high_density 6y agothe point of using python? ML libraries! that's the 99% case of python-only case. Other than that, I actually don't know... a lot of other languages now have interpreted-dev envs... (btw, I'd rather use Kotlin than java --- it's like Java++)
- booleandilemma 6y agoThis is the only reply to my question that makes sense, thanks! All of the other replies are subjective.
- pjmlp 6y agoEven there, many of those libraries are written in C, Fortran and C++. On can use any bindings to those libraries, including using modern versions of Fortran and C++ directly.
- aldanor 6y agoNot necessarily ML, more generally - numeric, as in NumPy/scipy/pandas, variety of optimisers, plotting and reporting tools and whatnot.
- jasonpeacock 6y agoThose are compiled languages, while Python is an interpreted language. You're comparing apples and oranges. Interpreted languages are quick to develop/quick to learn, and with the support of typing that same "quick prototype" can easily mature into a full application without needing to be re-written in a "real" language.
- dehrmann 6y ago
- Decabytes 6y agoI really like being able to add types in Python. But I wish it was more integrated into cPython, so I didn't have to call mypy on my file. I admit it's been awhile so maybe it's changed, but I like how in Racket I can just add types, change my language to #lang typed/racket and then boom we are type checking.
- currymj 6y agois there a high quality introduction to Python type checking for people who already know Python? maybe something very opinionated about how you're supposed to use the features, sort of a "Hitchhiker's Guide to Type Checking". if I Google around I find either documentation or fairly low quality blog posts.
- travisjungroth 6y agoIt’s going to be in the next edition of Fluent Python, but that says September release. Docs and the relevant PEP are your best bet for now.
- staticautomatic 6y agoI honestly think the Pydantic docs are a high quality introduction to type checking in Python.
- mixmastamyk 6y agoI like this site for tutorials of just about the right length: https://realpython.com/python-type-checking/ https://realpython.com/python-type-checking/
- Geminidog 6y agoWith pycharm you don't even have to run the code. Your type checker will highlight all errors real time.
- MR4D 6y agoGiven that Python type checkers are becoming all the rage, will we ever see a compiler that actually uses those types to increase performance?
- O5vYtytb 6y agoYes: https://github.com/python/mypy/tree/master/mypyc https://github.com/python/mypy/tree/master/mypyc
- mixmastamyk 6y agoCython has been around forever.
- parham 6y agoWhy would you add type checking to Python! Using another language with static typing would be the solution. It’s not like the browser where you have to use Javascript so you add another layer to your toolchain.
- jsmeaton 6y agoBecause you’re a python shop that wants some extra safety. Because you’ve got an existing code base. Because you want the benefits of a dynamic language with some of the benefits of type checking. Because the editor experience is way better when you selectively sprinkle in some types. Because you can choose how deep you go on types depending on the impact. We have avoided countless production issues just by annotating when something is Optional or not.
- HelloNurse 6y agoBecause type annotations are medium to high value documentation for your users. Because stating the type of something important can expose design issues (e.g. "wait a minute, it isn't always a X! in case Y it can be a Z! Forbid or support?") before they become a serious problem.
- pansa2 6y agoBecause someone suggested it, and Python’s main design principle nowadays is “include as many language features as possible”.
- anhner 6y agoSo that people who want to use it, can? You don't have to use it if you don't want to.
- pansa2 6y agoPython’s philosophy is supposed to be “one obvious way to do it”, not “you decide which way to do it, it’s optional”.
- frusciante19 6y agowait until they find out there are such things as IDEs which don't make you go to the command line to "run the type checker"...
- mpweiher 6y agoHmm...the basic development cycle described is horribly broken: Write a bunch of new code. Repeat 4-6 times, in rapid succession: Run program to manually test. Find basic error (like a missing import). Fix basic error. Debug/fix deeper errors in the new code With this, just about anything will help, including static typing. The problem is the following line Run program to manually test. You should avoid manual testing at (almost) all cost. Write an automated test instead. It is effort your are expending anyway, why throw it away?? And while you're there, write the test beforehand, to clarify the specification of the code you're trying to write. With such a TDD cycle, those problems wouldn't have happened. Repeat Write a test Make it pass [Commit] Refactor Commit Mind you, static typing can still help in this scenario, but not in getting the code to work and helping it remain functional. It helps as verified documentation that eases understanding when reading the code.
- mixmastamyk 6y agoYes, my first test is always running pyflakes, which finds the most egregious bugs quickly.
- anamax 6y agoAs is typical in these discussions, folks are confusing representation with kind. "type" systems help with representation errors, but those tend to be easy to find without static typing. One problem with static typing is that I'm forced to address all of the representation issues before I can do any testing. That "premature optimization" is hell on incremental development. Another problem is that static typing tends to encourage languages which are designed around making type checking easy. That impacts program design.
- deleted 6y ago[deleted]