7 ms·
Type hinting sucks
- emrah 4y ago> Type hints are great! But I was playing Devil's advocate on a thread recently where I claimed actually type hinting can be legitimately annoying, especially to old school Python programmers. > TL;DR Turning even the simplest function that relied on Duck Typing into a Type Hinted function that is useful can be painfully difficult.
- wscott 4y agoI was expecting the article to get to the end and have him invent an massive type system that allows you to pass any two types as long as your can add them together. Which, of course, is what it did at the beginning.
- pwdisswordfishc 4y agoIt seems the problem stems from a poorly defined purpose of the API, which is to say, from this being a shitty example. Why would someone use slow_add instead of the plus operator anyway?
- avidphantasm 4y agoYeah, it seemed like a straw man example. For me, type hints are most useful as a sort of meta domain specific language. Type hints can be hard to get right for abstract data types, and it’s often not worth it (and I rarely write ADTs in Python, relying on what the rich standard library offers). I find type hints to be very useful for “annotating” domain-specific code that carries out my application’s “business” logic. This rarely includes things like basic arithmetic functions.
- flohofwoe 4y agoI'm usually a strict proponent of static typing, but I discovered that for the things I use Python for (mostly quick'n'dirty cross-platform scripting, not "real programs") it's indeed quite a nuisance mainly for non-trivial cases where the type hints looked entirely too complex and 'dominated' the code way too much resulting in (paradoxically) less readability. In the end I quickly abandonded my experiments and returned to no type hints. I also find it kind of hilarious that type checking isn't built into the standard python interpreter, but you need a separate tool instead. It took me a couple of wasted hours until I discovered that my carefully type-hinted code was not actually type-checked, and after discovering that I was baffled that I can't just do something like "python --type-checked bla.py". TL;DR: for many use cases, Python's duck typing is a feature, not a bug :)
- MaxBarraclough 4y ago> I also find it kind of hilarious that type checking isn't built into the standard python interpreter, but you need a separate tool instead. Agreed, typically you don't add new syntax to a language only to silently ignore it. I think the plan may be to add runtime checking to the Python interpreter later.
- bsenftner 4y ago> I think the plan may be to add runtime checking to the Python interpreter later. And slow the run time back down several versions.
- flohofwoe 4y agoI don't think people select Python for it's performance ;) But the type-checking could be a command line parameter, used only during development.
- mfgs 4y agoIt already can be used for development, IDEs such as PyCharm perform quite effective live type checking based on type hints.
- Jasper_ 4y agoNote that PyCharm and mypy's semantics differ in some key ways: https://youtrack.jetbrains.com/issue/PY-54411/Type-checkers-should-move-towards-requiring-the-optional-type-to-be-made-explicit https://youtrack.jetbrains.com/issue/PY-54411/Type-checkers-.... https://youtrack.jetbrains.com/issue/PY-24834/Support-stricter-handling-of-typing.Union-types-similar-to-that-in-mypy https://youtrack.jetbrains.com/issue/PY-24834/Support-strict...
- gpderetta 4y agoUsually static types allow for better optimizations, but this is python, so the implementation is probably maximally pessimal.
- 2h 4y agothe problem is not type hinting. the problem is people trying to use a single function for any type. as can be seen in the article, that is extremely difficult and error prone. so just dont do that: def slow_add(a: int, b: int) -> int: def slow_add_float(a: float, b: float) -> float: OK its not as elegant or pretty, but its really clear what is going on. the problems described in the article crop up because people are trying to have the interpreter intuit what the programmer is wanting, when what the programmer is wanting might be ill defined or simply invalid.
- mjburgess 4y agoSo because the type system is far far far far less polymorphic than the underlying code, we're to monomorphize the code manually? Angry rant: maybe mypy (and its PEPs) can be designed well? Maybe tacking java on top of python was a terrible idea, and python is much better than this awful type system. cf. the syntactical catastrophe that is "protocols". Consider that to type `+` correctly, you're writing >dozen lines of python. This is mad, and the type system shouldnt have been designed by people this inexperienced with type systems.
- maweki 4y agoBy the fourth attempt the author seems to be half on the way to inventing typeclasses.
- Laaas 4y agoIt seems to me that the author isn't comfortable with static typing. The obvious solution (which might be impossible in MyPy?) is to have a two-parameter type class (protocol in MyPy?) of the lhs and rhs, and have the output be of an associated type. That should cover almost all cases.
- nigamanth 4y agoTbh, for smaller projects I feel this way. But when you're in a big workspace with lots of files and lots of variables, you forget after a while.
- speed_spread 4y agoGroovy taught me that opt-in typing always sucks. Either a language is statically typed from it's inception or it's not. Type hinting is like putting up doors with locks around a house that has no walls to begin with.
- simoncion 4y agoFor what it's worth, Erlang's dialyzer definitely does not suck.
- doomslice 4y ago… ten minutes later.
- realusername 4y agoWhen it works yeah, sometimes good luck figuring out what it wants
- jitl 4y agoIt’s done wonders for quality in JavaScript land
- bqmjjx0kac 4y agoI really like statically-typed languages where you can let the compiler fill in the blanks. Like Rust or Haskell at the farther extreme.
- est 4y agoOpinion: people don't actually need rigorus typing system, what they need: 1. preven type/interface errors at runtime. 2. get auto-complete for unfamiliar objects or function signature.
- tremon 4y agoThat's true, people don't need rigorous typing systems. However, 1. the way to verify types before deployment is usually through a rigorous type system. 2. auto-completion of type signatures is massively easier if the language has a rigorous type system. It's the tools that people use that need a type system, not the people themselves.
- The_Colonel 4y agoSure, but the experience tells us that the best way to achieve these goals is through a rigorous type system.
- thfuran 4y agoWhat do you think solves those better than a type system?
- jerpint 4y agoTypehints are a great way to document code. If the type hints get too complex they can be cumbersome though. For example passing an int or None if the value doesn’ exist
- chrismorgan 4y agoI find this an interesting example to ponder, because I think it actually reveals a weakness in Python’s data model, more than in its type hinting—a weakness inherent in class-based programming that is also found to a considerable extent in just about every dynamic language I can think of, but which is made particularly obvious in Python’s operators. The cause of most of the trouble here is that `lhs + rhs` isn’t simple sugar like a reasonable person might imagine if they don’t think through the implications long enough; but instead, turns into something like this terrific mess: def +(lhs, rhs): lhs_type = type(lhs) rhs_type = type(rhs) if do_radd_first := issubclass(rhs, lhs): if hasattr(rhs_type, '__radd__'): output = rhs_type.__radd__(rhs, lhs) if output != NotImplemented: return output if hasattr(lhs_type, '__add__'): output = lhs_type.__add__(lhs, rhs) if output != NotImplemented: return output if not do_radd_first and hasattr(rhs_type, '__radd__'): output = rhs_type.__radd__(rhs, lhs) if output != NotImplemented: return output raise TypeError(f'unsupported operand type(s) for +: {lhs_type.__name__!r} and {rhs_type.__name__!r}') (This is almost certainly imperfect in details, and may be imperfect in larger pieces; it’s a quick sketch based on memory from about nine years ago, plus a quick check of https://docs.python.org/3/reference/datamodel.html#object.__radd__ https://docs.python.org/3/reference/datamodel.html#object.__... where I had completely forgotten about the subclass thing. But I think it’s pretty close. Good luck figuring any of this out as a beginner, though, or even being confident of exactly what it does, because there’s no clear documentation anywhere on it, and the reference material misses details like the handling of NotImplemented in __radd__, so that I’m not in the slightest bit confident that my sketch is correct. If I still worked in Python, I’d probably turn this into a reference-style blog post, but I don’t.) And why is this so? Because with class-based programming, the only place you can attach behaviour to an object or type is on that object or type. Which means that for operator overloading, it must goes on the first operand. But there are many legitimate cases where you can’t do that, and not all operators are commutative (e.g. a - b ≠ b - a in general), so you pretty much have to support the reflected operators, rhs.__radd__(lhs) instead of lhs.__add__(rhs), and then you get worried about subclassing problems, and it all just gets painfully complicated. Is it any wonder, then, that a typing system would have trouble with it? Certainly Addable/RAddable are the wrong level of abstraction: you need a bound that covers both of them in one go, and I doubt you can do that with Protocol or similar, since you’re defining a pair of types, where one has this method, or the other has that method (and good luck handling NotImplemented scenarios). I imagine it needs to be built into the typing module as a new primitive. —⁂— By contrast, in Rust, you might possibly start with a concrete type (the first attempt), but you’d be more likely to just go straight to the fully-correct solution that this example is never able to reach in Python, of being generic with an Add trait bound <https://doc.rust-lang.org/std/ops/trait.Add.html https://doc.rust-lang.org/std/ops/trait.Add.html>: fn slow_add<A: std::ops::Add<B>, B>(a: A, b: B) -> A::Output { std::thread::sleep(std::time::Duration::from_secs_f32(0.1)); a + b } Certainly Rust does have the advantage of having started with and built upon a type system, rather than retrofitting it. But note how this allows you to use different left-hand side, right-hand side and output types (A::Output is here short for <A as std::ops::Add<B>>::Output, meaning “the type produced by A + B”; Output is what’s called an associated type), and threads the types through fully properly. And note more importantly how this doesn’t run into the __add__/__radd__ problems, because the implementation isn’t attached to the single type A, but is rather defined, as it were, for the (A, B) tuple: when you compile it, it’s like you have a lookup table keyed by a (Lhs, Rhs) tuple. Depending on what sorts of additions the left hand side type defines, the right hand side type may be able to define additions of its own. (If you’re interested in how this is done, so that different libraries can’t define conflicting additions, look up trait implementation coherence.) Rust thus demonstrates one solution to the problem this case exposes with using classes in this way: instead of putting data and behaviour together in classes, separate them. (It’s not all sunshine and roses: some things do map to class structures very nicely, so that implementing them in Rust can be painful and take time to figure out a decent alternative, especially when interacting with existing systems; but in general, I find myself strongly appreciating languages with this sort of data/behaviour division.) —⁂— My favourite demonstration of the advantages of separating data and behaviour is actually iterators: • In Rust, when you implement the Iterator trait on your type, you can call any iterator methods on it—built-in ones like .map() and .filter(), but also methods defined in other extension traits, e.g. https://docs.rs/itertools/latest/itertools/trait.Itertools.html#method.interleave https://docs.rs/itertools/latest/itertools/trait.Itertools.h.... This is why iterators are very popular in Rust: because they just work, with no trouble. • By contrast, in Python map() and filter() have to be globals, leading to messy code reading order and the preferred alternative approach of list comprehensions/generator expressions (which work pretty well, but are more limited, really only covering map, filter and flat_map in their capabilities). • In JavaScript, you get Array.prototype.{map, filter, …}, and those methods are defined as working on any iterator, not just an array, because otherwise it’d be just too painful—but unless you copy the methods you want (like NodeList has done with forEach, but not any other method!) you can’t just chain things automatically, and you can’t add new methods anywhere. • Ruby has a… different approach to all this, but I can’t remember all that much about it and this comment is long enough already.
- aflag 4y agoslow_add is not simple. It reimplements the Python + operator, which is quite complex, specially if you look at it from a type system perspective. As noted, its implementation is very core to the language and it has several edge cases. That's not usually what you do when you create a function. Rather, you have very specific types that you support. It's only "simple" because adding things together is a basic operation. Even if you write a function that encapsulates some funky arithmetic. You will probably be using more operations, to the point that anything but numbers will really make sense. That said, yes, adding types to an API that wasn't implemented with a typing system in mind can be challenging. But I feel like you get a better API at the end of that exercise.
- mynameisvlad 4y agoThis is exactly what bothered me with this article but I couldn't put it into words: > That said, yes, adding types to an API that wasn't implemented with a typing system in mind can be challenging. The title is just straight up clickbait. Type hinting doesn't suck, adding types to an existing project used by users who have accustomed themselves to an untyped system that lets them do whatever sucks. If this were typed as int, int from the get go, the article would be a hell of a lot shorter.
- JonChesterfield 4y agoIs anyone able to summarise how python got to this point? I haven't used it seriously since the tail end of the 2.7 woes but liked it a lot back then. Good native dictionary type and first class closures worked for me. Scattering of unit tests. The type annotation idea always seemed inconsistent with the language to me so I've ignored it. In particular I'm wondering if this is a consequence of the rumours that python scales badly - someone changes some module far away, runs the massive test suite and it passes, checks in their code, and your code promptly blows up because the interaction wasn't tested. There also seems to be a current enthusiasm for statically typing everything on hackernews which might be reflective of the wider industry, possibly making python acceptable collateral damage as it was on the wrong side. That's my conjecture though, would love to hear from someone closer to the game how type hints became such a big deal in the python world.
- creata 4y ago> I'm wondering if this is a consequence of the rumours that python scales badly My (completely uninformed) guess is that people want to statically type all the things not because it improves correctness, but because it makes editor facilities like code completion and jump-to-documentation so much more effective.
- d1l 4y agoWeb devs, same reason asyncio got railroaded through. Python is increasingly trying to cater to the extremely online and loud webdev crowd. Living in an ivory tower is fine if you're inside it.
- Etheryte 4y agoAt first glance this is a pretty odd take, would you mind elaborating? I wouldn't say Python devs and web devs have any more overlap than web devs and some other random group of devs you could select, so why is it the case here? It's always easy to say that it's [outgroup]'s fault that [ingroup] is going sour, but is it really the case?
- pansa2 4y agoMy understanding is that Python is used by millions of people to write programs measured in tens of lines, and by tens of large companies writing millions of lines. Many in the first group are happy with dynamic typing. Those in the second group would have been better off with a compiled language but are too invested in Python to switch to something different, so are trying to morph Python itself into something different. Unfortunately, the way Python’s governance works, all decisions about the future of the language are made by people in the second group. For example, IIRC the Python Steering Council is composed entirely of developers working for large tech companies - no scientists, finance workers etc at all.
- solarkraft 4y agoI think (optional) static typing is important for "serious" programming, so I've been experimenting with Python's type hints after of my good experiences with TypeScript (which fundamentally works in the same way). Oh boy, what a mess, especially if you have to keep compatibility with 3.7 (which has slightly different types for builtins). The ecosystem is definitely improving and I do think the benefit will eventually be worth it, but this example illustrates (albeit in a somewhat exaggerated way) why I would advise users to move cautiously. IDE completions are a good benefit for a start, it'll probably take a while until you can support strict checking (errors when types don't match).
- satya71 4y agoFor compatibility, use typing_extensions. You can use most of the latest typing features with older versions of Python.
- lordgroff 4y agoI'm coming more and more to the conclusion that if you need typing for Python, you might as well go full throttle and use Java (maybe TS/Deno?), unless you're in a specialized ecosystem like data science and you have no choice.
- alexmolas 4y agoIsn't that a problem of types in general? I mean, you have the same problem in a strongly typed language.
- garganzol 4y agoType hinting does not suck. But missing support for method overloading does.
- ghostwriter 4y ago* Declare `slow_add` to be used with instances implementing `__add__` protocol only. * Declare that `concat` should be used to achieve a similar result with other types implementing `Foldable` protocol. * Carry on.
- atorodius 4y agoThis really triggered me for some reason! I hope it's a joke :) (I am not sure exactly what triggered me but I think it's because at least to me it's very clear that in this very generic case, you just want to annotate with `Any`. Why go to such great lengths?)