13 ms·

I Want a New Duck (2020)

• neolog 6y ago
  I think this isn't quite what I want, because implicit structural typing has a single shared namespace of methods. If the Ducky protocol says "implementers should have .quack()" and I define FringeScientist().quack(), Mypy will think it's Ducky but it's just a naming coincidence. I want to say which protocols I'm implementing. Mypy can't tell whether my Foo.quack() is supposed to be an implementation of FringeScientist.quack() or Ducky.quack().
  • morelisp 6y ago
    In practice this happens an order of magnitude less than "I want to handle two things that match the same protocol that I didn't write and can't extend" and "I need to pass something that would work fine to a poorly-typed library" combined. That's before we even get into the philosophical issue of, who are you, the lowly library implementor, to say those aren't something appropriate to abstract over in my application?
    • erik_seaberg 6y ago
      How can two objects present exactly the same API without either one knowingly emulating the other? Examples always seem to use one method taking no args, which don’t seem realistic.
      • XorNot 6y ago
        What's missing in this example is stub typing of the output values. Scientist.quack really shouldn't be returning plain strings, and neither should ducky.quack if the data in them is contextually different. Things should only become basic types at the last possible moment.
        • neolog 6y ago
          Which language ecosystems operate like that in practice?
          • morelisp 6y ago
            Go has no subtyping and only supports polymorphism via structural types (and some weak support for untyped constant values). It also has very convenient syntax to define newtypes from primitives (or non-primitives, but this is less commonly used).
      • morelisp 6y ago
        > Examples always seem to use one method taking no args, which don’t seem realistic. This is one of the reasons it is so rare to actually have confusable types in a real program.
      • nemothekid 6y ago
        One thing I did come across once before is when two APIs do something similar such as parsing and provide the same name. A couple of years ago I remember I was working with an API in Go that the function Scan which was `fn(interface{}) -> Error`, which was used to decode SQL values. I pulled in another library which provided that function but in a separate context - text parsing. The interfaces matched, the compiler accepted it, but the application blew up at runtime.
      • SatvikBeri 6y ago
        A common example I ran into is machine learning methods and data transformations such as scaling have have `fit` and `transform` methods, and I'd like to write code that can take multiple such objects and compose them in a pipeline, along with some custom logic. Structural typing makes that easy to typecheck, nominal typing makes it difficult because those two classes don't have a parent class that contains those methods. Extending this, if the class has a `partial_fit` method, sometimes I want to use that, and fallback to `fit` when it doesn't.
        • neolog 6y ago
          Nominal typing just means someone has to declare that LinearClassifier implements Classifier. That could be done by the LinearClassifier maintainer or by the user.
          • morelisp 6y ago
            What languages let the user do this?
            • neolog 6y ago
              Python's zope.interface can do it. Julia's still figuring out their trait system, but some implementations allow that. I don't know Clojure but I wouldn't be surprised to hear if their Spec allows it.
              • morelisp 6y ago
                I don't think either zope.interface or spec is really thought of as a nominal typing system? Certainly not spec. zope.interface of course can late-bind classes to interfaces but only because it's actually a runtime thing and Python late-binds everything.
      • kuschku 6y ago
        Before Android supported Optional, there were dozens of independent reimplementations of the API. All exactly identical, but all in different namespaces and therefore not compatible. And the official one obviously couldn't be used.
      • lmm 6y ago
        Consider an immutable list and a read-only view of a mutable list (that another thread may mutate concurrently). They offer exactly the same API, but it's a major mistake to pass one where the other was expected.
  • coolreader18 6y ago
    Well, that's duck typing - looks like a duck, quacks like a duck, it's a Duck[y]. And while it's obviously hard to extrapolate simple examples like `duck_war` to real-world use-cases, I think signatures being different between methods of the same name on different protocols is generally enough to differentiate between them.
  • orblivion 6y ago
    Don't Golang interfaces work this way as well? (Not that I'm a fan).
  • l0b0 6y ago
    This is only a problem if the `FringeScientist.quack` signature is identical to Ducky.quack: $ cat test.py from dataclasses import dataclass from typing import Protocol class Ducky(Protocol): def quack(self) -> None: "Quack." class FringeScientist: def quack(self) -> bool: return True @dataclass class Duck: quiet: bool = False def quack(self) -> None: print("Quack." if self.quiet else "QUACK!") def duck_war(aggressor: Ducky, defender: Ducky) -> None: aggressor.quack() defender.quack() print("The only winning move is not to play.") duck = Duck() asshat = FringeScientist() duck_war(duck, asshat) $ mypy test.py test.py:32: error: Argument 2 to "duck_war" has incompatible type "FringeScientist"; expected "Ducky" [arg-type] test.py:32: note: Following member(s) of "FringeScientist" have conflicts: test.py:32: note: Expected: test.py:32: note: def quack(self) -> None test.py:32: note: Got: test.py:32: note: def quack(self) -> bool
  • brundolf 6y ago
    This is why Rust makes you explicitly implement a trait for a given type, even if the type doesn't require any new methods to be implemented
  • c6401 6y ago
    That's the point of protocols. It doesn't matter if it's a duck or not, what matters is that it can quack. Like in case of iterable, it doesn't matter what it is, what matters is that you can iterate over it. You can as well rely inheritance if you want.
• morelisp 6y ago
  It's so bizarre to me - though I guess in-line with me finding a lot of decisions around Python 3 bizarre - that after years of Python being one of duck typing's biggest defenders and success stories, mypy went with nominal typing by default. Both Go and TypeScript were right there showing you how effective it was.
  • musingsole 6y ago
    Mypy will only ever make it into a small fragment of Python code. Its existence is more likely an example of the those insistent on type checking crossing the picket line and bringing their last holdout features to the language than it is an acceptance that duck typing is not the way. For the record, duck typing is the way. Mypy is a curiosity to me; I'm happy those who need it are getting that itch scratched; I'm happier it's boxed away from what I work with.
    • throwaway894345 6y ago
      > Mypy will only ever make it into a small fragment of Python code. I don't think this has to be the case. If it got serious investment and the Python community was more open to making ergonomic syntax improvements so type annotations could be natural, we could see improvement. Similarly, I think it would need to figure out a saner solution to finding/loading type annotations, because the current set up and error messaging are immensely painful. I also don't know if Mypy will ever be sufficiently performant so long as it's written in Python (I think people really underestimate the difference between instant feedback and a delay of several seconds). Having good editor support would also drive adoption. If Mypy were able to improve on all of these distinct problem areas, I think more people would opt into type annotations naturally, but it certainly feels like Mypy is being treated as a thing to pacify the people who whine about static typing (of course I don't think that's the real intention, only how it comes across).
    • sseagull 6y ago
      I thought like you at one point, but then jumped into a moderately-sized project that I was unfamiliar with. It took months to get an intuition for what the data was that was being passed around. Inside a function you need modify, you are passed a foo. What can you do with it? Take the length? Add a number to it? You don't know unless you know the type, and then the only place that info exists is in your head. Why not write it down in a way that can be checked and therefore never get out of date, like comments or some weird reincarnation of Hungarian notation? As a bonus: Editors will tell you when you are passing incorrect types to functions before you even run your tests. No more strings accidentally treated as lists! I am now absolutely convinced that types are important, even in python.
• joobus 6y ago
  > With Mypy, you get all the benefits of high-level dynamic typing for rapid experimentation, and all the benefits of rigorous type checking... Except that most typed languages are far faster than python. Retrofitting existing python projects with types makes sense for reliability, but starting a new project with python and types makes less sense, imo.
  • jhayward 6y ago
    Using that logic, it never made sense to start any project with Python, correct? People use Python for its ease and speed of development, and extensive ecosystem. Things that need to be fast can have the appropriate effort expended to execute as fast as necessary.
    • memco 6y ago
      > People use Python for its ease and speed of development, and extensive ecosystem. Of these, I think only the ecosystem stands out anymore. For work purposes I wouldn't be able to move away from Python anytime soon, but if I could pick any language for new code it wouldn't be Python. If I had to pick the closest competitor, Julia seems like a great Python-like language in terms of speed and ease of development with much better typing and performance, but with a less extensive ecosystem. Nim might also fit that bill. If ecosystem was important, Go probably fills a similar space without being quite as steep of a learning curve as some of the other languages.
    • yellowapple 6y ago
      > it never made sense to start any project with Python, correct? Well as a Perl fan I certainly don't disagree with that conclusion ;) But yeah, you're definitely right that there's immense value to being able to quickly prototype.
  • mixmastamyk 6y ago
    Cython is an option when a (potential) 100x speedup is required. Choosing the right algorithm up front, thru the rapid experimentation Python gives, can sometimes make it faster than a real world implementation in a static language, that didn't get enough time to experiment.
• dleslie 6y ago
  It's a wee bit amusing to see the major icons of duck typing, python and javascript, slowly converge on the truth: static typing is a powerful and useful tool for mature projects, that should not be dismissed. Looks like mypy has discovered interfaces. Lovely.
  • war1025 6y ago
    The difference between "Interfaces" as I have seen them used most of the time, and "Protocols" as mypy uses them, is that usually an Interface is generally part of the inheritance hierarchy. A Protocol is just the same structural subtyping (duck typing) Python has always preferred.
    • thechao 6y ago
      These protocols match the old Indiana `concepts` pretty precisely; they appear to be more like Haskell type-classes than interfaces -- they're retroactive, not inherited.
    • deleted 6y ago
      [deleted]
  • 6gvONxR4sf7o 6y ago
    Seems more like a convergence towards gradual typing than to static typing.
    • ledauphin 6y ago
      I think you're spot on here. It turns out that a lot of software development is made easier with static types - but also that some of it is _not_. And having access to both without fighting the language can be very powerful. I use static Python types a lot at the application layer, where things have no need to be particularly extensible and having strict definitions for how things are put together brings clarity. But at lower levels of the system, Python's duck typing really shines because I can solve ten slight variations on the same problem with the same code. The same could be said of Typescript, or Clojure+spec, or really any other language where dynamic types are given to you for free (unlike, say, reflection in Java) but types can be added wherever you find them useful.
• Spivak 6y ago
  This is one of those things that I wish type systems could just do for us. The pattern of every single class in Java being an an interface with an InterfaceImpl is exhausting. Having to fight the type system just for testing seems silly. Just have every class's public bits define an interface and the class's name in code always refer to the interface. Then testers can just implement ClassName with no fanfare and you stop having to write "test-friendly" classes.
• scudd 6y ago
  I can't help but feel like if you want static typing, you're better off recognizing that python is not the right tool for the job. Admittedly I've never done serious work with mypy (or typescript), so I'm approaching the value proposition of dynamic typing at face value rather than experience. However, it seems like the primary benefit of these languages was ease and flexibility, ableit at the cost of structure. Or said differently, adding mypy feels like trying to get out of a trade-off decision. This situation reminds me of a talk Bryan Cantrill gave on platform core values, and as examples he gave his interpretation of the platform core values of languages like C, Awk, and Scala: https://youtu.be/2wZ1pCpJUIM?t=349 https://youtu.be/2wZ1pCpJUIM?t=349 For me, platform core values that stick out for python would be Approachability, Simplicity, and Velocity. I understand the posited value mypy brings to the table, but it feels in contention with the original core values that made python appealing to begin with.
  • seanmclo 6y ago
    At this point, another big benefit of Python is its huge number of libraries. This is really attractive to lots of people who also want static type checking. Oftentimes, the benefits of using Python in a code base that would benefit from static typing outweigh the costs. Especially when tools like MyPy exist, which aren't perfect but help tremendously.
    • nonameiguess 6y ago
      This can't be emphasized enough. I'm presently pushing mypy and type hints hard in a Python project (passing static analysis and type checks is required to get through the CI pipeline) and it's just a compromise. Honestly, I don't want a dynamic duck-typed language at all, but it is what it is. Right now, the single most important factor determining whether we have any future at all is speed to market, not the increased reliability you get from type safety. And Python just has libraries for basically everything. It's enabled us to spin up brand new services that do exactly what we need to do from scratch in a matter of hours sometimes. With competing priorities like that, this is the best we can do right now. The other factor is the rest of the team is already familiar with Python. They're mostly infrastructure developers. They're more likely to do everything in Bash if given the choice. I'm not going to teach them Scala or Haskell or Rust in the next three weeks before we need to deliver something. But I can at least teach them Python's optional type hints and mypy.
• Animats 6y ago
  I could never get why Python added unchecked static typing. That seems so useless. If the compiler checks it, it can use it to optimize. Huge win for built-in types, like integer and float. CPython has to box everything. PyPy has to try to guess and use just-in-time recompilation when it guesses wrong.
  • neolog 6y ago
    For now, it improves documentation and static typechecking. But Mypyc is an annotated-python-to-C compiler that's in the works.
  • mfgs 6y ago
    Type hints are super handy when using smart IDEs like PyCharm, which will list or highlight (most of) your type errors. This alone can greatly improve confidence in your program, especially for more complex cases. Getting that cheaply while still having types unchecked is a nice compromise between improved functionality and not changing the language majorly.
  • kdmytro 6y ago
    Unchecked types by themselves are useful as they help my text editor to provide more accurate autocomplete suggestions.
  • davesque 6y ago
    Python didn't add static typing. It added arbitrary annotations of arguments and variables (admittedly, with the use case of type checking in mind). Mypy uses that for type annotations and checking. Here are the two main reasons I think it's useful: * To the extent that it enables some static analysis on python code, it has often caught a number of errors I tend to make while writing code * It serves as a light form of documentation that tends to stay more fresh because there's an automated way to check its consistency (run the type checker) I don't expect that it will do as perfect a job as Rust or C++'s type system or even that it would be there for the same reasons. On the other hand, it's nice to just have more info about what the code is expected to do in the form of type annotations.
• Xophmeister 6y ago
  Where would you use `typing.Protocol` where you wouldn't use an abstract base class (or a bunch of them as mixins)? My understanding is that the latter is more "formal", whereas `Protocol` is better suited for gradual typing (i.e., moving a more "free-form" codebase into something MyPy won't complain about). The only time I've ever used `Protocol` is to define a type that makes it explicit that I need an object to have a `__str__` implementation: @runtime_checkable class Stringable(Protocol): """ Protocol type that supports the __str__ method """ @abstractmethod def __str__(self) -> str: """ Return the string representation of an object """ I've since learned that this is redundant, because all Python objects have an implicit `__str__` if one isn't specified (IIRC). When I didn't know this (when I implemented the above), I didn't use an ABC because obviously I can't guarantee all objects (e.g., those outside of my control) are subclasses of said ABC. The cases where this is true are vanishingly small, especially when you go big on dependency inversion.
  • Conlectus 6y ago
    This distinction is the same as the distinction between structural and nominal typing, described at the end of the article. Abstract base classes require everything to extend from a base-level object, and also inherit it's default implementations. This is a nominal (name-based) mechanism. Protocol (structural-based) subtyping instead only declares what methods something has to have, without requiring it tie itself to either to a concrete parent class and its behaviours. Edit: since you partially address this, you ask why you'd use structural typing rather than an abstract base class. I'll turn it around and ask why you would use an abstract base class, which requires modifying the children accordingly, when you could instead use a structural type. The usual preference comes down to whether you want to be explicit about implementing it or not, and whether you want to pull in default behaviour.
    • Xophmeister 6y ago
      > The usual preference comes down to whether you want to be explicit about implementing it or not, and whether you want to pull in default behaviour. Thanks :) I definitely prefer things to be explicit; regardless of it being a "Zen of Python" mantra. That said, I do see the value of `Protocol` for non-annotated/legacy code, in that it gives you a convenient mechanism for this. I would be worried about it becoming a crutch, if overused, but definitely better than no annotations at all! (I don't understand your point regarding an ABC's default implementation. Isn't the point of ABCs that they're not base classes, but interfaces that define the methods required without any implementation?)
• edoceo 6y ago
  Haha, I thought somehow Weird Al had made it to top. Here's the song for the curious https://m.youtube.com/watch?v=eOL2q8leiLw https://m.youtube.com/watch?v=eOL2q8leiLw From 1985.
  • mixmastamyk 6y ago
    A take off on the classic Huey Lewis and the News song. Yes I had the Sports album, and saw Al live ~two years ago. :-D
    • geocrasher 6y ago
      Ah, the Ill Advised Self Indulgent Vanity Tour- I went and saw him in Eugene, OR. It was a fantastic show. I'll never forget it. 10 year old me was freaking out that he was playing the oddballs like "Those were the days" and "Stuck in a closet with Vanna White". What a show!
      • mixmastamyk 6y ago
        Yes, he came up near our seat at one point. Was a great show also, but I missed the MJ-related numbers. Only later I figured out why.
        • geocrasher 6y ago
          Why was that? I have not heard. Edit: Answered my own question via Google: https://consequenceofsound.net/2019/06/weird-al-yankovic-michael-jackson-parodies-leaving-neverland https://consequenceofsound.net/2019/06/weird-al-yankovic-mic...
  • iainctduncan 6y ago
    In 1985 I pretty much though Weird Al was the king. Crazy that he's still kicking it!
  • dehrmann 6y ago
    And the followup: https://www.youtube.com/watch?v=Fk15H6PjBis https://www.youtube.com/watch?v=Fk15H6PjBis
• legerdemain 6y ago
  Is this not literally the distinction between an interface and an implementation? Like, we all know about the expression problem and how interfaces are hard to modify after the fact... But are test mocks and stubs not literally the prime use case for interfaces at the syntax level?
  • morelisp 6y ago
    > Is this not literally the distinction between an interface and an implementation? No. e.g. Java interfaces are just as nominal as classes. > But are test mocks and stubs not literally the prime use case for interfaces at the syntax level? Also no. If your language has structural interfaces it is often preferred to inheritance as a tool for polymorphism. Even in C++ where templates are not the easiest tool to wield it's often still preferable to inheritance for many reasons.
    • legerdemain 6y ago
      What? No. The author has a Duck, which has a quack() method. The quack() method is, presumably, identified by its name, argument types, and return type. Even in a language with purely nominal typing, you would make a Quacker interface, which is the set of things that quack() -> None, and then have both Duck and all of its test stubs be Quackers. This is very literally how people who write in languages without structural types do test stubs (and often dependency injection, and other mechanisms that need to select from among several implementations of something).
      • morelisp 6y ago
        Alright, I guess I don't know what you mean by "prime use case" then. Yes, you can use interfaces to make test doubles. No, that's not the historically motivating factor behind their inclusion in languages.
• war1025 6y ago
  My biggest pain as far as Python3 typing / mypy goes has been interfacing with major libraries and trying to add types in a way that isn't just me tossing `Any` everywhere. If memory serves, Twisted hasn't been too bad. SqlAlchemy is pretty well a nightmare though. I think I saw that the latest release has type stubs. That will be nice. PyQt is also less than ideal...
• davepeck 6y ago
  (For those living in the python + typing world, I'd definitely recommend trying Microsoft's pyright as an alternative to mypy. In my experience, it is substantially faster and produces higher quality error messages. It does not support mypy plugins, so -- for instance -- Django ORM dynamism isn't well captured. But I've found that the Django plugin for mypy is still pretty young; when used on a big codebase, it both slows down type checking considerably and regularly produces spurious errors. Random: as someone who makes heavy use of both Python and Typescript, it pleases me to no end to see a great parser and ast for python implemented in typescript.)
  • jdfellow 6y ago
    I've been using pyright for a couple of weeks because other LSP backends for Emacs weren't working well and only pyright seems to have any future. I am quite happy with its fast and effective type checking and it's not very hard to make it happy unlike mypy.
  • girvo 6y ago
    TypeScript will eat the world one day, and honestly I don’t mind at all. Such a surprisingly powerful system, while being simple enough to throw juniors in the deep end with. My personal website is being rebuilt on Deno at the moment! Writing type level operators is a lot of fun haha
• l0b0 6y ago
  The biggest problem with Python typing + mypy is that it's not complete: not everything can be type annotated strictly (as in, avoiding `Any`) and correctly (as in, matching actual types passing through the code and not just whatever shuts up mypy). One example is JSON-like `dict`s, since mypy doesn't support recursive types yet[1]. [1] https://github.com/python/mypy/issues/731 https://github.com/python/mypy/issues/731
• iainctduncan 6y ago
  This title made me laugh because as a kid I happily sang along with Huey in the 80's about wanting a new... truck. Didn't even stop to wonder how a truck would make one have all those feelings....
• fiedzia 6y ago
  We are slowly getting some bits of static analysis. Perhaps another 20 years, and Python users will invent compilers.
  • dragonwriter 6y ago
    > Perhaps another 20 years, and Python users will invent compilers. Python’s main implementation has a bytecode compiler (that’s where pyc/pyo files come from.) Mypyc compiles type-annotated python code to native-code python extensions. Cython is a native-code compiler for a superset of python. Both Mypyc and Cython are used for widely-used code.