6 ms·

Interfaces and Protocols in Python

134 points by BerislavLopac 6y ago

carlosf 6y ago
I've recently found about Protocol / structural subtyping in Python and found this sort of pattern really useful as duck typing is so common in py code. Highly recommend reading: https://www.python.org/dev/peps/pep-0544/ https://www.python.org/dev/peps/pep-0544/ https://docs.python.org/3/library/typing.html#nominal-vs-structural-subtyping https://docs.python.org/3/library/typing.html#nominal-vs-str... https://docs.python.org/3/library/typing.html#typing.Protocol https://docs.python.org/3/library/typing.html#typing.Protoco... Cool example: https://github.com/jordaneremieff/mangum/blob/main/mangum/types.py https://github.com/jordaneremieff/mangum/blob/main/mangum/ty...
decafninja 6y ago
Have some bad memories with zope.interface. Worked on a team composed mainly of ex enterprise .NET developers forced to work with Python (myself included). Some embraced the Python paradigm, others stubbornly clinged to old enterprise .NET habits. zope.interface was heavily used by the latter to essentially write something as close as possible to an enterprise .NET application, only in Python.
- macintux 6y ago
  I tried to create a web service using Zope in 2002 or so. I struggled for weeks trying to figure out how to get anything done, never did produce anything useful. Not sure that says more about Zope or my bootstrapping skills, which are admittedly pretty poor.
  - dagw 6y ago
    Yea, Zope was weird. I learnt python in around 2000 and immediately fell in love with the language. Then I needed to to do some web stuff and wanted to move beyond simple cgi scripts and for some reason decided to pick up Zope. Almost stopped using Python all together then and there.
  - smoe 6y ago
    I never worked with pure Zope, my first job in the mid 2000 was using Plone, a CMS build on top of it. Certainly overkill for simple websites, but I found it much more enjoyable than other big CMS of the time. It had some cool things, that I started missing once I moved on. How well the permission system, workflow engine and hierarchical content structure worked together, undo functionality, accessing things via urls like “/path/to/object/method” instead uf unreadable params, component based UI, having a single configuration file (zc.buildout) to have a reproducible environment for development and production, etc. In many regards they were ahead of the time I think, at least in the space of things between simple script driven websites and very enterprisey solutions.
- padobson 6y ago
  This is what I was thinking about reading this post. Why add things to Python to make it strongly typed? Why not just use Java or C#?
  - Gibbon1 6y ago
    Because Java and especially C# aren't 'cool'
  - ralmidani 6y ago
    Not an exhaustive list of possible reasons: - (number_of_people_who_enjoy_coding_with_python_more_than_coding_with_c_sharp_or_java > 0) == True - Some people have large existing codebases, and would rather add types gradually than have to port all their logic to another language. - In terms of being free and open source, Python > C# > Java (meta: can you even do a comparison like that in C# and/or Java?)
    - nerdponx 6y ago
      I thought Java being not-open-source in 2021 was a myth, no? What about OpenJDK (GPL licensed)?
      - ralmidani 6y ago
        GPL2 has no explicit patent grant, and Oracle's behavior does not signal good intentions.
  - nerdponx 6y ago
    1. Because people like Python. 2. The type system is both gradual and not enforced by default. This means you benefit from the rapid prototyping and general freedom of dynamically-typed code, but you have the option to layer on the additional safety of static type checking as the project grows. There aren't many other gradually-typed languages out there anyway; Julia, Typed Racket, and Common Lisp aren't exactly in widespread industry use for general-purpose server dev.
    - 6y ago
harpiaharpyja 6y ago
Or you can use an ABC with no implementation logic. I don't agree with the author quickly dismissing them at the beginning. They seem like a relevant solution to when you want nominal typing instead of structural. Yes, they can contain implementation logic. They don't have to though.
- O5vYtytb 6y ago
  For me, not having the metaclass overhead of abc is a benefit of protocols. Unfortunately you can't mix metaclass types.
neolog 6y ago
> Unlike Interfaces, Protocols can describe the types of things that already exist. Can't interfaces do that too? You can just say builtins.set implements the receiver interface.
- joshuamorton 6y ago
  You have to say that builtins.set implements the interface. If there's a receiver protocol, it'll just accept a set of that matches the protocol, no registration required.
  - deleted 6y ago
    [deleted]
- zeugmasyllepsis 6y ago
  Not without modifying the existing type. Since Interfaces rely on nominal typing, each type implementing an interface must explicitly declare that they do so. On the other hand, since Protocols are checked structurally you can declare a protocol after several types already implement it, and type checking will pass without having to modify the implementing types.
  - neolog 6y ago
    Explicitly declaring which types provide which interfaces is the whole point of explicit interfaces. But I don't think the "you can declare a protocol after several types already implement it" is any different under explicit vs implicit conformance checking. I can just say implementer(IReceiver)(builtins.set) at any time. There's no temporal restriction on when I can do that.
    - zeugmasyllepsis 6y ago
      Right - so if you're introducing type annotations into an existing project and have a method expecting some interface, used by many call-sites spread across various parts of the project, then you can go around wrapping the original type at each call-site as you've indicated. Or add a single type annotations to the method declaring that the parameter satisfies a protocol.

134 points by BerislavLopac 6y ago

I've recently found about Protocol / structural subtyping in Python and found this sort of pattern really useful as duck typing is so common in py code. Highly recommend reading: https://www.python.org/dev/peps/pep-0544/ https://www.python.org/dev/peps/pep-0544/ https://docs.python.org/3/library/typing.html#nominal-vs-structural-subtyping https://docs.python.org/3/library/typing.html#nominal-vs-str... https://docs.python.org/3/library/typing.html#typing.Protocol https://docs.python.org/3/library/typing.html#typing.Protoco... Cool example: https://github.com/jordaneremieff/mangum/blob/main/mangum/types.py https://github.com/jordaneremieff/mangum/blob/main/mangum/ty...

decafninja 6y ago

Have some bad memories with zope.interface. Worked on a team composed mainly of ex enterprise .NET developers forced to work with Python (myself included). Some embraced the Python paradigm, others stubbornly clinged to old enterprise .NET habits. zope.interface was heavily used by the latter to essentially write something as close as possible to an enterprise .NET application, only in Python.

macintux 6y ago
I tried to create a web service using Zope in 2002 or so. I struggled for weeks trying to figure out how to get anything done, never did produce anything useful. Not sure that says more about Zope or my bootstrapping skills, which are admittedly pretty poor.
- dagw 6y ago
  Yea, Zope was weird. I learnt python in around 2000 and immediately fell in love with the language. Then I needed to to do some web stuff and wanted to move beyond simple cgi scripts and for some reason decided to pick up Zope. Almost stopped using Python all together then and there.
- smoe 6y ago
  I never worked with pure Zope, my first job in the mid 2000 was using Plone, a CMS build on top of it. Certainly overkill for simple websites, but I found it much more enjoyable than other big CMS of the time. It had some cool things, that I started missing once I moved on. How well the permission system, workflow engine and hierarchical content structure worked together, undo functionality, accessing things via urls like “/path/to/object/method” instead uf unreadable params, component based UI, having a single configuration file (zc.buildout) to have a reproducible environment for development and production, etc. In many regards they were ahead of the time I think, at least in the space of things between simple script driven websites and very enterprisey solutions.
padobson 6y ago
This is what I was thinking about reading this post. Why add things to Python to make it strongly typed? Why not just use Java or C#?
- Gibbon1 6y ago
  Because Java and especially C# aren't 'cool'
- ralmidani 6y ago
  Not an exhaustive list of possible reasons: - (number_of_people_who_enjoy_coding_with_python_more_than_coding_with_c_sharp_or_java > 0) == True - Some people have large existing codebases, and would rather add types gradually than have to port all their logic to another language. - In terms of being free and open source, Python > C# > Java (meta: can you even do a comparison like that in C# and/or Java?)
  - nerdponx 6y ago
    I thought Java being not-open-source in 2021 was a myth, no? What about OpenJDK (GPL licensed)?
    - ralmidani 6y ago
      GPL2 has no explicit patent grant, and Oracle's behavior does not signal good intentions.
- nerdponx 6y ago
  1. Because people like Python. 2. The type system is both gradual and not enforced by default. This means you benefit from the rapid prototyping and general freedom of dynamically-typed code, but you have the option to layer on the additional safety of static type checking as the project grows. There aren't many other gradually-typed languages out there anyway; Julia, Typed Racket, and Common Lisp aren't exactly in widespread industry use for general-purpose server dev.
  - 6y ago

harpiaharpyja 6y ago

Or you can use an ABC with no implementation logic. I don't agree with the author quickly dismissing them at the beginning. They seem like a relevant solution to when you want nominal typing instead of structural. Yes, they can contain implementation logic. They don't have to though.

O5vYtytb 6y ago
For me, not having the metaclass overhead of abc is a benefit of protocols. Unfortunately you can't mix metaclass types.

neolog 6y ago

> Unlike Interfaces, Protocols can describe the types of things that already exist. Can't interfaces do that too? You can just say builtins.set implements the receiver interface.

joshuamorton 6y ago
You have to say that builtins.set implements the interface. If there's a receiver protocol, it'll just accept a set of that matches the protocol, no registration required.
- deleted 6y ago
  [deleted]
zeugmasyllepsis 6y ago
Not without modifying the existing type. Since Interfaces rely on nominal typing, each type implementing an interface must explicitly declare that they do so. On the other hand, since Protocols are checked structurally you can declare a protocol after several types already implement it, and type checking will pass without having to modify the implementing types.
- neolog 6y ago
  Explicitly declaring which types provide which interfaces is the whole point of explicit interfaces. But I don't think the "you can declare a protocol after several types already implement it" is any different under explicit vs implicit conformance checking. I can just say implementer(IReceiver)(builtins.set) at any time. There's no temporal restriction on when I can do that.
  - zeugmasyllepsis 6y ago
    Right - so if you're introducing type annotations into an existing project and have a method expecting some interface, used by many call-sites spread across various parts of the project, then you can go around wrapping the original type at each call-site as you've indicated. Or add a single type annotations to the method declaring that the parameter satisfies a protocol.