5 ms·

Yeah this is great, __init_subclass__ comes along to make dynamic injection of methods and attributes on class creation easier, just as the entire practice is f

by zzzeek 5y ago

Yeah this is great, __init_subclass__ comes along to make dynamic injection of methods and attributes on class creation easier, just as the entire practice is fast becoming fully obsolete because type checkers like pylance and mypy report these attributes and methods as errors.

sillysaurusx 5y ago
There are a few ways to deal with that. One is to declare the properties directly in the subclass: class Inject: def __init_subclass__(cls): cls.injected = 42 class Child(Inject): injected: int print(Child().injected) # 42 This technique isn't too useful in practice, though it has its place (e.g. when creating ctypes.Structures). But there's a clever way to simplify this. Remember that the subclass is a subclass -- it inherits from Inject: import typing as t class Inject: injected: t.ClassVar[int] def __init_subclass__(cls): cls.injected = 42 class Child(Inject): pass print(Child().injected) # 42 (ClassVar[int] ensures that type checkers know it's a variable attached to the class, not each instance.)
- zzzeek 5y ago
  yes you can do that, but if you want to do say, what dataclasses does, where it collects all the Field objects and creates a typed `__init__` method, typing can't do that without plugins. all the current typecheckers hardcode the specific behavior of "dataclasses" without there being any pep that allows other systems (like ORMs) to take advantage of the same thing without writing mypy plugins or simply not working for other type checkers that don't allow plugins.
  - sillysaurusx 5y ago
    Yeah, that’s true. Dataclass support is severely lacking. Is there a pep on the horizon for this?
    - zzzeek 5y ago
      im not really sure (highly doubtful, much simpler ideas like being able to make your own Tuple subclass are still not ready for primetime) but this would be a pretty tough pep to allow the general "dataclass" concept to be available to other libraries that want to use the same pattern, but not use dataclasses.
      - zzzeek 5y ago
        Update, see sibling reply. The pylance folks are killing it.
    - zem 5y ago
      yes, check out https://github.com/microsoft/pyright/blob/main/specs/dataclass_transforms.md https://github.com/microsoft/pyright/blob/main/specs/datacla...
      - zzzeek 5y ago
        WOW
ThePhysicist 5y ago
That's my observation as well. If you write typed Python code (or Typescript) you need to constrain yourself to a small subset of what the dynamic language can do. In some cases it's beneficial but often it can be quite stifling, so not sure I'm a big fan of gradual typing anymore.
shrimpx 5y ago
That's why I resist python types. A lot of the usefulness and interestingness of Python is in its dynamic features which aid in rapidly developing complex systems. Hard to imagine a strictly typed future for Python, when you then have little to trade off for its bad performance and lack of real concurrency. JavaScript has seen success as a typed language because it has being "the exclusive frontend language" going for it, and v8 is high performance. And the dynamic capabilities of JavaScript were never that interesting or powerful.
- ryanianian 5y ago
  > Hard to imagine a strictly typed future for Python On the contrary, the dynamic nature of python can make it seem like inaccessible black-magic where you don't even know where to look in the code to see what's even possible. Stronger types will empower users to understand their ecosystems. I've spent countless hours in rails/django land trying to figure out what's even possible with an object that the ORM or whatever gives me. The human-flavored docs kinda outline it, but if you're in IDE-land, context-switching to prose isn't productive. Types help humans develop and debug, but they make library authors jump through acrobatics to express what their magic does. This is a limitation of the type-checker implementation(s) not having a strong syntax or notion to capture the runtime-dynamic polymorphism, but it doesn't mean that the concept of types is a flawed idea or not worth using when appropriate. Don't throw the baby out with the bath-water.
  - shrimpx 5y ago
    I like types, but I have a hard time seeing them in Python. If I was forced to use only typed Python, I'd just use golang and get a ton of performance and concurrency that I could never get in typed Python. Good point about Django though. Django has so much historical stickiness that types probably look attractive for entrenched corporate projects that are hard to port to high performance natively typed languages.
dragonwriter 5y ago
> Yeah this is great, __init_subclass__ comes along to make dynamic injection of methods and attributes on class creation easier, just as the entire practice is fast becoming fully obsolete because type checkers like pylance and mypy report these attributes and methods as errors. The Python community and the set of people that treat mypy and pylance as dictators rather than tools to be used where appropriate and turned off where not are...not the same thing. (And the latter very much depends on tools built by the rest of the former that internally are wild and woolly, even if they present a nice cleanly typed interface to the consumer.)
- zzzeek 5y ago
  sure...im coming from the library producer perspective. it's not very easy for us to say, "just turn off the type checker when you use our library" :)
  - Too 5y ago
    Mypy is scoped to have different levels of checking per package. Normally it doesn’t scan third party libraries beyond the signature of public functions. See the follow_imports options.
  - dragonwriter 5y ago
    > it's not very easy for us to say, "just turn off the type checker when you use our library" :) Sure, and that is a problem with exposing certain kinds of dynamism when your customer base is the kind that is inflexible about typing. OTOH, a lot can still be done with dynamism on the inside and a typed public interface, and also there's a significant part of the community that accepts selective disabling of typechecking as an acceptable when dynamism provides adequate benefits.
j4mie 5y ago
Python is the most popular language on the planet because it’s a dynamic language, not in spite of it. If the type checker makes dynamic features difficult, ditch the type checker, not the dynamic features. I can’t wait for this pendulum to swing back the other way.
- Tanjreeve 5y ago
  Was there ever a point people were successfully building "backbone" systems in dynamically typed languages? I thought the pythons and Perl's of the world were always mainly doing scripts/interface applications.
  - stevesimmons 5y ago
    See [1] for a PyData London talk I did on "Python at Massive Scale", about JPMorgan's Athena trading and risk management system, used for FX, Commodities, Credit, Equities, etc trading. At that time, Athena had 4,500 Python developers doing 20,000 commits a week. Also see one recent HN discussion on "An Oral History of Bank Python" [2]. [1] https://www.youtube.com/watch?v=ZYD9yyMh9Hk https://www.youtube.com/watch?v=ZYD9yyMh9Hk [2] https://news.ycombinator.com/item?id=29104047 https://news.ycombinator.com/item?id=29104047
    - Tanjreeve 5y ago
      Thank you this is super interesting having only heard vague hand waving about bank python. I'm still a bit dubious on it if you don't have the manpower budget to keep this kind of custom setup running but it's really interesting to know how it's done by people who do.
  - zmmmmm 5y ago
    Dunno if you ever heard of this thing called Facebook ... but for a long time it was implemented almost entirely in PHP. The truth is, the argument that you cannot implement complex systems in dynamic languages has always been empirically fraught because there are so many counter examples. The problem is, even if static languages are "better" for such things, dynamic languages are certainly not bad enough at it to prevent them being successfully used for large scale systems.
robertlagrant 5y ago
Yes, this. This is where type checking becomes potentially harmful: you have to start creating loads of explicit subclasses instead of one dynamic class. And then you're writing slow Java.

by zzzeek 5y ago

sillysaurusx 5y ago

There are a few ways to deal with that. One is to declare the properties directly in the subclass: class Inject: def __init_subclass__(cls): cls.injected = 42 class Child(Inject): injected: int print(Child().injected) # 42 This technique isn't too useful in practice, though it has its place (e.g. when creating ctypes.Structures). But there's a clever way to simplify this. Remember that the subclass is a subclass -- it inherits from Inject: import typing as t class Inject: injected: t.ClassVar[int] def __init_subclass__(cls): cls.injected = 42 class Child(Inject): pass print(Child().injected) # 42 (ClassVar[int] ensures that type checkers know it's a variable attached to the class, not each instance.)

zzzeek 5y ago
yes you can do that, but if you want to do say, what dataclasses does, where it collects all the Field objects and creates a typed `__init__` method, typing can't do that without plugins. all the current typecheckers hardcode the specific behavior of "dataclasses" without there being any pep that allows other systems (like ORMs) to take advantage of the same thing without writing mypy plugins or simply not working for other type checkers that don't allow plugins.
- sillysaurusx 5y ago
  Yeah, that’s true. Dataclass support is severely lacking. Is there a pep on the horizon for this?
  - zzzeek 5y ago
    im not really sure (highly doubtful, much simpler ideas like being able to make your own Tuple subclass are still not ready for primetime) but this would be a pretty tough pep to allow the general "dataclass" concept to be available to other libraries that want to use the same pattern, but not use dataclasses.
    - zzzeek 5y ago
      Update, see sibling reply. The pylance folks are killing it.
  - zem 5y ago
    yes, check out https://github.com/microsoft/pyright/blob/main/specs/dataclass_transforms.md https://github.com/microsoft/pyright/blob/main/specs/datacla...
    - zzzeek 5y ago
      WOW

ThePhysicist 5y ago

That's my observation as well. If you write typed Python code (or Typescript) you need to constrain yourself to a small subset of what the dynamic language can do. In some cases it's beneficial but often it can be quite stifling, so not sure I'm a big fan of gradual typing anymore.

shrimpx 5y ago

That's why I resist python types. A lot of the usefulness and interestingness of Python is in its dynamic features which aid in rapidly developing complex systems. Hard to imagine a strictly typed future for Python, when you then have little to trade off for its bad performance and lack of real concurrency. JavaScript has seen success as a typed language because it has being "the exclusive frontend language" going for it, and v8 is high performance. And the dynamic capabilities of JavaScript were never that interesting or powerful.

ryanianian 5y ago
> Hard to imagine a strictly typed future for Python On the contrary, the dynamic nature of python can make it seem like inaccessible black-magic where you don't even know where to look in the code to see what's even possible. Stronger types will empower users to understand their ecosystems. I've spent countless hours in rails/django land trying to figure out what's even possible with an object that the ORM or whatever gives me. The human-flavored docs kinda outline it, but if you're in IDE-land, context-switching to prose isn't productive. Types help humans develop and debug, but they make library authors jump through acrobatics to express what their magic does. This is a limitation of the type-checker implementation(s) not having a strong syntax or notion to capture the runtime-dynamic polymorphism, but it doesn't mean that the concept of types is a flawed idea or not worth using when appropriate. Don't throw the baby out with the bath-water.
- shrimpx 5y ago
  I like types, but I have a hard time seeing them in Python. If I was forced to use only typed Python, I'd just use golang and get a ton of performance and concurrency that I could never get in typed Python. Good point about Django though. Django has so much historical stickiness that types probably look attractive for entrenched corporate projects that are hard to port to high performance natively typed languages.

dragonwriter 5y ago

> Yeah this is great, __init_subclass__ comes along to make dynamic injection of methods and attributes on class creation easier, just as the entire practice is fast becoming fully obsolete because type checkers like pylance and mypy report these attributes and methods as errors. The Python community and the set of people that treat mypy and pylance as dictators rather than tools to be used where appropriate and turned off where not are...not the same thing. (And the latter very much depends on tools built by the rest of the former that internally are wild and woolly, even if they present a nice cleanly typed interface to the consumer.)

zzzeek 5y ago
sure...im coming from the library producer perspective. it's not very easy for us to say, "just turn off the type checker when you use our library" :)
- Too 5y ago
  Mypy is scoped to have different levels of checking per package. Normally it doesn’t scan third party libraries beyond the signature of public functions. See the follow_imports options.
- dragonwriter 5y ago
  > it's not very easy for us to say, "just turn off the type checker when you use our library" :) Sure, and that is a problem with exposing certain kinds of dynamism when your customer base is the kind that is inflexible about typing. OTOH, a lot can still be done with dynamism on the inside and a typed public interface, and also there's a significant part of the community that accepts selective disabling of typechecking as an acceptable when dynamism provides adequate benefits.

j4mie 5y ago

Python is the most popular language on the planet because it’s a dynamic language, not in spite of it. If the type checker makes dynamic features difficult, ditch the type checker, not the dynamic features. I can’t wait for this pendulum to swing back the other way.

Tanjreeve 5y ago
Was there ever a point people were successfully building "backbone" systems in dynamically typed languages? I thought the pythons and Perl's of the world were always mainly doing scripts/interface applications.
- stevesimmons 5y ago
  See [1] for a PyData London talk I did on "Python at Massive Scale", about JPMorgan's Athena trading and risk management system, used for FX, Commodities, Credit, Equities, etc trading. At that time, Athena had 4,500 Python developers doing 20,000 commits a week. Also see one recent HN discussion on "An Oral History of Bank Python" [2]. [1] https://www.youtube.com/watch?v=ZYD9yyMh9Hk https://www.youtube.com/watch?v=ZYD9yyMh9Hk [2] https://news.ycombinator.com/item?id=29104047 https://news.ycombinator.com/item?id=29104047
  - Tanjreeve 5y ago
    Thank you this is super interesting having only heard vague hand waving about bank python. I'm still a bit dubious on it if you don't have the manpower budget to keep this kind of custom setup running but it's really interesting to know how it's done by people who do.
- zmmmmm 5y ago
  Dunno if you ever heard of this thing called Facebook ... but for a long time it was implemented almost entirely in PHP. The truth is, the argument that you cannot implement complex systems in dynamic languages has always been empirically fraught because there are so many counter examples. The problem is, even if static languages are "better" for such things, dynamic languages are certainly not bad enough at it to prevent them being successfully used for large scale systems.

robertlagrant 5y ago

Yes, this. This is where type checking becomes potentially harmful: you have to start creating loads of explicit subclasses instead of one dynamic class. And then you're writing slow Java.