3 ms·

Recently there's a strong tendency in the Python ecosystem to wrap perfectly fine idioms with even higher abstractions. Not sure if I like this, in the end it l

Recently there's a strong tendency in the Python ecosystem to wrap perfectly fine idioms with even higher abstractions. Not sure if I like this, in the end it leads to a lot of cargo culting and people writing code they don't fully understand because it contains a lot of metaclass and runtime magic. But I understand that a lot of developers find this attractive, and type annotations really provide a good channel for realizing such functionality, so maybe I'm just being grumpy here.

ospider 5y ago
For myself, better code completion is enough to lure me into the world of fully type-annotated python. I don't understand what you mean by saying "cargo cult", please enlighten me.
- jampekka 5y ago
  Cargo cult probably refers to doing something based on superficial assesment without understanding what is actually done. Originally refers to problems in science. https://calteches.library.caltech.edu/51/2/CargoCult.htm https://calteches.library.caltech.edu/51/2/CargoCult.htm
- nooorofe 5y ago
  > fully type-annotated python When I do that, I ask myself: "how is it different from writing Java code?". I don't have a clear answer. It becomes almost as verbose as Java. Java has better type validation (doesn't compile on mistake); it is faster (at least some payback for verbosity); real type system has additional advantages, for example using types as part of method signature.
deleted 5y ago
[deleted]
mister_mister 5y ago
I agree, Python is/was about simplicity. That is why for validation I use (Maat)[https://pypi.org/project/Maat/ https://pypi.org/project/Maat/] It's just a dictionary to describe the data, and it's easy to expand.
- techhazard 5y ago
  I also prefer my python to stay simple. Maat looks really interesting for this use-case.
jampekka 5y ago
One aspect may be that people just aren't that satisfied with some of the design decisions of Python. In this case the (perceived) issue is probably unnecessary verbosity in declaring classes, which I agree with. Due to the lack of macros changes are made through decorators and other metaprogramming magic, which brings about some collateral issues. I find it preferable that such changes are possible outside the central committee bureaucracy. You really don't want to end up like C++ w.r.t language evolution.
janto 5y ago
Yes, this higher abstractions tendency is clearly away from "Flat is better than nested". Creating good abstractions is so difficult (and important) that it should not be done lightly by everyone. The recent proliferation of type annotations is also trending towards the static typing mindset, which is also cargo culting on a massive scale.
- smohare 5y ago
  I think the comment about types takes a rather narrow view of Python use cases. Sure, if you’re some lone sysadmin using Python as an alternative to bash to implement some questionable ETL pipeline comprising a bunch of random scripts running off a single machine somewhere then I agree. You probably are not the target audience for type annotations. But even moderately complex libraries and applications are such a huge pain to develop, read, and maintain without some of the tooling that leverages type annotations. This is especially true in the ML / AI / Data Science world where a lot of the people implementing models have dubious coding practices. Pure dynamic typing paradigms simply have not delivered on their promises over the last 30 years. There are definitely some areas where they make sense, but I doubt we will see a massive readoption until our tooling becomes sufficiently intelligent. Imagine, for example, a probabilistic type inference based on both the structural aspects of the code and previous runs over actual data.
  - janto 5y ago
    People in the data science world with dubious coding practices should keep their data in commonly used types such as Pandas tables. The last thing we should want from them is encoding their data with ill-conceived class/type systems. Not sure if that is what you mean. I'd say Python's large scale adoption is exactly a good example of dynamic typing delivering on its promises.
- kortex 5y ago
  Static typing in python isn't "cargo culting." In fact I'd go even farther in the opposite direction, and say that if you are using python for production software engineering, and aren't typing most (>50%) of your function signatures, you are wasting everybody's time, your teams', code reviewers', and most of all your own time. Opinion, but empirical experience bears this out for me. I'm not talking mypy --strict here, heck I've yet to achieve that myself. But if your are writing functions with obvious realizations of interfaces and not typing them, all you are doing is creating more mental strain for the next consumer (often future you) of that function down the road. At $LASTCO I inherited some jupyter-notebook-copy-pasted datascience ML abomination of a pipeline. Dicts everywhere, mutation everywhere, zero docstrings, nary a type hint in sight. Took two exceptionally stressful months, with constant back and forth with the authors, to get it working in prod ("it works on my machine, I don't understand the problem"). $THISCO embraces type hints, functional style, etc. My stress level has actually normalized. Typing `foo(bar: Optional[float])` takes what, 2s, more than `foo(bar)`? Asking "hey Steve, what does "bar" take in function "foo"? on slack is already more time and characters than just annotating it, times every dev that doesn't know and has to ask.

ospider 5y ago

For myself, better code completion is enough to lure me into the world of fully type-annotated python. I don't understand what you mean by saying "cargo cult", please enlighten me.

jampekka 5y ago
Cargo cult probably refers to doing something based on superficial assesment without understanding what is actually done. Originally refers to problems in science. https://calteches.library.caltech.edu/51/2/CargoCult.htm https://calteches.library.caltech.edu/51/2/CargoCult.htm
nooorofe 5y ago
> fully type-annotated python When I do that, I ask myself: "how is it different from writing Java code?". I don't have a clear answer. It becomes almost as verbose as Java. Java has better type validation (doesn't compile on mistake); it is faster (at least some payback for verbosity); real type system has additional advantages, for example using types as part of method signature.

deleted 5y ago

[deleted]

mister_mister 5y ago

I agree, Python is/was about simplicity. That is why for validation I use (Maat)[https://pypi.org/project/Maat/ https://pypi.org/project/Maat/] It's just a dictionary to describe the data, and it's easy to expand.

techhazard 5y ago
I also prefer my python to stay simple. Maat looks really interesting for this use-case.

jampekka 5y ago

One aspect may be that people just aren't that satisfied with some of the design decisions of Python. In this case the (perceived) issue is probably unnecessary verbosity in declaring classes, which I agree with. Due to the lack of macros changes are made through decorators and other metaprogramming magic, which brings about some collateral issues. I find it preferable that such changes are possible outside the central committee bureaucracy. You really don't want to end up like C++ w.r.t language evolution.

janto 5y ago

Yes, this higher abstractions tendency is clearly away from "Flat is better than nested". Creating good abstractions is so difficult (and important) that it should not be done lightly by everyone. The recent proliferation of type annotations is also trending towards the static typing mindset, which is also cargo culting on a massive scale.

smohare 5y ago
I think the comment about types takes a rather narrow view of Python use cases. Sure, if you’re some lone sysadmin using Python as an alternative to bash to implement some questionable ETL pipeline comprising a bunch of random scripts running off a single machine somewhere then I agree. You probably are not the target audience for type annotations. But even moderately complex libraries and applications are such a huge pain to develop, read, and maintain without some of the tooling that leverages type annotations. This is especially true in the ML / AI / Data Science world where a lot of the people implementing models have dubious coding practices. Pure dynamic typing paradigms simply have not delivered on their promises over the last 30 years. There are definitely some areas where they make sense, but I doubt we will see a massive readoption until our tooling becomes sufficiently intelligent. Imagine, for example, a probabilistic type inference based on both the structural aspects of the code and previous runs over actual data.
- janto 5y ago
  People in the data science world with dubious coding practices should keep their data in commonly used types such as Pandas tables. The last thing we should want from them is encoding their data with ill-conceived class/type systems. Not sure if that is what you mean. I'd say Python's large scale adoption is exactly a good example of dynamic typing delivering on its promises.
kortex 5y ago
Static typing in python isn't "cargo culting." In fact I'd go even farther in the opposite direction, and say that if you are using python for production software engineering, and aren't typing most (>50%) of your function signatures, you are wasting everybody's time, your teams', code reviewers', and most of all your own time. Opinion, but empirical experience bears this out for me. I'm not talking mypy --strict here, heck I've yet to achieve that myself. But if your are writing functions with obvious realizations of interfaces and not typing them, all you are doing is creating more mental strain for the next consumer (often future you) of that function down the road. At $LASTCO I inherited some jupyter-notebook-copy-pasted datascience ML abomination of a pipeline. Dicts everywhere, mutation everywhere, zero docstrings, nary a type hint in sight. Took two exceptionally stressful months, with constant back and forth with the authors, to get it working in prod ("it works on my machine, I don't understand the problem"). $THISCO embraces type hints, functional style, etc. My stress level has actually normalized. Typing `foo(bar: Optional[float])` takes what, 2s, more than `foo(bar)`? Asking "hey Steve, what does "bar" take in function "foo"? on slack is already more time and characters than just annotating it, times every dev that doesn't know and has to ask.