8 ms·

Core dev Larry Hastings [0] puts it well. The cost-benefit case for this complicated language feature is limited. "I dislike the syntax and semantics expressed

by bede 6y ago

Core dev Larry Hastings [0] puts it well. The cost-benefit case for this complicated language feature is limited. "I dislike the syntax and semantics expressed in PEP 634. I see the match statement as a DSL contrived to look like Python, and to be used inside of Python, but with very different semantics. When you enter a PEP 634 match statement, the rules of the language change completely, and code that looks like existing Python code does something surprisingly very different. It also adds unprecedented new rules to Python, e.g. you can replace one expression with another in the exact same spot in your code, and if one has dots and the other doesn’t, the semantics of what the statement expresses changes completely. And it changes to yet a third set of semantics if you replace the expression with a single _. I think the bar for adding new syntax to Python at this point in its life should be set very high. The language is already conceptually pretty large, and every new feature means new concepts one must learn if one is to read an arbitrary blob of someone else’s Python code. The bigger the new syntax, the higher the bar should become, and so the bigger payoff the new syntax has to provide. To me, pattern matching doesn’t seem like it’s anywhere near big enough a win to be worth its enormous new conceptual load." This while we have elephants in the room such as packaging. Researching best practices to move away from setup.py right now takes you down a rabbit hole of (excellent) blog posts, and yet you still need a setup.py shim to use editable installs, because the new model simply doesn't yet support this fundamental feature. I can't afford to spend days immersing myself in packaging to the point of writing a PEP, but would help pay someone to do it well. I can see no way to fund packaging efforts directly on the PSF donations page (edit: see comment). It's great to see Pip improving but there is still not even a coherent guide that I can find for packaging using up-to-date best practices. This appears to be because the best practices are currently slightly broken. [0] https://discuss.python.org/t/gauging-sentiment-on-pattern-matching/5770/21 https://discuss.python.org/t/gauging-sentiment-on-pattern-ma...

1337shadow 6y ago
Did you try the "Sponsor" link in pypi.org ? It will take you there: https://pypi.org/sponsor/ https://pypi.org/sponsor/ > The Python Software Foundation is receiving $407,000 USD to support work on pip in 2020. Thank you to Mozilla (through its Mozilla Open Source Support Awards) and to the Chan Zuckerberg Initiative for this funding!
- cmlp 6y ago
  They should give that to the meson author instead. He knows what he's doing and has a clue about C/C++. Really, the whole distutils is a fancy way to produce a simple zip archive and put it in a well know location in site-packages. meson has the build figured out, it's just a way of installing that archive with a bit of metadata.
- bede 6y ago
  Thanks, donated.
mumblemumble 6y ago
> This while we have elephants in the room such as packaging. There is a part of me that wonders, at this point, if basically every new addition to the language itself is secretly just a medium for procrastinating on figuring out what to do about setup.py. I'm not as down on this PEP as some other seem to be, mind. It's just that, when I think about my actual pain points with python, this language addition starts to look like a very, very fancy bike shed.
- properdine 6y ago
  Have you taken a look at [PEP 517](https://www.python.org/dev/peps/pep-0517/ https://www.python.org/dev/peps/pep-0517/) ? It enables other tools to replace setup.py (e.g., poetry is pretty nice for making easy-to-package-and-publish pure python libraries).
- mixmastamyk 6y ago
  The people that work on packaging don't overlap much with those that work on the core language. The latter don't seem to care about it much, as far as I can tell.
- 1337shadow 6y ago
  I sort of wonder why I see so much users complaining about package management in Python, meanwhile I'm having a fantastic journey since 2008 with it, with over 50 published open source packages. "Sort of" because I'm suspecting that the users in question just do not want to integrate upstream changes continuously, so, they expect the package manager to help them procrastinating on dependency updates, which has proven to lead to disasters such as npm install, and I'm kind of worried that this is the direction they have been taking. But I admit I use setupmeta for auto definition of the version, and it just makes setup.py even better, but that's basically the only thing I like to add to setup.py, because it simplifies package publishing scripts. I haven't found any feature in pip to verify the gpg signatures that it allows us to upload packages with (python setup.py sdist upload --sign). As for pattern matching is not specific to Python, it's available in many other languages and is a joy to use in OCaml, I see no reason why Python would not have pattern matching.
rrdharan 6y ago
I would not call this “improving”: https://twitter.com/mildmojo/status/1354514025414066178?s=20 https://twitter.com/mildmojo/status/1354514025414066178?s=20 https://twitter.com/szferi/status/1352208874657472512?s=20 https://twitter.com/szferi/status/1352208874657472512?s=20 https://www.reddit.com/r/Python/comments/kfxibk/pypi_xmlrpc_search_api_has_been_disabled_due_to/ https://www.reddit.com/r/Python/comments/kfxibk/pypi_xmlrpc_... And relatedly they have a massive spam package problem: https://www.zdnet.com/article/pypi-gitlab-dealing-with-spam-attacks/ https://www.zdnet.com/article/pypi-gitlab-dealing-with-spam-...
- yunohn 6y ago
  Are you on the wrong article discussion? Or is this just a personal axe to grind?
rtpg 6y ago
Struggling a bit with Python and JS-related packaging stuff recently, I'm also wondering: in a clean-room environment, what does good Python packaging look like? Is it "get a single binary" stuff? PyOxidizer feels like the big win, but a part of me wonders if the original sin of allowing aribtrary code execution during installs really stops a "wrapper" tool from getting this right. https://pyoxidizer.readthedocs.io/en/v0.9.0/index.html https://pyoxidizer.readthedocs.io/en/v0.9.0/index.html
- ampdepolymerase 6y ago
  The gold standard for dynamic interpreted language package management is Yarn, and to a lesser extent npm. They are both cross platform, allows for easy publishing and building of native code dependencies, and support each package having its own conflicting dependencies, which means you don't have to do SAT solving. Furthermore, the metadata is held outside of the package, so package selection is much quicker than Python which requires downloading the entire library first and parsing the requirements.
  - kaba0 6y ago
    Is there a difference between package management for dynamic/static languages? Because without this arbitrary distinction npm is pretty much around last place on a ranking between package managers.
  - rurban 6y ago
    Nope, cpan still is the gold standard. Packages are tested before installation, and there is no need for hacks like pyenv or broken ruby envs. It also plays nicely with vendor packaging. No sat solving needed, deps are resolved dynamically, powered by simple Makefiles, not special baroque and limited declarations syntax.
  - 1337shadow 6y ago
    > support each package having its own conflicting dependencies So that you don't have to integrate upstream changes continuously... Not a way to build a sane ecosystem if you ask me. > package selection is much quicker than Python So is it because it downloads multiple versions of the same dependencies that it's so much slower than pip? Or is there more?
jennyyang 6y ago
Anything that breaks and changes semantics should not be allowed into the language. Let Python be Python, not a Frankenstein's monster of ideas like C++. If it were an idea that were Pythonic, you would not see the confusing examples I've seen in the comments. C++ is the poster child of trying to do too much with the language and it losing itself due to death-by-committee. It's very sad that Python has started down this road.
- pfalcon 6y ago
  Pattern matching is the brainchild of ML. Python, being a multi-paradigm language with the strong functional side, missed this simple in concept and powerful in practice language concept.
  - jnxx 6y ago
    > Python, being a multi-paradigm language with the strong functional side I would doubt that. Surely, things like Numpy are written in a functional fashion, but Python relies very much on statements, iteration, things not being an expression, almost every named symbol except string and number literals being mutable, and there are no block-scoped name bindings which are essential to functional languages. And the attempt to add the latter to Python might end in a far bigger train wreck than C++ is already. Mixing OOP and functional style works, more or less, for Scala, but everyone agrees that Scala is a hugely complex language. And in difference to Python, it has immutable values. What could turn out better would be to create a new language which runs interoperable in the same VM (much like Clojure runs alongside Java and can call into it). And that new language, perhaps with the file extension ".pfpy", would be almost purely functional, perhaps like a Scheme or Hy, without these dreaded parentheses. That would at least leverage Python's existing libraries.
  - jnxx 6y ago
    > Python, being a multi-paradigm language with the strong functional side Coming back to that, just a reminder that lambdas in Python are still gimped, closures do not work as expected because of -- scoping, and core developers in Python 3 tried to remove with "map" and "filter" tools that are considered quite essential for functional programming.
abraxaz 6y ago
> This while we have elephants in the room such as packaging. Researching best practices to move away from setup.py right now takes you down a rabbit hole of (excellent) blog posts, and yet you still need a setup.py shim to use editable installs, because the new model simply doesn't yet support this fundamental feature. You can do editable installs with poetry, I do it every day. Just run this: \rm -rv dist/; poetry build --format sdist && tar --wildcards -xvf dist/.tar.gz -O '/setup.py' > setup.py && pip3 install --prefix="${HOME}/.local/" --editable . More details here: https://github.com/python-poetry/poetry/issues/34#issuecomment-731054280 https://github.com/python-poetry/poetry/issues/34#issuecomme...
- bede 6y ago
  I do editable installs every day with setup.py – what is your point? Poetry is a very interesting third party solution to these issues, but not a best practice. Best practices matter in packaging.
jnxx 6y ago
This gives the impression that Python, similar to C++. might have entered a competition between popular languages which one can accumulate the most popular features. Obviously, pattern matching comes from the ML family and functional Lisps like Clojure. What makes it a difficult integration into Python is that in languages such as Rust, OCaml, Haskell, but also Racket and Clojure, almost everything is an expression, and name bindings are, apart from a few exceptions, always scoped. Consequently, pattern matching is an expression, not a statement. A similar issue is that Python 3 tried to become more "lazy", similar to how Haskell and Clojure are - this is the reason for replacing some list results with generators, which is a subtle breaking change. Lazy evaluation of sequences is nice-to-have on servers but its importance in Haskell and Clojure comes from them being functional languages which are geared towards a "pure" (side-effect-free) style, in which they differ a lot from Python. My impression is also that over time, Python has really absorbed a huge number of features from functional Lisp dialects. This might seem surprising. Well, here are some things that modern Lisps like SBCL, Schemes and functional languages on the one hand side, and Python 3 do have in common: * a Read-Eval-Print Loop (REPL) * strong dynamic typing * automatic memory management * memory safety * exceptions * a wide choice of built-in data types: lists, strings, vectors, arrays, dictionaries / hash maps, tuples, sets * keyword arguments and optional arguments in functions * handling of names in scopes and names spaces * closures and lambda functions * list comprehensions * pattern matching (limited support for tuples and lists in Python) * Unicode strings * arbitrarily long integers * complex numbers * rational numbers * number type are part of a hierarchical type hierarchy (numeric tower) * empty sequences, containers and strings are logically false * support for threads (but no real parallelism in Python) * low-level bit operations (a bit limited in Python) * easy way to call into C code * type annotations * if ... else can be used as an expression * string formatting is a mini language * hexadecimal, octal and binary literals * standard functions for functional programming like map and filter * support for OOP (e.g. by the Common Lisp Object System) * support for asynchronous execution What is notably missing from this list is parallelism (as opposed to concurrency); Python simply does not support it in a practical way, while some functional languages do support it extremely well. The net result of this creeping featuritis appears to be that Python, which is still classified as "beginner friendly", is now actually significantly more complex than a language like Racket or Clojure, perhaps because these features are often not integrated that well. I even think that Rust, while clearly being targeted at a very different domain, is more streamlined and well-composed than Python. What is also worth mentioning is that these functional languages have seen steady improvements in compilers and performance of generated code, with the result that Rust code is now frequently at least as fast as C code, SBCL is within arms reach of C, and Clojure and Racket are about in the same league as Java - while Python code in comparison continues to be extremely slow: https://benchmarksgame-team.pages.debian.net/benchmarksgame/fastest/python3-java.html https://benchmarksgame-team.pages.debian.net/benchmarksgame/... https://benchmarksgame-team.pages.debian.net/benchmarksgame/fastest/lisp.html https://benchmarksgame-team.pages.debian.net/benchmarksgame/... https://benchmarksgame-team.pages.debian.net/benchmarksgame/fastest/racket-java.html https://benchmarksgame-team.pages.debian.net/benchmarksgame/... https://benchmarksgame-team.pages.debian.net/benchmarksgame/fastest/ocaml-java.html https://benchmarksgame-team.pages.debian.net/benchmarksgame/...

by bede 6y ago

1337shadow 6y ago

Did you try the "Sponsor" link in pypi.org ? It will take you there: https://pypi.org/sponsor/ https://pypi.org/sponsor/ > The Python Software Foundation is receiving $407,000 USD to support work on pip in 2020. Thank you to Mozilla (through its Mozilla Open Source Support Awards) and to the Chan Zuckerberg Initiative for this funding!

cmlp 6y ago
They should give that to the meson author instead. He knows what he's doing and has a clue about C/C++. Really, the whole distutils is a fancy way to produce a simple zip archive and put it in a well know location in site-packages. meson has the build figured out, it's just a way of installing that archive with a bit of metadata.
bede 6y ago
Thanks, donated.

mumblemumble 6y ago

> This while we have elephants in the room such as packaging. There is a part of me that wonders, at this point, if basically every new addition to the language itself is secretly just a medium for procrastinating on figuring out what to do about setup.py. I'm not as down on this PEP as some other seem to be, mind. It's just that, when I think about my actual pain points with python, this language addition starts to look like a very, very fancy bike shed.

properdine 6y ago
Have you taken a look at [PEP 517](https://www.python.org/dev/peps/pep-0517/ https://www.python.org/dev/peps/pep-0517/) ? It enables other tools to replace setup.py (e.g., poetry is pretty nice for making easy-to-package-and-publish pure python libraries).
mixmastamyk 6y ago
The people that work on packaging don't overlap much with those that work on the core language. The latter don't seem to care about it much, as far as I can tell.
1337shadow 6y ago
I sort of wonder why I see so much users complaining about package management in Python, meanwhile I'm having a fantastic journey since 2008 with it, with over 50 published open source packages. "Sort of" because I'm suspecting that the users in question just do not want to integrate upstream changes continuously, so, they expect the package manager to help them procrastinating on dependency updates, which has proven to lead to disasters such as npm install, and I'm kind of worried that this is the direction they have been taking. But I admit I use setupmeta for auto definition of the version, and it just makes setup.py even better, but that's basically the only thing I like to add to setup.py, because it simplifies package publishing scripts. I haven't found any feature in pip to verify the gpg signatures that it allows us to upload packages with (python setup.py sdist upload --sign). As for pattern matching is not specific to Python, it's available in many other languages and is a joy to use in OCaml, I see no reason why Python would not have pattern matching.

rrdharan 6y ago

I would not call this “improving”: https://twitter.com/mildmojo/status/1354514025414066178?s=20 https://twitter.com/mildmojo/status/1354514025414066178?s=20 https://twitter.com/szferi/status/1352208874657472512?s=20 https://twitter.com/szferi/status/1352208874657472512?s=20 https://www.reddit.com/r/Python/comments/kfxibk/pypi_xmlrpc_search_api_has_been_disabled_due_to/ https://www.reddit.com/r/Python/comments/kfxibk/pypi_xmlrpc_... And relatedly they have a massive spam package problem: https://www.zdnet.com/article/pypi-gitlab-dealing-with-spam-attacks/ https://www.zdnet.com/article/pypi-gitlab-dealing-with-spam-...

yunohn 6y ago
Are you on the wrong article discussion? Or is this just a personal axe to grind?

rtpg 6y ago

Struggling a bit with Python and JS-related packaging stuff recently, I'm also wondering: in a clean-room environment, what does good Python packaging look like? Is it "get a single binary" stuff? PyOxidizer feels like the big win, but a part of me wonders if the original sin of allowing aribtrary code execution during installs really stops a "wrapper" tool from getting this right. https://pyoxidizer.readthedocs.io/en/v0.9.0/index.html https://pyoxidizer.readthedocs.io/en/v0.9.0/index.html

ampdepolymerase 6y ago
The gold standard for dynamic interpreted language package management is Yarn, and to a lesser extent npm. They are both cross platform, allows for easy publishing and building of native code dependencies, and support each package having its own conflicting dependencies, which means you don't have to do SAT solving. Furthermore, the metadata is held outside of the package, so package selection is much quicker than Python which requires downloading the entire library first and parsing the requirements.
- kaba0 6y ago
  Is there a difference between package management for dynamic/static languages? Because without this arbitrary distinction npm is pretty much around last place on a ranking between package managers.
- rurban 6y ago
  Nope, cpan still is the gold standard. Packages are tested before installation, and there is no need for hacks like pyenv or broken ruby envs. It also plays nicely with vendor packaging. No sat solving needed, deps are resolved dynamically, powered by simple Makefiles, not special baroque and limited declarations syntax.
- 1337shadow 6y ago
  > support each package having its own conflicting dependencies So that you don't have to integrate upstream changes continuously... Not a way to build a sane ecosystem if you ask me. > package selection is much quicker than Python So is it because it downloads multiple versions of the same dependencies that it's so much slower than pip? Or is there more?

jennyyang 6y ago

Anything that breaks and changes semantics should not be allowed into the language. Let Python be Python, not a Frankenstein's monster of ideas like C++. If it were an idea that were Pythonic, you would not see the confusing examples I've seen in the comments. C++ is the poster child of trying to do too much with the language and it losing itself due to death-by-committee. It's very sad that Python has started down this road.

pfalcon 6y ago
Pattern matching is the brainchild of ML. Python, being a multi-paradigm language with the strong functional side, missed this simple in concept and powerful in practice language concept.
- jnxx 6y ago
  > Python, being a multi-paradigm language with the strong functional side I would doubt that. Surely, things like Numpy are written in a functional fashion, but Python relies very much on statements, iteration, things not being an expression, almost every named symbol except string and number literals being mutable, and there are no block-scoped name bindings which are essential to functional languages. And the attempt to add the latter to Python might end in a far bigger train wreck than C++ is already. Mixing OOP and functional style works, more or less, for Scala, but everyone agrees that Scala is a hugely complex language. And in difference to Python, it has immutable values. What could turn out better would be to create a new language which runs interoperable in the same VM (much like Clojure runs alongside Java and can call into it). And that new language, perhaps with the file extension ".pfpy", would be almost purely functional, perhaps like a Scheme or Hy, without these dreaded parentheses. That would at least leverage Python's existing libraries.
- jnxx 6y ago
  > Python, being a multi-paradigm language with the strong functional side Coming back to that, just a reminder that lambdas in Python are still gimped, closures do not work as expected because of -- scoping, and core developers in Python 3 tried to remove with "map" and "filter" tools that are considered quite essential for functional programming.

abraxaz 6y ago

> This while we have elephants in the room such as packaging. Researching best practices to move away from setup.py right now takes you down a rabbit hole of (excellent) blog posts, and yet you still need a setup.py shim to use editable installs, because the new model simply doesn't yet support this fundamental feature. You can do editable installs with poetry, I do it every day. Just run this: \rm -rv dist/; poetry build --format sdist && tar --wildcards -xvf dist/.tar.gz -O '/setup.py' > setup.py && pip3 install --prefix="${HOME}/.local/" --editable . More details here: https://github.com/python-poetry/poetry/issues/34#issuecomment-731054280 https://github.com/python-poetry/poetry/issues/34#issuecomme...

bede 6y ago
I do editable installs every day with setup.py – what is your point? Poetry is a very interesting third party solution to these issues, but not a best practice. Best practices matter in packaging.

jnxx 6y ago

This gives the impression that Python, similar to C++. might have entered a competition between popular languages which one can accumulate the most popular features. Obviously, pattern matching comes from the ML family and functional Lisps like Clojure. What makes it a difficult integration into Python is that in languages such as Rust, OCaml, Haskell, but also Racket and Clojure, almost everything is an expression, and name bindings are, apart from a few exceptions, always scoped. Consequently, pattern matching is an expression, not a statement. A similar issue is that Python 3 tried to become more "lazy", similar to how Haskell and Clojure are - this is the reason for replacing some list results with generators, which is a subtle breaking change. Lazy evaluation of sequences is nice-to-have on servers but its importance in Haskell and Clojure comes from them being functional languages which are geared towards a "pure" (side-effect-free) style, in which they differ a lot from Python. My impression is also that over time, Python has really absorbed a huge number of features from functional Lisp dialects. This might seem surprising. Well, here are some things that modern Lisps like SBCL, Schemes and functional languages on the one hand side, and Python 3 do have in common: * a Read-Eval-Print Loop (REPL) * strong dynamic typing * automatic memory management * memory safety * exceptions * a wide choice of built-in data types: lists, strings, vectors, arrays, dictionaries / hash maps, tuples, sets * keyword arguments and optional arguments in functions * handling of names in scopes and names spaces * closures and lambda functions * list comprehensions * pattern matching (limited support for tuples and lists in Python) * Unicode strings * arbitrarily long integers * complex numbers * rational numbers * number type are part of a hierarchical type hierarchy (numeric tower) * empty sequences, containers and strings are logically false * support for threads (but no real parallelism in Python) * low-level bit operations (a bit limited in Python) * easy way to call into C code * type annotations * if ... else can be used as an expression * string formatting is a mini language * hexadecimal, octal and binary literals * standard functions for functional programming like map and filter * support for OOP (e.g. by the Common Lisp Object System) * support for asynchronous execution What is notably missing from this list is parallelism (as opposed to concurrency); Python simply does not support it in a practical way, while some functional languages do support it extremely well. The net result of this creeping featuritis appears to be that Python, which is still classified as "beginner friendly", is now actually significantly more complex than a language like Racket or Clojure, perhaps because these features are often not integrated that well. I even think that Rust, while clearly being targeted at a very different domain, is more streamlined and well-composed than Python. What is also worth mentioning is that these functional languages have seen steady improvements in compilers and performance of generated code, with the result that Rust code is now frequently at least as fast as C code, SBCL is within arms reach of C, and Clojure and Racket are about in the same league as Java - while Python code in comparison continues to be extremely slow: https://benchmarksgame-team.pages.debian.net/benchmarksgame/fastest/python3-java.html https://benchmarksgame-team.pages.debian.net/benchmarksgame/... https://benchmarksgame-team.pages.debian.net/benchmarksgame/fastest/lisp.html https://benchmarksgame-team.pages.debian.net/benchmarksgame/... https://benchmarksgame-team.pages.debian.net/benchmarksgame/fastest/racket-java.html https://benchmarksgame-team.pages.debian.net/benchmarksgame/... https://benchmarksgame-team.pages.debian.net/benchmarksgame/fastest/ocaml-java.html https://benchmarksgame-team.pages.debian.net/benchmarksgame/...