5 ms·

I think python packaging has gotten LOTS better in the last few years. I find it quite pleasurable to use these days. From binary wheels (including on differen

by renesd 10y ago

I think python packaging has gotten LOTS better in the last few years. I find it quite pleasurable to use these days. From binary wheels (including on different linux architectures), to things like local caching of packages (taking LOTS of load off the main servers). To the organisation github of pypa [0], to `python -m venv` working. Also lots of work around standardising things in peps, and writing documentation for people. I would like to applaud all the hard work people have done over the years on python packaging. It really is quite nice these days, and I look forward to all the improvements coming up (like pipenv!). I'd suggest people checkout fades [1] (for running scripts and automatically downloading dependencies in a venv), as well as conda [2] the alternative package manager. [0] https://github.com/pypa/ https://github.com/pypa/ [1] https://fades.readthedocs.io/en/release-5/readme.html#what-does-it-do https://fades.readthedocs.io/en/release-5/readme.html#what-d... [2] http://conda.pydata.org/docs/intro.html http://conda.pydata.org/docs/intro.html

sametmax 10y ago
+1. Relatively to what we have before, it's so much better. But compared to the JS/Rust ecosystem, we are behind. Now it's hard to compete with JS on some stuff : it's the only language in the most popular dev plateform (the web) and it has one implicit standardized async model by default. It's hard to compete with rust on some stuff : it's compiled and is fast, can provide stand alone binaries easily and has a checker that can avoid many bugs. But this. The package manager. We can compete. And yet we are late. It's partially my fault since it's a project I had in mind for years and never took the time to work on. It's partially everybody's fault I guess :)
- Dowwie 10y ago
  It was definitely your fault. Thanks for nothing.
  - sametmax 10y ago
    :) In the open source world, and given my skills, I always feel a bit responsible when a tool doesn't exist.
- deleted 10y ago
  [deleted]
- HowDoesItWork 10y ago
  Wow, Javascript, really? I am guessing you don't actually work with NPM a lot.
  - overcast 10y ago
    I work with NPM everyday, and it works swimmingly. npm install packagename --save, doesn't get much easier than that.
  - jessaustin 10y ago
    One suspects it's you who hasn't distributed or installed many modules on either python or node. So many of the problems that python has, simply don't exist for node, because it finds modules in a bottom-up hierarchical fashion. That allows a single app or module to use modules that in turn use different versions of other modules, and not to worry about what other modules are doing, or how other modules are installed, or how node is installed, or what version of node is installed. This prevents the traditional "dependency hell" that has plagued devs for decades. Thanks to tools like browserify and webpack, the browser may also benefit from this organization. On top of all that, npm itself just does so many things right. It's quite happy to install from npm repos, from dvcs repos, from regular directories, or from anything that looks like a directory. It just needs to find a single file called "package.json". It requires no build step to prepare a module for upload to an npm repo, but it easily allows for one if that's necessary. package.json itself is basically declarative, but provides scripting hooks for imperative actions if necessary. At every opportunity, npm allows devs to do what they need to do, the easy way. In a sense, node and npm are victims of their own quality. The types of "issues" (e.g. too many deps, too many layers of deps, too many versions of a particular dep, deps that are too trivial, etc.) about which anal code puritans complain with respect to node simply couldn't arise on other platforms, because dependency hell would cause the tower of module dependencies to collapse first. node happily chugs along, blithely ignoring the "problems". Personally, I used to be able to build python packages for distribution, but since I've been spoiled by node and npm for several years I've found I simply can't do that for python anymore. It is so much harder.
    - philsnow 10y ago
- nerdwaller 10y ago
  I'd strongly disagree with Python being behind relative to JS. They're roughly on par (with Python being slightly ahead due to smarter caching and more deterministic builds). Unfortunately neither does namespacing nor good versioning, especially in the cache. Take for example Maven (from Java), it caches namespaced and versioned dependencies and is pretty reliably deterministic in the build process. Verses node which makes horribly assumptions that allow dependencies of libraries to be resolved and dumped into the same library namespace as your application. It also allows users to use the shrinkwrap feature, which imposes the dependency's settings on me (say, for example, I want to use my own npmjs proxy, this is bypassed...)
sjellis 10y ago
This is actually one of the big problems, I think: Python packaging involves knowing a number of different things and reading various resources to get the full picture. Recently, I built a small CLI tool in Python, and learned all of the bits needed to build, test and package my application "the right way". I knew Python syntax before, but it was a lot of effort to set this up. The difference in the experience between Python and Rust or .NET Core is actually shocking, and most it isn't down to anything that Python couldn't do, just the current state of the tooling.
- d0mine 10y ago
  Could you provide some specific examples of the "shocking" difference?
  - bastawhiz 10y ago
    `setup.py` is shockingly awful compared to most other solutions.
  - sjellis 10y ago
    Python best practice: figure out the correct directory structure by reading docs and looking at GitHub repositories, learn how to write setup.py & setup.cfg & requirements.txt & MANIFEST.in files, setup py.test and tox (because Python 2 still lives), write your README in RST format (as used by nothing else ever), and for bonus points: write your own Makefile. Get depressed when you realize that target platforms either don't have Python or have the wrong version. Rust: type "cargo new", README and doc comments in Markdown, type "cargo test" and "cargo build". I'm being deliberately snarky, but you get the point: there has been a slow accretion of complexity over a very long time, and most of it is not the language itself.
    - d0mine 10y ago
      Thank you. It is exactly what I have asked for. The Python best practice: you do not need "correct directory structure" to start programming in Python--there is no boilerplate structure to create (and that would require a tool): REPL, Jupyter notebook in a browser, a single file (module) is enough in a simple case. There is no single "correct directory structure" for more complex cases either e.g., a Django web app and Ubuntu's Appindicator would probably have different structures and it is likely that there are tools that create corresponding project skeletons such as "django-admin startproject mysite". Usually, the packaging/deploying are not nice & neat and differ depending on your project, chosen platform--either you follow an example or you have to study a never ending list of incomplete/contradicting standards/semi-documented rules & conventions--it is unrelated to Python (even if tools such as Ansible, Salt are implemented in Python). Naturally, there are things that can be improved. btw, reStructuredText is a great text format for documenting Python projects (it scales from a simple README.rst to a whole book). You can always convert README.md written in your favorite Markdown flavour using pandoc or a similar tool. Markdown is great for a short human-generated plaintext content common on Github, Stack Overflow.
nerdponx 10y ago
Conda is great software but the Conda/Anaconda ecosystem is a mess.
- pwang 10y ago
  Can you elaborate?

by renesd 10y ago

sametmax 10y ago

+1. Relatively to what we have before, it's so much better. But compared to the JS/Rust ecosystem, we are behind. Now it's hard to compete with JS on some stuff : it's the only language in the most popular dev plateform (the web) and it has one implicit standardized async model by default. It's hard to compete with rust on some stuff : it's compiled and is fast, can provide stand alone binaries easily and has a checker that can avoid many bugs. But this. The package manager. We can compete. And yet we are late. It's partially my fault since it's a project I had in mind for years and never took the time to work on. It's partially everybody's fault I guess :)

Dowwie 10y ago
It was definitely your fault. Thanks for nothing.
- sametmax 10y ago
  :) In the open source world, and given my skills, I always feel a bit responsible when a tool doesn't exist.
deleted 10y ago
[deleted]
HowDoesItWork 10y ago
Wow, Javascript, really? I am guessing you don't actually work with NPM a lot.
- overcast 10y ago
  I work with NPM everyday, and it works swimmingly. npm install packagename --save, doesn't get much easier than that.
- jessaustin 10y ago
  One suspects it's you who hasn't distributed or installed many modules on either python or node. So many of the problems that python has, simply don't exist for node, because it finds modules in a bottom-up hierarchical fashion. That allows a single app or module to use modules that in turn use different versions of other modules, and not to worry about what other modules are doing, or how other modules are installed, or how node is installed, or what version of node is installed. This prevents the traditional "dependency hell" that has plagued devs for decades. Thanks to tools like browserify and webpack, the browser may also benefit from this organization. On top of all that, npm itself just does so many things right. It's quite happy to install from npm repos, from dvcs repos, from regular directories, or from anything that looks like a directory. It just needs to find a single file called "package.json". It requires no build step to prepare a module for upload to an npm repo, but it easily allows for one if that's necessary. package.json itself is basically declarative, but provides scripting hooks for imperative actions if necessary. At every opportunity, npm allows devs to do what they need to do, the easy way. In a sense, node and npm are victims of their own quality. The types of "issues" (e.g. too many deps, too many layers of deps, too many versions of a particular dep, deps that are too trivial, etc.) about which anal code puritans complain with respect to node simply couldn't arise on other platforms, because dependency hell would cause the tower of module dependencies to collapse first. node happily chugs along, blithely ignoring the "problems". Personally, I used to be able to build python packages for distribution, but since I've been spoiled by node and npm for several years I've found I simply can't do that for python anymore. It is so much harder.
  - philsnow 10y ago
nerdwaller 10y ago
I'd strongly disagree with Python being behind relative to JS. They're roughly on par (with Python being slightly ahead due to smarter caching and more deterministic builds). Unfortunately neither does namespacing nor good versioning, especially in the cache. Take for example Maven (from Java), it caches namespaced and versioned dependencies and is pretty reliably deterministic in the build process. Verses node which makes horribly assumptions that allow dependencies of libraries to be resolved and dumped into the same library namespace as your application. It also allows users to use the shrinkwrap feature, which imposes the dependency's settings on me (say, for example, I want to use my own npmjs proxy, this is bypassed...)

sjellis 10y ago

This is actually one of the big problems, I think: Python packaging involves knowing a number of different things and reading various resources to get the full picture. Recently, I built a small CLI tool in Python, and learned all of the bits needed to build, test and package my application "the right way". I knew Python syntax before, but it was a lot of effort to set this up. The difference in the experience between Python and Rust or .NET Core is actually shocking, and most it isn't down to anything that Python couldn't do, just the current state of the tooling.

d0mine 10y ago
Could you provide some specific examples of the "shocking" difference?
- bastawhiz 10y ago
  `setup.py` is shockingly awful compared to most other solutions.
- sjellis 10y ago
  Python best practice: figure out the correct directory structure by reading docs and looking at GitHub repositories, learn how to write setup.py & setup.cfg & requirements.txt & MANIFEST.in files, setup py.test and tox (because Python 2 still lives), write your README in RST format (as used by nothing else ever), and for bonus points: write your own Makefile. Get depressed when you realize that target platforms either don't have Python or have the wrong version. Rust: type "cargo new", README and doc comments in Markdown, type "cargo test" and "cargo build". I'm being deliberately snarky, but you get the point: there has been a slow accretion of complexity over a very long time, and most of it is not the language itself.
  - d0mine 10y ago
    Thank you. It is exactly what I have asked for. The Python best practice: you do not need "correct directory structure" to start programming in Python--there is no boilerplate structure to create (and that would require a tool): REPL, Jupyter notebook in a browser, a single file (module) is enough in a simple case. There is no single "correct directory structure" for more complex cases either e.g., a Django web app and Ubuntu's Appindicator would probably have different structures and it is likely that there are tools that create corresponding project skeletons such as "django-admin startproject mysite". Usually, the packaging/deploying are not nice & neat and differ depending on your project, chosen platform--either you follow an example or you have to study a never ending list of incomplete/contradicting standards/semi-documented rules & conventions--it is unrelated to Python (even if tools such as Ansible, Salt are implemented in Python). Naturally, there are things that can be improved. btw, reStructuredText is a great text format for documenting Python projects (it scales from a simple README.rst to a whole book). You can always convert README.md written in your favorite Markdown flavour using pandoc or a similar tool. Markdown is great for a short human-generated plaintext content common on Github, Stack Overflow.

nerdponx 10y ago

Conda is great software but the Conda/Anaconda ecosystem is a mess.

pwang 10y ago
Can you elaborate?