4 ms·

I think that at this point all the ceremony around setting up and deploying a python project outweighs it's 'easy to read and use' aspects. Unless there is a li

I think that at this point all the ceremony around setting up and deploying a python project outweighs it's 'easy to read and use' aspects. Unless there is a library you can't live without or rewrite, it seems like a language with better tooling and real benefits of a type system is a better choice. If you're mainstream: Go or Java. If you're edgy: Nim, Scala, or Crystal All of those have much more sane type, build, and packaging systems. @perl-people, was this a solved problem when Perl was big? Or is python walking the same roads?

tinco 6y ago
It most definitely was not a solved problem when Perl was big, as far as I know Ruby is the language that finally solved it with bundler, which was released after Rails was, so that's like 2007 or something. The recipe for the 'solved' deployment: - a compiler that ships with a build tool (so building is homogenous in the community) - a centralized or at least uniformly accessible package repository (so dependency acquisition is homogenous in the community) - a common file format for describing dependencies (so dependency resolution is homogenous in the community) - a common file format for locking dependency versions (so deployment can be done reliably without vendoring dependencies) - optional but very nice: a tool for managing compiler versions so it's easy to switch/upgrade projects Any programming language that has all of these boxes ticked is a modern programming language in my book. As far as I know Ruby is the first that ticked all of them, but other ones I've used that have this: Node.js, Go, Rust, Haskell, Python (though it's a bit messy). I'm pretty sure C# checks them as well nowadays, but I haven't used it in over 5 years so I'm not sure. Same for Java.
- sansnomme 6y ago
  Somebody needs to tell the Racket people about this. Dependency management in Racket is still C-style no-version-pinning-anything-goes. The third party dependency managers (see below) are all primitive and do not support multiple versions of the same libraries which is a standard feature in Go/Node.js/Rust/Java class loaders https://github.com/Bogdanp/racksnaps https://github.com/Bogdanp/racksnaps
- autorun 6y ago
  all should be handled by pip. all efforts should be centralized into pip, maybe using a plugin system, whatever. I think a language should have its own official tooling that can achieve those without installing any external shit
BiteCode_dev 6y ago
As said in upper comment, use zipapps. Don't ship your entire env in prod. Deployment then just become: - build zipapp - upload zipapp - ensure python interpretter - run zipapp Which is just one more step than go.
- totalperspectiv 6y ago
  Zipapp doesn't support c extensions though? And that is a large chunk of libraries that I use.
  - BiteCode_dev 6y ago
    It does, if your extensions are provided as wheels. In which case the resulting pyz file will be runnable on any machine using the OS those wheels are compiled for. Let's try an example with numpy: $ py -m venv foo $ foo\Scripts\activate $ pip install numpy $ code hello_numpy.py # import numpy as np # def main(): # print(np.arange(15).reshape(3, 5)) $ python hello_numpy.py [[ 0 1 2 3 4] [ 5 6 7 8 9] [10 11 12 13 14]] Now with shiv: $ copy hello_numpy.py foo\Lib\site-packages $ shiv -e hello_numpy.main --site-packages test\Lib\site-packages\ -o hello_numpy.pyz $ python hello_numpy.pyz [[ 0 1 2 3 4] [ 5 6 7 8 9] [10 11 12 13 14]] So it works fine, but remember: - it will only run on the system this particular numpy wheel has been designed to run on. In my case cp38-win_amd64. - it will comes bundle with numpy. Numpy is quite big, which means your hello world pyz will be 14.1 Mo. - it needs to unzip, so the first run will be slow
    - potatochup 6y ago
      Yeah, Shivs work ok. I've run into some stupid issues with the self-extracting directory being have too long path names on Windows and such, so it's not perfect.
Dowwie 6y ago
if you're long-term focused and investing in a career as a programmer: Rust
MiroF 6y ago
> Unless there is a library you can't live without or rewrite There are quite a few of these. I couldn't do any of my ML work in a language other than Python or C++. Calling what Go has a "type system" is a stretch IMO.
- elcritch 6y ago
  I’ve been toying with Nim lately to wrap C++ ML libraries, which gives a nice Pythonic syntax but compiles to a binary wrapping the ML lib. Seems to work well for Torch. There's a nice wrapper library nimtorch for Nim [1]. It's a bit out of date bit would be easy to update, probably. Well easier than bundling pytorch on an embedded device. Even manually wrapping the needed C++ libraries isn't that hard in Nim, IMHO. Overall looking at Python deployment story after using Elixir for the past couple of years makes me cringe a bit. Rather I have no idea how to do it. Deterministic versions, lock files, and container/tarball (or binary) support seems a given in 2020. 1: https://medium.com/@giovanni_94706/introducing-nimtorch-b8b0fa749464 https://medium.com/@giovanni_94706/introducing-nimtorch-b8b0...
  - MiroF 6y ago
    > Well easier than bundling pytorch on an embedded device. I don't follow - wouldn't nimtorch also require bundling pytorch on an embedded device?
    - elcritch 6y ago
      Yes and no, Pytorch (like Tensorflow for Python) is just a wrapper around a core C++ library. So you can compile a binary from C++ linking to the pytorch libs without including Python. Technically it's pytorch, but without Python and Python dependency management which is way simpler. Nimtorch let's you wrap the C++ api of pytorch with nice Pythonic looking code, a GC, but using only C++ and possibly statically linked. Win win. See: https://pytorch.org/tutorials/advanced/cpp_frontend.html https://pytorch.org/tutorials/advanced/cpp_frontend.html
      - 6y ago

tinco 6y ago

It most definitely was not a solved problem when Perl was big, as far as I know Ruby is the language that finally solved it with bundler, which was released after Rails was, so that's like 2007 or something. The recipe for the 'solved' deployment: - a compiler that ships with a build tool (so building is homogenous in the community) - a centralized or at least uniformly accessible package repository (so dependency acquisition is homogenous in the community) - a common file format for describing dependencies (so dependency resolution is homogenous in the community) - a common file format for locking dependency versions (so deployment can be done reliably without vendoring dependencies) - optional but very nice: a tool for managing compiler versions so it's easy to switch/upgrade projects Any programming language that has all of these boxes ticked is a modern programming language in my book. As far as I know Ruby is the first that ticked all of them, but other ones I've used that have this: Node.js, Go, Rust, Haskell, Python (though it's a bit messy). I'm pretty sure C# checks them as well nowadays, but I haven't used it in over 5 years so I'm not sure. Same for Java.

sansnomme 6y ago
Somebody needs to tell the Racket people about this. Dependency management in Racket is still C-style no-version-pinning-anything-goes. The third party dependency managers (see below) are all primitive and do not support multiple versions of the same libraries which is a standard feature in Go/Node.js/Rust/Java class loaders https://github.com/Bogdanp/racksnaps https://github.com/Bogdanp/racksnaps
autorun 6y ago
all should be handled by pip. all efforts should be centralized into pip, maybe using a plugin system, whatever. I think a language should have its own official tooling that can achieve those without installing any external shit

BiteCode_dev 6y ago

As said in upper comment, use zipapps. Don't ship your entire env in prod. Deployment then just become: - build zipapp - upload zipapp - ensure python interpretter - run zipapp Which is just one more step than go.

totalperspectiv 6y ago
Zipapp doesn't support c extensions though? And that is a large chunk of libraries that I use.
- BiteCode_dev 6y ago
  It does, if your extensions are provided as wheels. In which case the resulting pyz file will be runnable on any machine using the OS those wheels are compiled for. Let's try an example with numpy: $ py -m venv foo $ foo\Scripts\activate $ pip install numpy $ code hello_numpy.py # import numpy as np # def main(): # print(np.arange(15).reshape(3, 5)) $ python hello_numpy.py [[ 0 1 2 3 4] [ 5 6 7 8 9] [10 11 12 13 14]] Now with shiv: $ copy hello_numpy.py foo\Lib\site-packages $ shiv -e hello_numpy.main --site-packages test\Lib\site-packages\ -o hello_numpy.pyz $ python hello_numpy.pyz [[ 0 1 2 3 4] [ 5 6 7 8 9] [10 11 12 13 14]] So it works fine, but remember: - it will only run on the system this particular numpy wheel has been designed to run on. In my case cp38-win_amd64. - it will comes bundle with numpy. Numpy is quite big, which means your hello world pyz will be 14.1 Mo. - it needs to unzip, so the first run will be slow
  - potatochup 6y ago
    Yeah, Shivs work ok. I've run into some stupid issues with the self-extracting directory being have too long path names on Windows and such, so it's not perfect.

Dowwie 6y ago

if you're long-term focused and investing in a career as a programmer: Rust

MiroF 6y ago

> Unless there is a library you can't live without or rewrite There are quite a few of these. I couldn't do any of my ML work in a language other than Python or C++. Calling what Go has a "type system" is a stretch IMO.

elcritch 6y ago
I’ve been toying with Nim lately to wrap C++ ML libraries, which gives a nice Pythonic syntax but compiles to a binary wrapping the ML lib. Seems to work well for Torch. There's a nice wrapper library nimtorch for Nim [1]. It's a bit out of date bit would be easy to update, probably. Well easier than bundling pytorch on an embedded device. Even manually wrapping the needed C++ libraries isn't that hard in Nim, IMHO. Overall looking at Python deployment story after using Elixir for the past couple of years makes me cringe a bit. Rather I have no idea how to do it. Deterministic versions, lock files, and container/tarball (or binary) support seems a given in 2020. 1: https://medium.com/@giovanni_94706/introducing-nimtorch-b8b0fa749464 https://medium.com/@giovanni_94706/introducing-nimtorch-b8b0...
- MiroF 6y ago
  > Well easier than bundling pytorch on an embedded device. I don't follow - wouldn't nimtorch also require bundling pytorch on an embedded device?
  - elcritch 6y ago
    Yes and no, Pytorch (like Tensorflow for Python) is just a wrapper around a core C++ library. So you can compile a binary from C++ linking to the pytorch libs without including Python. Technically it's pytorch, but without Python and Python dependency management which is way simpler. Nimtorch let's you wrap the C++ api of pytorch with nice Pythonic looking code, a GC, but using only C++ and possibly statically linked. Win win. See: https://pytorch.org/tutorials/advanced/cpp_frontend.html https://pytorch.org/tutorials/advanced/cpp_frontend.html
    - 6y ago