7 ms·

Cool, but I'm so tired of Python that I've started to avoid any projects written with it. Dealing with dependencies, 2 vs. 3, pip, virtualenv, etc is such a pai

Cool, but I'm so tired of Python that I've started to avoid any projects written with it. Dealing with dependencies, 2 vs. 3, pip, virtualenv, etc is such a pain. I think the Java/Go/Rust model where all dependencies are included in the binary is the way to go.

idoubtit 7y ago
In my opinion, the worst problem is the lack of a standard way to handle dependencies. I recently had a hard time deploying a small Python project over a few heterogeneous Linux machines. For example, this "simple, lightweight" program has dependencies, as one can see in its .travis.yaml file. That's why the readme has the command `pip install commonmark` (plus more dependencies for dev). But, wait, doesn't `pip` install as root in /usr/local/ by default? (BTW, the documentation of `pip install` is mute on the subject.) So I should use pipenv, but last time I tried it was broken and the ticket had low activity. So virtualenv + pip, but it's a mess since I like using many terminals. I've heard about poetry, but that's not standard at all...
- jacquesm 7y ago
  Python deployment is a shitshow. Unfortunately mission critical stuff (looking at you letsencrypt) is also written in it which means that your systems are built on very shaky foundations.
  - kalia35 7y ago
    What ? This seems like a bold affirmation. I find it very simple actually : apt install python3 virtualenv virtualenv env . env/bin/activate pip install -r requirements.txt Done. And then with fabric you can automate deployments in a few lines of codes !
    - jacquesm 7y ago
      Right up until it nukes your perfectly working server. The happy case works, it always does. The problem is in the edge cases and the exceptions. This probably goes for the bulk of all software.
      - kalia35 7y ago
        Well in ten years I have never had a nuked server by pip or virtualenv. I am sure there are edge cases but that must be true for all the ways of deploying stuff. I guess the ones you know well appear simple to you !
        jacquesm 7y ago
        I've had a production server be hosed by running 'letsencrypt-auto'. My bad I guess but still, that's one that I really did not see coming.
        StavrosK 7y ago
        I kind of agree with both of you. I don't think Python deployment is that bad, as most modern managers get it right (I prefer Poetry nowadays). However something like letsencrypt-auto, which AFAIK installs its own dependencies, is not something I'd run. I've actually never used it precisely because it's too magical. Programs shouldn't install their own dependencies anywhere outside of a virtualenv.
        jklehm 7y ago
        Have you heard of our Lord and Savior pipx? Solves the python tool isolation issue nicely.
        StavrosK 7y ago
        I haven't used it, I use virtualfish which does a great job as well.
  - fanf2 7y ago
    We like `dehydrated` for ACME
    - tedmiston 7y ago
      +1 https://github.com/lukas2511/dehydrated https://github.com/lukas2511/dehydrated
      - caro_douglos 7y ago
        Have you always used this or did you make a transition after LE burned you?
        tedmiston 7y ago
        I heard about dehydrated from PythonAnywhere [1]. I haven't ever been burned by LE from a Python library, but I have had issues in the past with the kube-lego Helm chart not renewing as it should (easily fixed by deleting the pod though). [I know kube-lego is in maintenance mode now in favor of cert-manager.] [1]: https://help.pythonanywhere.com/pages/LetsEncrypt https://help.pythonanywhere.com/pages/LetsEncrypt
  - xmprt 7y ago
    People who know what they're doing can build a lot of things with python. It just so happens that I am not one of those people so I try to avoid it with a ten foot stick. I still use things built with python everyday.
- mixmastamyk 7y ago
  It is not hard, it is just folks desimplifying things to solve largely theoretical problems. Just pip install to the location you choose. sudo or —user is fine 99% of the time. Venvs are only needed for big work projects.
- kalia35 7y ago
  Why is virtualenv a mess with multiple terminals ? pip install for root unless you activate a virtualenv in which case it installs in it. If you make an alias of . env/bin/activate it is very simple, no ?
- marmaduke 7y ago
  You should use pip install --user, which pip will tell you about if you run pip install -h. Virtualenv is usually excessive unless you have to handle lots of version problems
- tedmiston 7y ago
  Dockerizing Python apps provides a nice isolation that makes managing dependencies across multiple apps much easier.
drunken-serval 7y ago
Or go to the dark side and install Wordpress. It’s actually not as bad as I thought it would be. I modified one of the twenty-x themes that come with it and installed a few plugins to make a decent personal site. It took less time that it would have to create anything with remotely similar functionality. I run it off a single vps that has no other purpose. If it gets hacked, nothing of value is lost. I can revoke the api keys for the external services I use and restore to a new server from my backups.
- x3haloed 7y ago
  More like the vulnerable side.
  - drunken-serval 7y ago
    Any software has vulnerabilities. You have to keep the operating system on any server patched and set up basic defenses even for a static site.
    - theamk 7y ago
      Well, there are more and less vulnerable things. For example, making sure that software cannot modify itself will ensure that even if server is vulnerable, it won’t get permanently compromised. Having a server that does not execute any file with right extension makes sure that sanitization errors do not lead to code execution. Having admin system be separate from main site makes sure XSS cannot cause compromises.
  - stevenicr 7y ago
    with a default wp install - you are right, however it's trivial to add a couple of security plugins and turn on auto-update. adding the "ipgeoblock plugin" wipes out most attacks straight away. with some of my wp sites that got attacked a lot on a regular basis, I use a 'static html generator' plugin - and delete all the wp php files - no way to login, add comments or hack the wp core or plugins or themes, since they are not in use when you convert it all the static html. On wp sites where I actually add content with regularity, I don't delete the wp files, and just use shield, sucuri, ipgeoblock, plugin things like that depending on threat.
suyash 7y ago
I would have considered if this project was using Node.js instead Python as well.
- dastx 7y ago
  Sure, let's install 1 million dependencies instead of 5.
  - bigiain 7y ago
    gulp build Error: cannot find dependency 'left pad.js'
cxr 7y ago
> I think the Java/Go/Rust model where all dependencies are included in the binary is the way to go. This is how triple scripts are designed[1] to work. One proof-of-concept just so happens to be a minimalist static site generator called triickl[2]. You can drop it onto (virtually) any machine, double click, and point it at a source directory for your site—subject to the expectation that it follows a certain jekyll-like structure—and then you're off to the races. I'm using it as the build step for the triplescripts.org home page itself and my personal site. 1. https://triplescripts.org/format#self-contained https://triplescripts.org/format#self-contained 1. https://releases.triplescripts.org/triickl/ https://releases.triplescripts.org/triickl/
diminoten 7y ago
Have you tried writing a web app in Java/Go/Rust vs. Python in the past year?
joshklein 7y ago
You raise a fair general point about Python dependencies, but that issue is usually related to poorly considered use of third-party dependencies or the need to distribute binaries and leverage C bindings. But here we have exactly the kind of project that should be written in Python, forked and cloned by you, then tweaked and maintained as if you wrote it yourself. From a brief glance through the code, and taking the submission title at face value, this project is a refreshingly appropriate use of the right language. I offer this as someone who writes Python and Go, generally preferring Go anywhere possible. I plan to poke around some more in this repo to see if I can easily replace my use of Hugo with this, since I use only a tiny fraction of the features in Hugo and have found it challenging to hack on it as a low-priority weekend project. For turning personal blog posts from markdown to HTML with some general convenience functions around maintaining the front-matter and templates, I can’t think of something more appropriate than a Python script sitting inside your blog directory with all of its dependencies except the interpreter itself checked in with your templates and content.
- RobertRoberts 7y ago
  I won't bother with Python again because these conflicts happen on your first time even trying out the language. (running linux, any better on Windows?) It was such a mess that I can't even describe the frustration. And I even got Apache working with SVN! (40 pages of docs and no testing points from start till it's fully working, yuck) EDIT: Since I got downvotes so quick, tell me why anyone should choose Python over the many alternative out now?
  - joshklein 7y ago
    There’s too much to untangle for a brief explanation, but if this really happened when you tried writing Python on Linux for the first time, or using a program written in Python you installed through your distribution’s package manager, file a bug report with your distribution.
    - RobertRoberts 7y ago
      It wasn't "bugs" it was all the hoops to jump through, conflicting hoops, multiple installed versions, pip vs all the other package installers, that also caused conflicts, etc... Since I am not a Python dev, I would be unable to provide a useful report even if it were bugs.
theamk 7y ago
I’ll by Go or Rust, but Java? I have spent too much time fixing java software recently.. it’s dependencies are a complete pain.
tedmiston 7y ago
You can create and distribute all-in-one Python executables with PyInstaller. I've used it in multistage Docker builds to produce very clean, minimal production images like what people commonly do in Go. https://pyinstaller.readthedocs.io/en/stable/usage.html https://pyinstaller.readthedocs.io/en/stable/usage.html

idoubtit 7y ago

In my opinion, the worst problem is the lack of a standard way to handle dependencies. I recently had a hard time deploying a small Python project over a few heterogeneous Linux machines. For example, this "simple, lightweight" program has dependencies, as one can see in its .travis.yaml file. That's why the readme has the command `pip install commonmark` (plus more dependencies for dev). But, wait, doesn't `pip` install as root in /usr/local/ by default? (BTW, the documentation of `pip install` is mute on the subject.) So I should use pipenv, but last time I tried it was broken and the ticket had low activity. So virtualenv + pip, but it's a mess since I like using many terminals. I've heard about poetry, but that's not standard at all...

jacquesm 7y ago
Python deployment is a shitshow. Unfortunately mission critical stuff (looking at you letsencrypt) is also written in it which means that your systems are built on very shaky foundations.
- kalia35 7y ago
  What ? This seems like a bold affirmation. I find it very simple actually : apt install python3 virtualenv virtualenv env . env/bin/activate pip install -r requirements.txt Done. And then with fabric you can automate deployments in a few lines of codes !
  - jacquesm 7y ago
    Right up until it nukes your perfectly working server. The happy case works, it always does. The problem is in the edge cases and the exceptions. This probably goes for the bulk of all software.
    - kalia35 7y ago
      Well in ten years I have never had a nuked server by pip or virtualenv. I am sure there are edge cases but that must be true for all the ways of deploying stuff. I guess the ones you know well appear simple to you !
      - jacquesm 7y ago
        I've had a production server be hosed by running 'letsencrypt-auto'. My bad I guess but still, that's one that I really did not see coming.
        StavrosK 7y ago
        I kind of agree with both of you. I don't think Python deployment is that bad, as most modern managers get it right (I prefer Poetry nowadays). However something like letsencrypt-auto, which AFAIK installs its own dependencies, is not something I'd run. I've actually never used it precisely because it's too magical. Programs shouldn't install their own dependencies anywhere outside of a virtualenv.
        jklehm 7y ago
        Have you heard of our Lord and Savior pipx? Solves the python tool isolation issue nicely.
        StavrosK 7y ago
        I haven't used it, I use virtualfish which does a great job as well.
- fanf2 7y ago
  We like `dehydrated` for ACME
  - tedmiston 7y ago
    +1 https://github.com/lukas2511/dehydrated https://github.com/lukas2511/dehydrated
    - caro_douglos 7y ago
      Have you always used this or did you make a transition after LE burned you?
      - tedmiston 7y ago
        I heard about dehydrated from PythonAnywhere [1]. I haven't ever been burned by LE from a Python library, but I have had issues in the past with the kube-lego Helm chart not renewing as it should (easily fixed by deleting the pod though). [I know kube-lego is in maintenance mode now in favor of cert-manager.] [1]: https://help.pythonanywhere.com/pages/LetsEncrypt https://help.pythonanywhere.com/pages/LetsEncrypt
- xmprt 7y ago
  People who know what they're doing can build a lot of things with python. It just so happens that I am not one of those people so I try to avoid it with a ten foot stick. I still use things built with python everyday.
mixmastamyk 7y ago
It is not hard, it is just folks desimplifying things to solve largely theoretical problems. Just pip install to the location you choose. sudo or —user is fine 99% of the time. Venvs are only needed for big work projects.
kalia35 7y ago
Why is virtualenv a mess with multiple terminals ? pip install for root unless you activate a virtualenv in which case it installs in it. If you make an alias of . env/bin/activate it is very simple, no ?
marmaduke 7y ago
You should use pip install --user, which pip will tell you about if you run pip install -h. Virtualenv is usually excessive unless you have to handle lots of version problems
tedmiston 7y ago
Dockerizing Python apps provides a nice isolation that makes managing dependencies across multiple apps much easier.

drunken-serval 7y ago

Or go to the dark side and install Wordpress. It’s actually not as bad as I thought it would be. I modified one of the twenty-x themes that come with it and installed a few plugins to make a decent personal site. It took less time that it would have to create anything with remotely similar functionality. I run it off a single vps that has no other purpose. If it gets hacked, nothing of value is lost. I can revoke the api keys for the external services I use and restore to a new server from my backups.

x3haloed 7y ago
More like the vulnerable side.
- drunken-serval 7y ago
  Any software has vulnerabilities. You have to keep the operating system on any server patched and set up basic defenses even for a static site.
  - theamk 7y ago
    Well, there are more and less vulnerable things. For example, making sure that software cannot modify itself will ensure that even if server is vulnerable, it won’t get permanently compromised. Having a server that does not execute any file with right extension makes sure that sanitization errors do not lead to code execution. Having admin system be separate from main site makes sure XSS cannot cause compromises.
- stevenicr 7y ago
  with a default wp install - you are right, however it's trivial to add a couple of security plugins and turn on auto-update. adding the "ipgeoblock plugin" wipes out most attacks straight away. with some of my wp sites that got attacked a lot on a regular basis, I use a 'static html generator' plugin - and delete all the wp php files - no way to login, add comments or hack the wp core or plugins or themes, since they are not in use when you convert it all the static html. On wp sites where I actually add content with regularity, I don't delete the wp files, and just use shield, sucuri, ipgeoblock, plugin things like that depending on threat.

suyash 7y ago

I would have considered if this project was using Node.js instead Python as well.

dastx 7y ago
Sure, let's install 1 million dependencies instead of 5.
- bigiain 7y ago
  gulp build Error: cannot find dependency 'left pad.js'

cxr 7y ago

> I think the Java/Go/Rust model where all dependencies are included in the binary is the way to go. This is how triple scripts are designed[1] to work. One proof-of-concept just so happens to be a minimalist static site generator called triickl[2]. You can drop it onto (virtually) any machine, double click, and point it at a source directory for your site—subject to the expectation that it follows a certain jekyll-like structure—and then you're off to the races. I'm using it as the build step for the triplescripts.org home page itself and my personal site. 1. https://triplescripts.org/format#self-contained https://triplescripts.org/format#self-contained 1. https://releases.triplescripts.org/triickl/ https://releases.triplescripts.org/triickl/

diminoten 7y ago

Have you tried writing a web app in Java/Go/Rust vs. Python in the past year?

joshklein 7y ago

You raise a fair general point about Python dependencies, but that issue is usually related to poorly considered use of third-party dependencies or the need to distribute binaries and leverage C bindings. But here we have exactly the kind of project that should be written in Python, forked and cloned by you, then tweaked and maintained as if you wrote it yourself. From a brief glance through the code, and taking the submission title at face value, this project is a refreshingly appropriate use of the right language. I offer this as someone who writes Python and Go, generally preferring Go anywhere possible. I plan to poke around some more in this repo to see if I can easily replace my use of Hugo with this, since I use only a tiny fraction of the features in Hugo and have found it challenging to hack on it as a low-priority weekend project. For turning personal blog posts from markdown to HTML with some general convenience functions around maintaining the front-matter and templates, I can’t think of something more appropriate than a Python script sitting inside your blog directory with all of its dependencies except the interpreter itself checked in with your templates and content.

RobertRoberts 7y ago
I won't bother with Python again because these conflicts happen on your first time even trying out the language. (running linux, any better on Windows?) It was such a mess that I can't even describe the frustration. And I even got Apache working with SVN! (40 pages of docs and no testing points from start till it's fully working, yuck) EDIT: Since I got downvotes so quick, tell me why anyone should choose Python over the many alternative out now?
- joshklein 7y ago
  There’s too much to untangle for a brief explanation, but if this really happened when you tried writing Python on Linux for the first time, or using a program written in Python you installed through your distribution’s package manager, file a bug report with your distribution.
  - RobertRoberts 7y ago
    It wasn't "bugs" it was all the hoops to jump through, conflicting hoops, multiple installed versions, pip vs all the other package installers, that also caused conflicts, etc... Since I am not a Python dev, I would be unable to provide a useful report even if it were bugs.

theamk 7y ago

I’ll by Go or Rust, but Java? I have spent too much time fixing java software recently.. it’s dependencies are a complete pain.

tedmiston 7y ago

You can create and distribute all-in-one Python executables with PyInstaller. I've used it in multistage Docker builds to produce very clean, minimal production images like what people commonly do in Go. https://pyinstaller.readthedocs.io/en/stable/usage.html https://pyinstaller.readthedocs.io/en/stable/usage.html