8 ms·

Is there a model dependency management system for some other language that addresses all these issues? Dependency hell is everywhere.

dehrmann 7y ago
It's not a "model," but if you're able to 1) use fewer dependencies 2) use stable dependencies 3) use dependencies with fewer dependencies, it helps with dependency hell. I've even made commits to projects to reduce their dependency count.
adrianN 7y ago
I find rust with cargo and go with go-modules to be pretty nice to work with.
RobertRoberts 7y ago
Does it have to be? I have found that I would rather code my own versions of some libraries so I have control over it. Even if there is some extra long term maintenance and some up front dev costs, it's paid off already a number of times.
- mark_l_watson 7y ago
  A little off topic, but this is why I really like a Common Lisp with Quicklisp: library dependencies are stored in a convenient location locally and the libraries I write can be treated the same way (with a trivial config change to add Quicklisp load paths to my own library project directories).
itronitron 7y ago
maven for java
ajmurmann 7y ago
Rust's cargo, JS's yarn and the grand daddy of them all Ruby's bundler address all these issues. Even newer versions of Gradle support a workflow where you specify the versions you know you want and just on everything else, including transitive dependencies, down.
- jbverschoor 7y ago
  Yeah.. not sure why the love for Ruby faded in startup-world
  - PikachuEXE 7y ago
    I think many people still "love" ruby I never liked Python from day 1 of my work (I am Ruby on Rails Dev for my job) Wish there are more useful projects written in ruby :)
    - fellellor 7y ago
      I am trying to find a place in the industry - again, starting from RoR. I absolutely love Ruby. And all this talk of "Ruby dying" makes me feel sad. The rational thing to do is to move on, and learn something popular, like node.js but the more I see Ruby in action, I just can't pull myself away from it. I had managed to get a job as a Java developer a long time ago, but at that time all I could do was barely write toy apps in Java and I had no exposure to stuff like design patterns. The whole experience left me in a bad place. Now after all these years, Ruby feels like a breath of fresh air, and the texts that I have come across on the subject - Design Patterns in Ruby, Practical OOD in Ruby, Ruby under a Microscope etc. have increased my interest in the language. But more and more frequent articles on Ruby's decline are pretty disheartening.
      - jhbadger 7y ago
        Ruby's future may actually not be Ruby itself. Probably the major problem with Ruby is its performance, which is slow even compared to other interpreted languages. While I'm not sure it is really production ready yet, Crystal is very interesting -- it's a native compiled statically typed language that nevertheless feels very much like Ruby. Check it out if you haven't.
        jbverschoor 7y ago
        It’s faster than python. And crystal is often faster than go.
atoav 7y ago
> Dependency hell is everywhere. Excuse me, but as a long time Python user I have to disagree. I started using Rust two years ago and Rust’s dependency managment is easily the best thing I ever saw (keep in mind that I didn’t see everything, so there is a chance there are better things out there). The project-/dependency-manager Cargo¹ is more “pythonic” than anything Python ever came up with and where others mumbled ”dependency hell is everywhere” the Rust people seem to have thought something like: ”there must be a way to do this properly”. The whole thing gives me hope for Python, but angers me everytime I start a new python project. Poetry is good, but there should be an official way to do this. It saddens me to see, that some people seem to just have given up on dependency managment alltogether and declared it unsolvable (which it is not, and probably never has been). ¹: https://doc.rust-lang.org/cargo/index.html https://doc.rust-lang.org/cargo/index.html
- jnwatson 7y ago
  There's a pattern to this. The later the dependency manager was created, the better it is. This is a hard problem space where each new language got to use the lessons learned on the earlier ones. Cargo, though, has a silver bullet. If it can't find a solution to determine a single version for a package, it simply includes more than one version in the object code. That would take a lot of work to duplicate in Python.
  - atoav 7y ago
    They definitly gave this a thought while designing the library system (”crates”) for the language. I am not sure if it is feasible to retrofit such a solution to something like python.. Python 4 maybe?
  - icebraining 7y ago
    Python has had multi-version installation of libraries for a long time: https://packaging.python.org/guides/multi-version-installs/ https://packaging.python.org/guides/multi-version-installs/ Unfortunately, pip came along and took over, despite lacking support for that. It would have been nice if a better packaging tool had replaced easy_install, but alas.
marmada 7y ago
Python is uniquely ill-suited for dependency management compared to many other languages. For some reason dependencies are installed into the interpreter itself (I know what I just said is very imprecise/inaccurate but I think it gets the point across). In JS, which also has a single interpreter installed across the system (or multiple if you use nvm), the packages aren't installed "directly" into the interpreter, which removes the need for things like virtual-envs, thus making life a lot easier. I wish Python did something like this. That being said, pipenv is making things easier. However, I think pipenv is a workaround more fundamental problems.
- icebraining 7y ago
  Not trying to defend Python, but it's not hard to make it work like JS without virtualenv: pip install -r requirements.txt --target=python_modules Then to run: PYTHONPATH=python_modules python myscript.py This will keep all the dependencies in the local "python_modules" directory. I've used this on a custom SaaS platform, and it worked quite well.
  - StavrosK 7y ago
    Also, that's more or less what virtualenv does. It would be nice if this was default, but IIRC it's coming.
  - mark_l_watson 7y ago
    Thanks for that! I am tired of messing with conda, virtualenv, etc. and since I use simple Makefiles to build and run most of my code, I can easily stick with the standard latest stable version Python installation when using your trick. EDIT: a question: when I have to use Python, I like to break up my code into small libraries and make wheel files for my own use. How do you handle your own libraries? Do you have one special local directory that you build wheel files to and then reference that library in your requirements.txt files?
    - icebraining 7y ago
      > How do you handle your own libraries? Do you have one special local directory that you build wheel files to and then reference that library in your requirements.txt files? We didn't build wheels. We had a centralized git host (Gitlab, but any of them works) with all our libraries, and just added the git url (git+https://... https://...) to the requirements.txt

dehrmann 7y ago

It's not a "model," but if you're able to 1) use fewer dependencies 2) use stable dependencies 3) use dependencies with fewer dependencies, it helps with dependency hell. I've even made commits to projects to reduce their dependency count.

adrianN 7y ago

I find rust with cargo and go with go-modules to be pretty nice to work with.

RobertRoberts 7y ago

Does it have to be? I have found that I would rather code my own versions of some libraries so I have control over it. Even if there is some extra long term maintenance and some up front dev costs, it's paid off already a number of times.

mark_l_watson 7y ago
A little off topic, but this is why I really like a Common Lisp with Quicklisp: library dependencies are stored in a convenient location locally and the libraries I write can be treated the same way (with a trivial config change to add Quicklisp load paths to my own library project directories).

itronitron 7y ago

maven for java

ajmurmann 7y ago

Rust's cargo, JS's yarn and the grand daddy of them all Ruby's bundler address all these issues. Even newer versions of Gradle support a workflow where you specify the versions you know you want and just on everything else, including transitive dependencies, down.

jbverschoor 7y ago
Yeah.. not sure why the love for Ruby faded in startup-world
- PikachuEXE 7y ago
  I think many people still "love" ruby I never liked Python from day 1 of my work (I am Ruby on Rails Dev for my job) Wish there are more useful projects written in ruby :)
  - fellellor 7y ago
    I am trying to find a place in the industry - again, starting from RoR. I absolutely love Ruby. And all this talk of "Ruby dying" makes me feel sad. The rational thing to do is to move on, and learn something popular, like node.js but the more I see Ruby in action, I just can't pull myself away from it. I had managed to get a job as a Java developer a long time ago, but at that time all I could do was barely write toy apps in Java and I had no exposure to stuff like design patterns. The whole experience left me in a bad place. Now after all these years, Ruby feels like a breath of fresh air, and the texts that I have come across on the subject - Design Patterns in Ruby, Practical OOD in Ruby, Ruby under a Microscope etc. have increased my interest in the language. But more and more frequent articles on Ruby's decline are pretty disheartening.
    - jhbadger 7y ago
      Ruby's future may actually not be Ruby itself. Probably the major problem with Ruby is its performance, which is slow even compared to other interpreted languages. While I'm not sure it is really production ready yet, Crystal is very interesting -- it's a native compiled statically typed language that nevertheless feels very much like Ruby. Check it out if you haven't.
      - jbverschoor 7y ago
        It’s faster than python. And crystal is often faster than go.

atoav 7y ago

> Dependency hell is everywhere. Excuse me, but as a long time Python user I have to disagree. I started using Rust two years ago and Rust’s dependency managment is easily the best thing I ever saw (keep in mind that I didn’t see everything, so there is a chance there are better things out there). The project-/dependency-manager Cargo¹ is more “pythonic” than anything Python ever came up with and where others mumbled ”dependency hell is everywhere” the Rust people seem to have thought something like: ”there must be a way to do this properly”. The whole thing gives me hope for Python, but angers me everytime I start a new python project. Poetry is good, but there should be an official way to do this. It saddens me to see, that some people seem to just have given up on dependency managment alltogether and declared it unsolvable (which it is not, and probably never has been). ¹: https://doc.rust-lang.org/cargo/index.html https://doc.rust-lang.org/cargo/index.html

jnwatson 7y ago
There's a pattern to this. The later the dependency manager was created, the better it is. This is a hard problem space where each new language got to use the lessons learned on the earlier ones. Cargo, though, has a silver bullet. If it can't find a solution to determine a single version for a package, it simply includes more than one version in the object code. That would take a lot of work to duplicate in Python.
- atoav 7y ago
  They definitly gave this a thought while designing the library system (”crates”) for the language. I am not sure if it is feasible to retrofit such a solution to something like python.. Python 4 maybe?
- icebraining 7y ago
  Python has had multi-version installation of libraries for a long time: https://packaging.python.org/guides/multi-version-installs/ https://packaging.python.org/guides/multi-version-installs/ Unfortunately, pip came along and took over, despite lacking support for that. It would have been nice if a better packaging tool had replaced easy_install, but alas.

marmada 7y ago

Python is uniquely ill-suited for dependency management compared to many other languages. For some reason dependencies are installed into the interpreter itself (I know what I just said is very imprecise/inaccurate but I think it gets the point across). In JS, which also has a single interpreter installed across the system (or multiple if you use nvm), the packages aren't installed "directly" into the interpreter, which removes the need for things like virtual-envs, thus making life a lot easier. I wish Python did something like this. That being said, pipenv is making things easier. However, I think pipenv is a workaround more fundamental problems.

icebraining 7y ago
Not trying to defend Python, but it's not hard to make it work like JS without virtualenv: pip install -r requirements.txt --target=python_modules Then to run: PYTHONPATH=python_modules python myscript.py This will keep all the dependencies in the local "python_modules" directory. I've used this on a custom SaaS platform, and it worked quite well.
- StavrosK 7y ago
  Also, that's more or less what virtualenv does. It would be nice if this was default, but IIRC it's coming.
- mark_l_watson 7y ago
  Thanks for that! I am tired of messing with conda, virtualenv, etc. and since I use simple Makefiles to build and run most of my code, I can easily stick with the standard latest stable version Python installation when using your trick. EDIT: a question: when I have to use Python, I like to break up my code into small libraries and make wheel files for my own use. How do you handle your own libraries? Do you have one special local directory that you build wheel files to and then reference that library in your requirements.txt files?
  - icebraining 7y ago
    > How do you handle your own libraries? Do you have one special local directory that you build wheel files to and then reference that library in your requirements.txt files? We didn't build wheels. We had a centralized git host (Gitlab, but any of them works) with all our libraries, and just added the git url (git+https://... https://...) to the requirements.txt