6 ms·

It seems like a lot of these virtual environment tools are designed to work around the assumption that packages are installed globally on a system. I don’t see

It seems like a lot of these virtual environment tools are designed to work around the assumption that packages are installed globally on a system. I don’t see why this is so hard to avoid with per-project version pinning. I’ve never had this problem with Java. This seems like a lot of effort and complexity to work around global package installation. Just stop assuming that packages are global and this would all go away. How do we know that? Because other languages don’t require this kind of workaround!

tootie 5y ago
Java (well Maven) I believe does this the most clever and simple way. It does have a global folder of dependencies but will gladly stash multiple versions of the same library. Local projects just link to the right version. That saves downloads and allows for simple version pinning. And JVMs are backwards compatible so having a new JVM can still run bytecode for past versions with just a flag. Features like this are so much more valuable than anything that goes into the actual language syntax.
- npteljes 5y ago
  I come from the Java world and it's mindblowing to see that mature, widespread languages like Ruby, Python or JavaScript require all kinds of hackery to run properly. Dependency management with Maven is a breeze in comparison.
  - Tainnor 5y ago
    What do you mean by Ruby "hackery"? I'll agree that how to do things properly with Ruby isn't obvious when you're new to the language - just as it isn't obvious how to do things in Java (or other JVM languages) when you're coming from Ruby or another language.[0] But Bundler doesn't work all that differently from Maven, in the end. Instead of running your commands through `./mwnw`, you run them through `bundle exec`[1]. Dependency management is straightforward and mostly hassle-free (the problems arise "only" when you use C extensions which, to be fair, is not that rare, but I don't really see a way around this in a language where you need C for certain fast things). In particular, Bundler by default[2] also stores dependencies centrally like maven to avoid duplication and then selects the appropriate version of a given dependency (unlike e.g. NPM which stores everything locally). Rbenv (or alternative tools such as chruby or asdf) solves a problem completely distinct from Bundler or Maven, it's about selecting the right Ruby version for your project, a problem that also needs to be solved with Java. IME, many developers just let the IDE deal with that, but Ruby developers are more used to running things from CLI, so that's why rbenv exists. In any case, the most popular analogue for Java would be sdkman which works, mostly, the same way (and which, IME, is even a bit hackier than rbenv). JS and Python are different. Node is a mess IMHO, and Python isn't able to agree on which package manager to use (I would hope people would migrate to Poetry, which seems to be doing things right). [0]: For example, one thing that I find very annoying over in Java-land is that it's a real hassle to make sure things behave the same way when called from the CLI as opposed to from the IDE, such as setting the active spring profile. This is because, mostly, the IDE circumvents maven/gradle (you can run things through the build system, but typically the output, e.g. test output, is better when using the IDE-internal system, at least with IntelliJ). [1]: There are "hackeries" to get rid of having to run everything through "bundle exec", such as binstubs and direnv, although I would mostly call them "shell script tricks", and anyway, if you instead just alias "bundle exec" to something shorter or create a wrapper script, it's not really worse than `./mwnw`. [2]: This can be overriden and is actually a good idea for production builds, especially if you use a multi-stage docker build since then it's easy to copy over all your dependencies.
    - tootie 5y ago
      I think you're missing that Maven does actually solve the rvm/nvm/pyenv problem. You can specify the bytecode version in the config and compile/run any backward compatible version. Your actual JDK need only be new enough, not an exact match and it doesn't need to change between projects. I have never had a single compiler or runtime version compatibility issue even with native dependencies ever with Java. The IDE situation can have some subtle edge cases, but no other language (save C#) actually gains so much utility from an IDE. Stack traces, break points, runtime inspection and profiling are all nearly flawless in IntelliJ (or Visual Studio) in ways that are monstrously difficult with any dynamic language.
      - Tainnor 5y ago
        Yes, but now you're not comparing maven vs. Bundler, you're comparing statically vs. dynamically typed languages. It's almost impossible to get the strong guarantees you get in a statically typed language in a dynamically typed one. If that's the discussion you want to have, well, I personally also prefer statically typed languages, but that's beyond the point. That said, I've had issues, e.g. on CI, that didn't manifest locally because of a mismatch in Java versions (something with JAXB, IIRC), and Java is maybe sometimes a bit too cavalier about the issue.
    - npteljes 5y ago
      By Ruby, Python and JavaScript hackery I was meaning rvm, venv / pip, and npm. I have the most experience with rvm, and the hackery part comes from the experience when I created a web interface that could instruct the server to do ruby tasks. Turned out that rvm (and also bundler) does a bunch of stuff to the environment of the user, that's sometimes inaccessible, even if you invoke a "login shell" programatically. One of these "hacks" is doing a bunch of stuff when the user navigates to a Ruby project directory with cd, which is when rvm creates or uses the existing virtual env for the project - named is a specific way based on the directory name. I solved this of course but the discovery was quite annoying. Regarding Python, I don't have much experience, other than the general feeling that it's fragile and that specific projects work with specific dependency managegers / virtual environments, and they are not really interchangeable or anything. I either could replicate the env or I couldn't. I definitely don't have much of a Python experience. Now regarding your Java IDE annoyance, I share the same experience. Also the case with Tomcat. IDEs do so much hackery so that user experience is convenient, that you would be hard pressed to replicate the results, and often need to test both because it just doesn't work the same way, due to the different dependencies used, deployment method etc. Now that I think about it, every single one of the programming ecosystems I used were quite annoying.
  - __MatrixMan__ 5y ago
    I only spent two years in contact with Java. I was initially very pleased with how easy dependency management was but it soured over time. I'd respond to production disasters with hacky bash scripts and then we'd spend the next sprint incorporating that functionality into the server so the hacks weren't necessary. We kept catching ourselves working really hard to optimize what we thought at first needed to be custom code but what ended up being easily achievable by leaning on well-known non-java tools like `diff`. There's something about the way Javs encapsulates everything into a JAR that discourages Java devs from using anything on the outside. If you can live in a world where everything you need is in the JAR, life is good, but that hasn't frequently been my world.
    - tootie 5y ago
      I spent many years working with Java and never used bash for deployment. Maven covers all the bases. And when you have something really custom, you write a maven plugin. Which can also be managed with Maven.
      - __MatrixMan__ 5y ago
        Oh the bash scripts had nothing to do with deployment, they just fixed the data that the buggy deployment created. It's only relevant because the high level of encapsulation that made deps easy to manage also represented a barrier. Using diff and awk was fast enough, the reimplementation in java wasn't (happened just a few times). Language ecosystems with a more permeable encapsulation style make it easier to reference tools outside of that ecosystem at the cost of making it harder to get your deps straight.
  - robertlagrant 5y ago
    Bear in mind basically one guy maintains the Python package installer and default packages repo, or at least he did for many years. It's open source.
  - MBCook 5y ago
    I’m in the same spot. I’ve spent my career in Java and as I’ve started to enter the modern JS world through using React in projects… wow. I was amazed to find such a mess. Each project has full copies of all sources of all modules you use? They’re not shared in a global cache? There’s just hundreds of thousands of individual files? Yarn does a better job on some of this, but it all feels like stepping into an alternate reality where no one has ever heard of Maven. Maven predates Node by 5 years. Even if Node was developed without knowledge of Maven surely someone has brought this up in the last 13 years, right? It’s all very surprising to me, and I guess an interesting lesson in “parallel” evolution.
nicoburns 5y ago
It's less about package versions (which are indeed versioned on a per-project basis in sane ecosystems - python being a notable exception) and more about versions of the language toolchain itself. e.g. you might want one project on Java 17 and another on Java 8. These tools allow you to seamlessly switch between those.
- twblalock 5y ago
  Java tooling already provided that and it’s trivial to install multiple JDK versions on a system. I probably have 4. Most IDEs have a dropdown to pick the one you want, there are CLI tools for it too, and overall it’s about as complex as a $PATH variable. It’s also possible to install multiple versions of Python on a system. The dependency management is the real problem — having more than one version of a runtime is not the hard part.
  - nicoburns 5y ago
    If Java provides good first-party support for this then it may not make sense to use asdf for Java. Rust is similar in that rustup is first-party the recommended tool to install Rust with. I don't bother with asdf for Rust. Other ecosystems like python/ruby/javascript don't provide this as a first party tool meaning there are lot's of 3rd party alternatives. For these ecosystems asdf is nice in that it allows you to use one tool across all the ecosystems. > The dependency management is the real problem I'm not sure how this works in Java, but in many ecosystems like JavaScript and Rust dependency management isn't an issue at all. Installing a package defaults to a local install and even things like dependencies transitively depending on different versions of the same package are supported out the box and just work.
- smoe 5y ago
  What do you refer to by saying Python is an exception? I haven't used anything but different language and package versions on a per-project basis since around 2007. I had some project with deployments on client servers where their sysadmin insisted on having everything installed as os packages, but this at least from my perspective hasn't been the norm for quite a while now. Not to say that I like the Python package management story.
folkrav 5y ago
It's not a virtual environment tool, though. It's a tool to manage installed runtimes and switch between different versions in a shell. You'd still manage your virtual environments in whatever way your language prescribes.

tootie 5y ago

Java (well Maven) I believe does this the most clever and simple way. It does have a global folder of dependencies but will gladly stash multiple versions of the same library. Local projects just link to the right version. That saves downloads and allows for simple version pinning. And JVMs are backwards compatible so having a new JVM can still run bytecode for past versions with just a flag. Features like this are so much more valuable than anything that goes into the actual language syntax.

npteljes 5y ago
I come from the Java world and it's mindblowing to see that mature, widespread languages like Ruby, Python or JavaScript require all kinds of hackery to run properly. Dependency management with Maven is a breeze in comparison.
- Tainnor 5y ago
  What do you mean by Ruby "hackery"? I'll agree that how to do things properly with Ruby isn't obvious when you're new to the language - just as it isn't obvious how to do things in Java (or other JVM languages) when you're coming from Ruby or another language.[0] But Bundler doesn't work all that differently from Maven, in the end. Instead of running your commands through `./mwnw`, you run them through `bundle exec`[1]. Dependency management is straightforward and mostly hassle-free (the problems arise "only" when you use C extensions which, to be fair, is not that rare, but I don't really see a way around this in a language where you need C for certain fast things). In particular, Bundler by default[2] also stores dependencies centrally like maven to avoid duplication and then selects the appropriate version of a given dependency (unlike e.g. NPM which stores everything locally). Rbenv (or alternative tools such as chruby or asdf) solves a problem completely distinct from Bundler or Maven, it's about selecting the right Ruby version for your project, a problem that also needs to be solved with Java. IME, many developers just let the IDE deal with that, but Ruby developers are more used to running things from CLI, so that's why rbenv exists. In any case, the most popular analogue for Java would be sdkman which works, mostly, the same way (and which, IME, is even a bit hackier than rbenv). JS and Python are different. Node is a mess IMHO, and Python isn't able to agree on which package manager to use (I would hope people would migrate to Poetry, which seems to be doing things right). [0]: For example, one thing that I find very annoying over in Java-land is that it's a real hassle to make sure things behave the same way when called from the CLI as opposed to from the IDE, such as setting the active spring profile. This is because, mostly, the IDE circumvents maven/gradle (you can run things through the build system, but typically the output, e.g. test output, is better when using the IDE-internal system, at least with IntelliJ). [1]: There are "hackeries" to get rid of having to run everything through "bundle exec", such as binstubs and direnv, although I would mostly call them "shell script tricks", and anyway, if you instead just alias "bundle exec" to something shorter or create a wrapper script, it's not really worse than `./mwnw`. [2]: This can be overriden and is actually a good idea for production builds, especially if you use a multi-stage docker build since then it's easy to copy over all your dependencies.
  - tootie 5y ago
    I think you're missing that Maven does actually solve the rvm/nvm/pyenv problem. You can specify the bytecode version in the config and compile/run any backward compatible version. Your actual JDK need only be new enough, not an exact match and it doesn't need to change between projects. I have never had a single compiler or runtime version compatibility issue even with native dependencies ever with Java. The IDE situation can have some subtle edge cases, but no other language (save C#) actually gains so much utility from an IDE. Stack traces, break points, runtime inspection and profiling are all nearly flawless in IntelliJ (or Visual Studio) in ways that are monstrously difficult with any dynamic language.
    - Tainnor 5y ago
      Yes, but now you're not comparing maven vs. Bundler, you're comparing statically vs. dynamically typed languages. It's almost impossible to get the strong guarantees you get in a statically typed language in a dynamically typed one. If that's the discussion you want to have, well, I personally also prefer statically typed languages, but that's beyond the point. That said, I've had issues, e.g. on CI, that didn't manifest locally because of a mismatch in Java versions (something with JAXB, IIRC), and Java is maybe sometimes a bit too cavalier about the issue.
  - npteljes 5y ago
    By Ruby, Python and JavaScript hackery I was meaning rvm, venv / pip, and npm. I have the most experience with rvm, and the hackery part comes from the experience when I created a web interface that could instruct the server to do ruby tasks. Turned out that rvm (and also bundler) does a bunch of stuff to the environment of the user, that's sometimes inaccessible, even if you invoke a "login shell" programatically. One of these "hacks" is doing a bunch of stuff when the user navigates to a Ruby project directory with cd, which is when rvm creates or uses the existing virtual env for the project - named is a specific way based on the directory name. I solved this of course but the discovery was quite annoying. Regarding Python, I don't have much experience, other than the general feeling that it's fragile and that specific projects work with specific dependency managegers / virtual environments, and they are not really interchangeable or anything. I either could replicate the env or I couldn't. I definitely don't have much of a Python experience. Now regarding your Java IDE annoyance, I share the same experience. Also the case with Tomcat. IDEs do so much hackery so that user experience is convenient, that you would be hard pressed to replicate the results, and often need to test both because it just doesn't work the same way, due to the different dependencies used, deployment method etc. Now that I think about it, every single one of the programming ecosystems I used were quite annoying.
- __MatrixMan__ 5y ago
  I only spent two years in contact with Java. I was initially very pleased with how easy dependency management was but it soured over time. I'd respond to production disasters with hacky bash scripts and then we'd spend the next sprint incorporating that functionality into the server so the hacks weren't necessary. We kept catching ourselves working really hard to optimize what we thought at first needed to be custom code but what ended up being easily achievable by leaning on well-known non-java tools like `diff`. There's something about the way Javs encapsulates everything into a JAR that discourages Java devs from using anything on the outside. If you can live in a world where everything you need is in the JAR, life is good, but that hasn't frequently been my world.
  - tootie 5y ago
    I spent many years working with Java and never used bash for deployment. Maven covers all the bases. And when you have something really custom, you write a maven plugin. Which can also be managed with Maven.
    - __MatrixMan__ 5y ago
      Oh the bash scripts had nothing to do with deployment, they just fixed the data that the buggy deployment created. It's only relevant because the high level of encapsulation that made deps easy to manage also represented a barrier. Using diff and awk was fast enough, the reimplementation in java wasn't (happened just a few times). Language ecosystems with a more permeable encapsulation style make it easier to reference tools outside of that ecosystem at the cost of making it harder to get your deps straight.
- robertlagrant 5y ago
  Bear in mind basically one guy maintains the Python package installer and default packages repo, or at least he did for many years. It's open source.
- MBCook 5y ago
  I’m in the same spot. I’ve spent my career in Java and as I’ve started to enter the modern JS world through using React in projects… wow. I was amazed to find such a mess. Each project has full copies of all sources of all modules you use? They’re not shared in a global cache? There’s just hundreds of thousands of individual files? Yarn does a better job on some of this, but it all feels like stepping into an alternate reality where no one has ever heard of Maven. Maven predates Node by 5 years. Even if Node was developed without knowledge of Maven surely someone has brought this up in the last 13 years, right? It’s all very surprising to me, and I guess an interesting lesson in “parallel” evolution.

nicoburns 5y ago

It's less about package versions (which are indeed versioned on a per-project basis in sane ecosystems - python being a notable exception) and more about versions of the language toolchain itself. e.g. you might want one project on Java 17 and another on Java 8. These tools allow you to seamlessly switch between those.

twblalock 5y ago
Java tooling already provided that and it’s trivial to install multiple JDK versions on a system. I probably have 4. Most IDEs have a dropdown to pick the one you want, there are CLI tools for it too, and overall it’s about as complex as a $PATH variable. It’s also possible to install multiple versions of Python on a system. The dependency management is the real problem — having more than one version of a runtime is not the hard part.
- nicoburns 5y ago
  If Java provides good first-party support for this then it may not make sense to use asdf for Java. Rust is similar in that rustup is first-party the recommended tool to install Rust with. I don't bother with asdf for Rust. Other ecosystems like python/ruby/javascript don't provide this as a first party tool meaning there are lot's of 3rd party alternatives. For these ecosystems asdf is nice in that it allows you to use one tool across all the ecosystems. > The dependency management is the real problem I'm not sure how this works in Java, but in many ecosystems like JavaScript and Rust dependency management isn't an issue at all. Installing a package defaults to a local install and even things like dependencies transitively depending on different versions of the same package are supported out the box and just work.
smoe 5y ago
What do you refer to by saying Python is an exception? I haven't used anything but different language and package versions on a per-project basis since around 2007. I had some project with deployments on client servers where their sysadmin insisted on having everything installed as os packages, but this at least from my perspective hasn't been the norm for quite a while now. Not to say that I like the Python package management story.

folkrav 5y ago

It's not a virtual environment tool, though. It's a tool to manage installed runtimes and switch between different versions in a shell. You'd still manage your virtual environments in whatever way your language prescribes.