3 ms·

Say what? Every JVM program I have seen has an attached shell script to set up dependency path. Here is a first example: https://github.com/apache/spark/blob/1

by theamk 7y ago

Say what? Every JVM program I have seen has an attached shell script to set up dependency path. Here is a first example: https://github.com/apache/spark/blob/121f9338cefbb1c800fabfea5152899a58176b00/bin/spark-class https://github.com/apache/spark/blob/121f9338cefbb1c800fabfe... At least half of that is figuring out what the classpath should be. And the worst part, it is all exposed to user, too -- when I was setting up Spark, I had to muck with class paths a lot. So JVM has 3 big problems: (1) No single way to set up dependency path, each project is different; (2) No way to set up classpath in JVM language, has to use some other one (like bash); (3) Somehow, user has to fix it for many semin-advanced programs. No other ecosystem I know of is this insane! Python, node.js, even C++ are all better!

EdwardDiego 7y ago
Okay, so you've changed the context from "I'm a developer" to "I'm an end user" - how many Python end users are having to bugger about with virtualenvs? Okay, so, speaking for myself, we generally ship fat jars or capsules these Docker days. I can't speak as to why Apache Spark chose to do it that way, I don't run their project. But if I'm guessing I suspect it's so you can provide your own Hadoop dependencies as needed, as there's a bunch of varying distributions that people use (Cloudera, Hortonworks etc.) and they're trying to play nice with the existing Hadoop ecosystem - probably also to make it possible to extend it with any other jars that you want to use (although spark-shell has a nice feature where you provide additional dependencies on the command line, or within the shell, using Maven artifact coordinates) For an example of what's involved using an uberjar https://docs.cyclopsgroup.org/jmxterm https://docs.cyclopsgroup.org/jmxterm > java -jar jmxterm-1.0.0-uber.jar That's it.
- m45t3r 7y ago
  No, OP case is for developers setting up their development machines. I feel his pain since I work with Clojure, that 99% of time it simply works however in the 1% of time that it breaks its is painful to fix. BTW, the uberjar equivalent in Python is something like PEX/shiv/zipapps.
- theamk 7y ago
  And yet, even jmxterm has a shell wrapper script: https://github.com/jiaqi/jmxterm/blob/master/src/main/script/jmxterm.sh https://github.com/jiaqi/jmxterm/blob/master/src/main/script... looks like a custom one.. and non-relocatable, too -- if I wanted to install "jmxterm" to my home dir, I'd need to manually edit the script. (and yes, this is totally "I am developer" issue, not just "I am a user" one -- the dev had to write the script, they had to debug it, and dev would be handling bug reports about inability to invoke from command line when installed to other location)
  - EdwardDiego 7y ago
    > looks like a custom one.. and non-relocatable, too -- if I wanted to install "jmxterm" to my home dir, I'd need to manually edit the script Funny how you had to go searching to find that script, when the installation instructions are simply - 1) download uberjar, 2) run uberjar. Also, you had to go searching to find a script 8 years old, in a project with a commit 20 days ago, to attempt to prove your point. Surely you must realise you're grasping at this stage.

by theamk 7y ago

EdwardDiego 7y ago

Okay, so you've changed the context from "I'm a developer" to "I'm an end user" - how many Python end users are having to bugger about with virtualenvs? Okay, so, speaking for myself, we generally ship fat jars or capsules these Docker days. I can't speak as to why Apache Spark chose to do it that way, I don't run their project. But if I'm guessing I suspect it's so you can provide your own Hadoop dependencies as needed, as there's a bunch of varying distributions that people use (Cloudera, Hortonworks etc.) and they're trying to play nice with the existing Hadoop ecosystem - probably also to make it possible to extend it with any other jars that you want to use (although spark-shell has a nice feature where you provide additional dependencies on the command line, or within the shell, using Maven artifact coordinates) For an example of what's involved using an uberjar https://docs.cyclopsgroup.org/jmxterm https://docs.cyclopsgroup.org/jmxterm > java -jar jmxterm-1.0.0-uber.jar That's it.

m45t3r 7y ago
No, OP case is for developers setting up their development machines. I feel his pain since I work with Clojure, that 99% of time it simply works however in the 1% of time that it breaks its is painful to fix. BTW, the uberjar equivalent in Python is something like PEX/shiv/zipapps.
theamk 7y ago
And yet, even jmxterm has a shell wrapper script: https://github.com/jiaqi/jmxterm/blob/master/src/main/script/jmxterm.sh https://github.com/jiaqi/jmxterm/blob/master/src/main/script... looks like a custom one.. and non-relocatable, too -- if I wanted to install "jmxterm" to my home dir, I'd need to manually edit the script. (and yes, this is totally "I am developer" issue, not just "I am a user" one -- the dev had to write the script, they had to debug it, and dev would be handling bug reports about inability to invoke from command line when installed to other location)
- EdwardDiego 7y ago
  > looks like a custom one.. and non-relocatable, too -- if I wanted to install "jmxterm" to my home dir, I'd need to manually edit the script Funny how you had to go searching to find that script, when the installation instructions are simply - 1) download uberjar, 2) run uberjar. Also, you had to go searching to find a script 8 years old, in a project with a commit 20 days ago, to attempt to prove your point. Surely you must realise you're grasping at this stage.