12 ms·

I write software in Java. I can't stand python - too wishy-washy. But my main complaint about Python is trying to use someone else's software. Then it degenera

I write software in Java. I can't stand python - too wishy-washy. But my main complaint about Python is trying to use someone else's software. Then it degenerates into a maze of twisty little passages of trying to install the correct set of dependencies, not knowing where those files are installed (which makes it difficult on a cluster, as you may need to do the install on every node separately). Then you try using pip and it fails because some element of what it is trying to install isn't compatible with your operating system for reason 1. So you try anaconda instead, and that fails in a different way, like it runs a C compiler on some file (!?) which fails because $REASON. I have numerous examples of Python software where I have followed the (detailed, precise) installation instructions, and it doesn't work. Whereas with Java, you just dump the necessary jars into a directory, include them in the classpath, and it works. Don't get me started on R.

BiteCode_dev 6y ago
I came to say say that's not a language problem, that's a packaging problem. But that would be misleading. The reason packaging is still a problem in Python are linked to the language: - to get perfs, you need compiled extensions, which java doesn't need. So a Jar can be fast and portable, while in Python it's an "or" proposition. Wheels help, but they still are OS dependent, so you need a build per OS. - PYZ arrived very late in Python, while WAR were a very early thing in Java. Few people even know you can distribute a whole Python program in a single zip file with dependencies using something like shiv (shiv.readthedocs.io). So few people do it, and it's a shame. We do it for 0bin though: https://github.com/Tygs/0bin/releases https://github.com/Tygs/0bin/releases. And it doesn't solve the wheel problem, although nuitka does but it a higher prices to pay. - packaging tooling have been a mess for 2 decades. Setuptools/distutils/setuptools2, easy_install/pip, and setup.cfg/setup.py/pyproject.toml. We failed as a community on this one. It's getting better, but it's a slow process. - the Python community is at least half composed of amateurs. It's a good thing for the dynamism of the community, but it also means the software are not very polished. Hard packaging + amateurs = packages that are releases when they reach "just good enough" stage, not one inch more. It also means you'll get things like leaking stack traces, config files in src and the likes. So yeah, compared to Java, Python packaging and distribution sucks. But compared to rust all packaging and distribution for all languages do, so there is that.
- mnw21cam 6y ago
  It certainly doesn't help that the software I'm interested in is written for science, so there is a tendency to make it work just well enough on the author's machine to allow a paper to be published, and then abandon it.
  - ta988 6y ago
    There is a clear training problem in science. But I believe part of it is that the knowledge on how to write good software is stuck in companies. You can't learn it in school either. I keep hearong about companies that have to make their new hires unlearn what they learned in class... So where would scientists learn good practices? (in my experience books arent the answer either)
    - mnw21cam 6y ago
      I don't think it is primarily a problem in training. It is a problem in incentive. For the most part, scientific software is written to fulfil a goal, and usually that goal is in order to publish a paper. Paper publication is what scientists are judged on, not the quality of the software they write. The software only needs to work well enough to get the paper published. Once the paper is published, all incentive to maintain or improve the software disappears, especially if the message of the paper is primarily the results obtained using the software, because then all of the citations will be referring to those results, not the software. There are two main exceptions to this rule. One is when an actual software engineer ends up working as a scientist and having to write papers (like me). Then, the paper is more likely to be about the software itself, and the citations of the paper are from people who have also used the same software. This creates an incentive to make the software high quality, in order to get those citations. The other exception is when an institute decides that the world needs a decent piece of software to fulfil a vital role, and actually funds its development. Oh, I should probably mention - I just helped out with a programming course for undergraduate medical science students. Most of the time I was tearing my hair out about the terrible programming practices being taught by the module leader, who admittedly had absolutely no computer science qualifications whatsoever (and in fact didn't know the difference between a forward slash and a backslash). I think a solution to this is to recognise software quality as a common benefit, and divert funding towards it, allowing real software engineers to have an actual meaningful career in university. Universities are usually all over common benefit stuff, so you would have thought this would be easy enough. The Research Software Engineer programme in the UK is a start.
MrPowers 6y ago
Python's packaging is a lot better with Poetry. Use Poetry and build the wheel with poetry build. The lockfile lets you create an exact virtual environment. Like you mentioned, wheel files are not nearly as customizable as JARs. You can easily pick & choose the dependencies to include in JARs (fat JARs) and change namespaces of dependencies via shading to prevent collisions. I've generally found it easier to add JARs to Spark clusters compared to attaching wheel files to PySpark clusters.
- dunefox 6y ago
  I'm all for a good project-based package manager for Python, but I have tried Poetry several times over the last years and not once has it worked as intended. It always failed to install packages for some reason.
the_gipsy 6y ago
Java is so much worse on this front. > with Java, you just dump the necessary jars into a directory, include them in the classpath, and it works. Ant, maven, or gradle, and then some jars, somewhere? What's the classpath and whats $JAVA_HOME for this project? Oh and now I have to make it work in the IDE too, click through a billion menus and dialogs, sync the build system, sync the dependencies, configure runtimes and classpaths, try to rebuild for the 10th time, pray.
- jsiepkes 6y ago
  Nobody has started a project with Ant in the last decade. Of all the Java projects on Github Ant is probably not even 1%. > Oh and now I have to make it work in the IDE too, click through a billion menus and dialogs, You click "Project from existing sources" in IntellIJ and it will recognize both Maven and Gradle projects and just work? Sync the dependencies? configure runtimes and classpaths? What? With Maven and Gradle? Your talking Ant here.
  - matwood 6y ago
    I only had to go a few comments to see someone (the person you're responding to) think Java is the same as it was 15 years ago.
  - the_gipsy 6y ago
    I'm talking about a maven project I have to use sometimes at work. Let me walk you through all the shit I had to do: - Install and set the right java version system-wide - Copy some config files for the project system-wide (granted, this is not java specific, but still) - Click in some IDE menu to enable some build settings - Click in some IDE menu to install some "Lombok" plugin - Click in some IDE menu to select the right jdk - Click in some IDE menu to edit the run configuration: - select main class - paste program args - paste env vars - select right “Use classpath of module:” option - Click in some IDE menu to fiddle with "Spring Loader path": modules, dependencies, reorder the dependencies (!) - Click in some IDE menu to configure getting the actual logs - Click maven-sync, rebuild-project multiple times, restart IDE until it suddenly works The first guy who got it to work had to write a guide with screenshots! I know you will say, this is all fault of whoever made this crap project, or my fault for not fixing it. However this was my experience, and it has been similar every time in my life that I had to touch some java project (although this last one is the worst).
jitix 6y ago
Most python software use the standard venev setup with pip and requirements.txt, and most java software are distributed as fat jars. Nobody manually sets up dependencies anymore. As a Java/Python/Go developer I can say that packaging is a solved problem as of 2021.
- JAlexoid 6y ago
  That's really not true. I like to point out Airflow as an example. It's still a complete nightmare to deal with and it has many steps for just packaging the application. https://github.com/apache/airflow https://github.com/apache/airflow I have less trouble packaging and running a 3 year unmaintained Maven Java application, than actively maintained Airflow. And let's not forget that I can target Java 8, without having to install Java 8 on my machine. (Which only adds to Python's complexity)
ppod 6y ago
What do you mean about R? Its packaging and distribution system is great!
michaelcampbell 6y ago
> I can't stand python - too wishy-washy. That's an interesting take; my experience has been (at least the community) is quite proudly dogmatic on the One Right Way To Do It, for all values of "It". I also don't like Python, but I think for different reasons and perhaps a bit less than you. Never used R, but I understand it to be quite fiddly.
- dunefox 6y ago
  > my experience has been (at least the community) is quite proudly dogmatic on the One Right Way To Do It, for all values of "It". This hasn't been true lately. Python is growing ever more complex.
  - michaelcampbell 6y ago
    I hear you and agree to an extent; my suspicion is the influx of non-CS self-styled "data scientists" using Python for all the ML hype has diluted the historical rabidness. But, for people still working in "computer science-y" domains, I haven't seen the historical dogmatism lessen; in many cases people are digging in.
- 131012 6y ago
  R is an analysis tool. You don't write software in R.
  - timita 6y ago
    Well, maybe _you_ don't write software in R. Others can write a pretty capable server[0] using just that. [0] https://github.com/opencpu/opencpu/tree/master/R https://github.com/opencpu/opencpu/tree/master/R
    - for_xyz 6y ago
      Of course you can write anything in R but that doesn't mean it will be possible to maintain the codebase for the next 10 maybe 20 years including dev team changes. In other words R is wrong tool to use for building the software just from maintainability standpoint.
MichaelRazum 6y ago
You are kind of right. Install non standard stuff in python is sometimes pain. BUT you can do it absolutely right. Also most of the time only few packages make you trouble. So you just need sometimes a fix for them. That's it. You even can download it and make it completely offline. Instagram is running on python... so it is battle tested. By the way, I would always try to avoid anaconda in production.
theflyinghorse 6y ago
> main complaint about Python is trying to use someone else's software. Then it degenerates into a maze of twisty little passages of trying to install the correct set of dependencies, not knowing where those files are installed Java is far worse on this front. Let's take a look at JEE or Spring - The entire experience is akin to summoning a demon with waving reciting correct incantations in correct places while having no idea about what even goes on in the background. There are MessageChannels, MessageProducers, ClientFactories that interact in mysterious ways and initiated in also very mysterious ways. And then there's EJB's and their containers which are just blackboxes of automagic. Or how about Hibernate?
- kaba0 6y ago
  Well, it does considerably more things than a hello world python app. So at least compare it to Python’s full blown backend server with ORM and the like.
  - BiteCode_dev 6y ago
    Django: zero dependencies for years, you could literally copy paste the code and it worked, ORM included. It now has 2 dependencies. It is a 1000 times easier to use than hibernate, despite being one of the worse ORM in Python. That the problem with old Java libs like Swing or Hibernate: they have been designed for the 90' and don't leverage the potential of what a modern Java version can do.
    - lenkite 6y ago
      You can use lean Java persistence frameworks with lesser dependencies if you wish. I think mybatis only has 4/5 dependencies and 2 of them are the the standard slf4/logback dependencies. There are many modern java libraries that are very lean and mean out there and package into small-sized redistributable assemblies.
    - JAlexoid 6y ago
      Pointing to Swing or Hibernate is akin to pointing to GoLang of 2012 or Python of 2001.
TomSwirly 6y ago
I have to say that I use Python a great deal and I just never run into these issues. Nor have I ever even used Anaconda, I just use virtualenvs and pip and it all works. It's my guess that you are using a lot of Python libraries that are wrappers around C or C++ libraries which have dodgy cross-platform support, because I can't see another way this is happening. If so, it's hardly fair to blame Python for that...

BiteCode_dev 6y ago

I came to say say that's not a language problem, that's a packaging problem. But that would be misleading. The reason packaging is still a problem in Python are linked to the language: - to get perfs, you need compiled extensions, which java doesn't need. So a Jar can be fast and portable, while in Python it's an "or" proposition. Wheels help, but they still are OS dependent, so you need a build per OS. - PYZ arrived very late in Python, while WAR were a very early thing in Java. Few people even know you can distribute a whole Python program in a single zip file with dependencies using something like shiv (shiv.readthedocs.io). So few people do it, and it's a shame. We do it for 0bin though: https://github.com/Tygs/0bin/releases https://github.com/Tygs/0bin/releases. And it doesn't solve the wheel problem, although nuitka does but it a higher prices to pay. - packaging tooling have been a mess for 2 decades. Setuptools/distutils/setuptools2, easy_install/pip, and setup.cfg/setup.py/pyproject.toml. We failed as a community on this one. It's getting better, but it's a slow process. - the Python community is at least half composed of amateurs. It's a good thing for the dynamism of the community, but it also means the software are not very polished. Hard packaging + amateurs = packages that are releases when they reach "just good enough" stage, not one inch more. It also means you'll get things like leaking stack traces, config files in src and the likes. So yeah, compared to Java, Python packaging and distribution sucks. But compared to rust all packaging and distribution for all languages do, so there is that.

mnw21cam 6y ago
It certainly doesn't help that the software I'm interested in is written for science, so there is a tendency to make it work just well enough on the author's machine to allow a paper to be published, and then abandon it.
- ta988 6y ago
  There is a clear training problem in science. But I believe part of it is that the knowledge on how to write good software is stuck in companies. You can't learn it in school either. I keep hearong about companies that have to make their new hires unlearn what they learned in class... So where would scientists learn good practices? (in my experience books arent the answer either)
  - mnw21cam 6y ago
    I don't think it is primarily a problem in training. It is a problem in incentive. For the most part, scientific software is written to fulfil a goal, and usually that goal is in order to publish a paper. Paper publication is what scientists are judged on, not the quality of the software they write. The software only needs to work well enough to get the paper published. Once the paper is published, all incentive to maintain or improve the software disappears, especially if the message of the paper is primarily the results obtained using the software, because then all of the citations will be referring to those results, not the software. There are two main exceptions to this rule. One is when an actual software engineer ends up working as a scientist and having to write papers (like me). Then, the paper is more likely to be about the software itself, and the citations of the paper are from people who have also used the same software. This creates an incentive to make the software high quality, in order to get those citations. The other exception is when an institute decides that the world needs a decent piece of software to fulfil a vital role, and actually funds its development. Oh, I should probably mention - I just helped out with a programming course for undergraduate medical science students. Most of the time I was tearing my hair out about the terrible programming practices being taught by the module leader, who admittedly had absolutely no computer science qualifications whatsoever (and in fact didn't know the difference between a forward slash and a backslash). I think a solution to this is to recognise software quality as a common benefit, and divert funding towards it, allowing real software engineers to have an actual meaningful career in university. Universities are usually all over common benefit stuff, so you would have thought this would be easy enough. The Research Software Engineer programme in the UK is a start.

MrPowers 6y ago

Python's packaging is a lot better with Poetry. Use Poetry and build the wheel with poetry build. The lockfile lets you create an exact virtual environment. Like you mentioned, wheel files are not nearly as customizable as JARs. You can easily pick & choose the dependencies to include in JARs (fat JARs) and change namespaces of dependencies via shading to prevent collisions. I've generally found it easier to add JARs to Spark clusters compared to attaching wheel files to PySpark clusters.

dunefox 6y ago
I'm all for a good project-based package manager for Python, but I have tried Poetry several times over the last years and not once has it worked as intended. It always failed to install packages for some reason.

the_gipsy 6y ago

Java is so much worse on this front. > with Java, you just dump the necessary jars into a directory, include them in the classpath, and it works. Ant, maven, or gradle, and then some jars, somewhere? What's the classpath and whats $JAVA_HOME for this project? Oh and now I have to make it work in the IDE too, click through a billion menus and dialogs, sync the build system, sync the dependencies, configure runtimes and classpaths, try to rebuild for the 10th time, pray.

jsiepkes 6y ago
Nobody has started a project with Ant in the last decade. Of all the Java projects on Github Ant is probably not even 1%. > Oh and now I have to make it work in the IDE too, click through a billion menus and dialogs, You click "Project from existing sources" in IntellIJ and it will recognize both Maven and Gradle projects and just work? Sync the dependencies? configure runtimes and classpaths? What? With Maven and Gradle? Your talking Ant here.
- matwood 6y ago
  I only had to go a few comments to see someone (the person you're responding to) think Java is the same as it was 15 years ago.
- the_gipsy 6y ago
  I'm talking about a maven project I have to use sometimes at work. Let me walk you through all the shit I had to do: - Install and set the right java version system-wide - Copy some config files for the project system-wide (granted, this is not java specific, but still) - Click in some IDE menu to enable some build settings - Click in some IDE menu to install some "Lombok" plugin - Click in some IDE menu to select the right jdk - Click in some IDE menu to edit the run configuration: - select main class - paste program args - paste env vars - select right “Use classpath of module:” option - Click in some IDE menu to fiddle with "Spring Loader path": modules, dependencies, reorder the dependencies (!) - Click in some IDE menu to configure getting the actual logs - Click maven-sync, rebuild-project multiple times, restart IDE until it suddenly works The first guy who got it to work had to write a guide with screenshots! I know you will say, this is all fault of whoever made this crap project, or my fault for not fixing it. However this was my experience, and it has been similar every time in my life that I had to touch some java project (although this last one is the worst).

jitix 6y ago

Most python software use the standard venev setup with pip and requirements.txt, and most java software are distributed as fat jars. Nobody manually sets up dependencies anymore. As a Java/Python/Go developer I can say that packaging is a solved problem as of 2021.

JAlexoid 6y ago
That's really not true. I like to point out Airflow as an example. It's still a complete nightmare to deal with and it has many steps for just packaging the application. https://github.com/apache/airflow https://github.com/apache/airflow I have less trouble packaging and running a 3 year unmaintained Maven Java application, than actively maintained Airflow. And let's not forget that I can target Java 8, without having to install Java 8 on my machine. (Which only adds to Python's complexity)

ppod 6y ago

What do you mean about R? Its packaging and distribution system is great!

michaelcampbell 6y ago

> I can't stand python - too wishy-washy. That's an interesting take; my experience has been (at least the community) is quite proudly dogmatic on the One Right Way To Do It, for all values of "It". I also don't like Python, but I think for different reasons and perhaps a bit less than you. Never used R, but I understand it to be quite fiddly.

dunefox 6y ago
> my experience has been (at least the community) is quite proudly dogmatic on the One Right Way To Do It, for all values of "It". This hasn't been true lately. Python is growing ever more complex.
- michaelcampbell 6y ago
  I hear you and agree to an extent; my suspicion is the influx of non-CS self-styled "data scientists" using Python for all the ML hype has diluted the historical rabidness. But, for people still working in "computer science-y" domains, I haven't seen the historical dogmatism lessen; in many cases people are digging in.
131012 6y ago
R is an analysis tool. You don't write software in R.
- timita 6y ago
  Well, maybe _you_ don't write software in R. Others can write a pretty capable server[0] using just that. [0] https://github.com/opencpu/opencpu/tree/master/R https://github.com/opencpu/opencpu/tree/master/R
  - for_xyz 6y ago
    Of course you can write anything in R but that doesn't mean it will be possible to maintain the codebase for the next 10 maybe 20 years including dev team changes. In other words R is wrong tool to use for building the software just from maintainability standpoint.

MichaelRazum 6y ago

You are kind of right. Install non standard stuff in python is sometimes pain. BUT you can do it absolutely right. Also most of the time only few packages make you trouble. So you just need sometimes a fix for them. That's it. You even can download it and make it completely offline. Instagram is running on python... so it is battle tested. By the way, I would always try to avoid anaconda in production.

theflyinghorse 6y ago

> main complaint about Python is trying to use someone else's software. Then it degenerates into a maze of twisty little passages of trying to install the correct set of dependencies, not knowing where those files are installed Java is far worse on this front. Let's take a look at JEE or Spring - The entire experience is akin to summoning a demon with waving reciting correct incantations in correct places while having no idea about what even goes on in the background. There are MessageChannels, MessageProducers, ClientFactories that interact in mysterious ways and initiated in also very mysterious ways. And then there's EJB's and their containers which are just blackboxes of automagic. Or how about Hibernate?

kaba0 6y ago
Well, it does considerably more things than a hello world python app. So at least compare it to Python’s full blown backend server with ORM and the like.
- BiteCode_dev 6y ago
  Django: zero dependencies for years, you could literally copy paste the code and it worked, ORM included. It now has 2 dependencies. It is a 1000 times easier to use than hibernate, despite being one of the worse ORM in Python. That the problem with old Java libs like Swing or Hibernate: they have been designed for the 90' and don't leverage the potential of what a modern Java version can do.
  - lenkite 6y ago
    You can use lean Java persistence frameworks with lesser dependencies if you wish. I think mybatis only has 4/5 dependencies and 2 of them are the the standard slf4/logback dependencies. There are many modern java libraries that are very lean and mean out there and package into small-sized redistributable assemblies.
  - JAlexoid 6y ago
    Pointing to Swing or Hibernate is akin to pointing to GoLang of 2012 or Python of 2001.

TomSwirly 6y ago

I have to say that I use Python a great deal and I just never run into these issues. Nor have I ever even used Anaconda, I just use virtualenvs and pip and it all works. It's my guess that you are using a lot of Python libraries that are wrappers around C or C++ libraries which have dodgy cross-platform support, because I can't see another way this is happening. If so, it's hardly fair to blame Python for that...