6 ms·

I'm working on a project that encompasses both JVM (Gradle, Kotlin) and Golang. My hot take: JVM build tools, especially Gradle, are a soup of unnecessary comp

I'm working on a project that encompasses both JVM (Gradle, Kotlin) and Golang. My hot take: JVM build tools, especially Gradle, are a soup of unnecessary complexity, and people working in that ecosystem have Stockholm Syndrome. In Golang, I spend about 99% of my time dealing with code. In JVM land, I'm spending 30% just dealing with the build system. It's actually insane, and the community at large thinks this is normal. The amount of time it takes to publish a multi-platform Kotlin library for the first time can be measured in days. I published my first Golang library in minutes, by comparison.

threeseed 2y ago
> the community at large thinks this is normal Half are ignorant. Other half are like me and just stuck with no options. But the tooling ecosystem on the JVM truly is horrific.
- iamcalledrob 2y ago
  I think there are a lot of "JVM Lifers" who are so deep in the ecosystem they are unaware how much better things can be. Anecdote: I wanted to publish a ~100LoC multiplatform Kotlin library -- just some bindings. I publish these sorts of things for Go with just a "git push". Steps were: 1. Spend a few hours trying to understand Maven Central/Sonotype, register and get "verified". They're in the middle of some kind of transition so everything is deprecated or unstable. 2. Figure out signing, because of course published packages must be signed. Now I have a secret to keep track of too, great. 3. Discover that there is no stable Gradle plugin for publishing to the "new" Maven Central, it's coming soon... Choose one of the handful of community plugins with a handful of stars on GitHub. 4. Spend a few hours troubleshooting a "Gradle build failed to end" error, which ended up being due to signing not finding a signing key. 3rd party plugin didn't handle errors properly, and a bug in Gradle meant that my secret wasn't picked up from local.properties. 4. Eventually discover that because Kotlin Multiplatform can't be cross-compiled, there is no way to actually publish a multiplatform library without spinning up a bunch of CI runners. And you can't just publish code -- JVM packages have to contain compiled artifacts. 5. Realise this now involves maintaining GitHub Actions and Gradle, which is an ongoing cost. 6. Give up. The harm that this kind of complexity must be causing to the ecosystem is immeasurable.
  - iainmerrick 2y ago
    I’d just like to add, NPM gets a lot of flak (mostly deservedly) but it too is still vastly easier than anything in the JVM ecosystem. Even with all the headaches around modules versus CJS, and JS versus TypeScript, NPM is a lot easier than Gradle. Notably, you have a choice of alternate tools (eg pnpm, yarn, bun) that interoperate pretty well. I guess my point is, Gradle and Maven are specifically and outstandingly bad.
    - lihaoyi 2y ago
      If you think gradle and maven are bad, you should try Mill! There is more to build tooling than gradle or maven, the field has evolved significantly since those tools launched 15-20 years ago, and Mill tries to do things better
    - ackfoobar 2y ago
      I must be missing something here. Don't the tools you mentioned do a lot less than Gradle? Gradle knows test depends on compile, which depends on code generation (say protobuf) - with caching and change detection. Compare that to chaining up the commands in the `scripts` section of `package.json`. EDIT: another comment making this point: https://news.ycombinator.com/item?id=41969847 https://news.ycombinator.com/item?id=41969847
      - iainmerrick 2y ago
        I could be convinced if those features of Gradle actually worked well, or even worked properly, like dependency management does in e.g. Bazel. In practice, Gradle really seems to fall down on the basic task of just being able to build stuff in the first place. It feels like you’re constantly fighting version hell just to find a Gradle version and plugins that work together, let alone your actual code dependencies. And if you actually do need to do something slightly more complicated, like code generation, it’s very difficult to work with and the docs are really bad.
        ackfoobar 2y ago
user1241320 2y ago
One note from having worked with both that I don’t see mentioned: Golang dependencies are sources you basically pull and compile with your own code. In JVM-land dependencies are precompliled packages (jars). This adds one little step.
- iamcalledrob 2y ago
  ...or a big step, if cross-compiling is required (e.g. Kotlin Multiplatform) I'm surprised there is no source-only dependency solution for JVM -- it'd solve this issue. Pull down the source and build on the fly. Perhaps there is and I'm unaware?
  - jitl 2y ago
    Well since the builds tend to be monstrously complicated for some reason, and there’s no standard build tool, maybe it’s more impossible than possible to consider source based distribution. Or it would be like JavaScript where you still need a build and publish step to turn “developer Java / other languages” into “vanilla source distributable Java”.
  - pkolaczk 2y ago
    I'm afraid Java/Scala/Kotlin compilers are too slow to make that convenient. Even currently building pure Java projects can take minutes when it's compiling just like 300k lines. What if it had to compile millions of lines from all the dependencies?
    - kaba0 2y ago
      The actual compilation step is 100% not the bottleneck - it can go as fast as 10k-50k lines per second! (According to the Mill benchmark, but that’s the Mill-independent part). Comparatively, Go does “only” 16k lines per second based on some HN comments.
      - pkolaczk 2y ago
        But you’re likely comparing on different hardware though. Go compiling only 16k lines per second is hard to believe for me. Maybe they meant on single CPU core. Rustc compiles over 50k lines per second on my MBP in debug mode and Go must be definitely faster, as everyone knows rust is very slow to compile. But anyway, you may be right. I just ran mvn install for the second time with no source change on my current project. It took 57 seconds.
cryptos 2y ago
You speak from my soul! I'm in the Java world for a really long time now and I'm wondering for years why the build tools need to be so complicated an annoying. I know Go, Node.js and bit of Rust and all have more pleasant easier to use build tools! The JVM (or GraalVM) as an ecosystem is just fine and probably one of the best, but build tools might be achille's heel. Maybe it would be a good idea for Oracle to invest into that area ...
- mike_hearn 2y ago
  My experience of JS projects is that build tools are frequently ad-hoc. That is, there simply isn't a general build tool at all, but just a large pile of scripts calling under-documented libraries. Parallelization, caching and quite often even portability are just missing. To justify this statement consider this blog post I wrote a while ago about porting GitHub Desktop (an Electron app) from its prior build/deployment system to Conveyor [1]. Conveyor is a tool for shipping desktop apps and is implemented as a single-purpose build system. The relevant part is this commit: https://github.com/hydraulic-software/github-desktop/commit/5247d87871cf0ddc1d51e486c77ccc9896831bec https://github.com/hydraulic-software/github-desktop/commit/... The amount of code that can be deleted is huge! Some of it is in-process code that isn't needed with Conveyor (setting up Squirrel etc), but a lot is just shell scripts that happen to be written in JS. Replacing that with a real build system not only simplifies the codebase but means the build steps are fully parallelized, fully incremental, easier to debug, portable (the build can run on any platform), progress is reported in a uniform way and so on. So whilst the JS ecosystem's approach to build tools may be "simple" in some way, in the sense that there's no dominant build tool like Maven or Gradle, that simplicity does cost you in other ways. [1] https://hydraulic.dev/blog/8-packaging-electron-apps.html https://hydraulic.dev/blog/8-packaging-electron-apps.html (Disclosure: Conveyor is a commercial product made by my company)
dikei 2y ago
> The amount of time it takes to publish a multi-platform Kotlin library for the first time can be measured in days. I published my first Golang library in minutes, by comparison. It's a bit Apple & Orange comparison: publishing a JVM only Kotlin library is quite easy, it's the multiplatform part that takes time.
- jitl 2y ago
  Last time I published a JVM library I had to Open A Jira Ticket to request the rights to publish a package on the main package registry. Then I had to verify I owned the DNS name prefix for my package by fiddling the DNS records at my hosting provider. It took days just to get authorized! Not including the time needed to like, figure out how to make JARs happen. In go: `git push` to a public repo In js: `npm publish` after making an NPM account
  - pkolaczk 2y ago
    In rust: `cargo publish` after making an account on crates.io
  - mdaniel 2y ago
    Merely as a "for your consideration," GitLab ships with its own Maven repository (along with npm, docker, Nuget, and a bazillion others)[1] so you have total sovereignty over the publishing auth story. I can appreciate going with Central can be a DX win if you're distributing a library, since having folks add <repository> lines to their pom.xml or settings.xml is a hassle, but at least you get to decide which hassle you prefer :-D In fairness, GitHub also finally got on board the train, too: https://docs.github.com/en/actions/use-cases-and-examples/publishing-packages/publishing-java-packages-with-maven#publishing-packages-to-github-packages https://docs.github.com/en/actions/use-cases-and-examples/pu... 1: https://docs.gitlab.com/ee/user/packages/maven_repository/ https://docs.gitlab.com/ee/user/packages/maven_repository/
  - vips7L 2y ago
    Sometimes barrier to entry is good. For example, both npm and cargo struggle with package name squatting and malicious packages that are miss spellings of common packages.
neeleshs 2y ago
Interesting. I spend nearly zero time with my maven setup and almost all the time is in coding. I am genuinely curious to know where that 30% time goes? Is it waiting for builds?
philipwhiuk 2y ago
> I published my first Golang library in minutes, by comparison. For what platform(s)? Or did you really just push the source code?
- pkolaczk 2y ago
  That's the trick. You publish the source code. And it's still faster to build all dependencies from source than maven / gradle manages to resolve and download the binary dependencies ;)
  - brabel 2y ago
    That's true, Maven is ridiculously slow to resolve dependencies while Gradle only really works with reasonable speed if you allow it to hog your system with a deamon. I myself wrote a dependency resolver that matches Maven in functionality, and even a large project that uses Spring Boot and its dozens of dependencies can be resolved in a couple of seconds. About 10x faster than Maven or something like that. If you look at Maven's source code you'll see why. It's the worst kind of Java Enterprise overengineering you can imagine, complete with its own dependency injection framework, everything is pluggable (for no reason, really, do you really need to replace HTTPS for your protocols?? In Plexus you can), to the point that all the de-coupling results in lots of things duplicating functionality everywhere. I am not sure but I would bet Maven parses your POM at least 10 times to do anything due to the de-coupled nature of it.
    - lihaoyi 2y ago
      Maven is actually pretty behind in terms of JVM dependency resolution. Mill uses Coursier, same as my last company did, and when my last company switched from Maven to Coursier we saw a 2 order of magnitude speedup, with resolution commands that used to take 30min finish in a few seconds to give the exact same artifacts and versions. I actually have no idea why these other resolvers are so slow, or why Coursier is so fast, but this slowness is very much a "maven" or "gradle" thing rather than a "jvm" thing. And Mill using coursier does significantly better!
pkolaczk 2y ago
I've been working on Java-based systems for about 20 years now, and I fully relate to that. Same experience. This is so annoying that I prefer to use Rust over Java even in areas where things like better performance or better type system don't matter. But being able to start a fresh project with one `cargo init` and a few `cargo add` invocations to add any dependencies... well, this is priceless.
- mdaniel 2y ago
  Are you aware of Maven Archetypes[1]? I believe they were the "cookiecutter" before cookiecutter existed, although I am 10000000% on-board that their discovery story is total garbage :-( 1: https://maven.apache.org/archetype/index.html https://maven.apache.org/archetype/index.html and https://maven.apache.org/archetype/maven-archetype-plugin/usage.html https://maven.apache.org/archetype/maven-archetype-plugin/us...
  - pkolaczk 2y ago
    But I don’t want to copy a full project with prepopulated list of dependencies chosen by someone else. I want to start small and add dependencies I need. It’s like LEGO vs Playmobil. I want LEGO. ;)
- brabel 2y ago
  Interesting that you ended up going all the way to Rust land instead of just using one of the multiple tools that have been created to help with this, like: * Spring Boot (it has a UI to create projects where you pick Java version, DB, build tool, some libs etc): https://spring.io/guides/gs/spring-boot https://spring.io/guides/gs/spring-boot * JHipster - the nuclear option, pick what you want a la carte: https://www.jhipster.tech/ https://www.jhipster.tech/ * JBang - a cute CLI for this: https://www.jbang.dev/ https://www.jbang.dev/ * Maven Archetypes - the old fashioned way (existed before "create-app" kind of tools appeared): https://maven.apache.org/guides/introduction/introduction-to-archetypes.html https://maven.apache.org/guides/introduction/introduction-to... And most IDEs also have "new project" wizzards.
- rileymichael 2y ago
imtringued 2y ago
I'm in JVM land. I spend very little time dealing with the build system. It is actually insane how well it works. Also, why does it matter how long it takes to publish a library for the first time? It sounds like a non-issue to me. I have written dozens of libraries and published them to a local artifactory instance because it simply doesn't matter if your company specific code is accessible to the world or not.

threeseed 2y ago

> the community at large thinks this is normal Half are ignorant. Other half are like me and just stuck with no options. But the tooling ecosystem on the JVM truly is horrific.

iamcalledrob 2y ago
I think there are a lot of "JVM Lifers" who are so deep in the ecosystem they are unaware how much better things can be. Anecdote: I wanted to publish a ~100LoC multiplatform Kotlin library -- just some bindings. I publish these sorts of things for Go with just a "git push". Steps were: 1. Spend a few hours trying to understand Maven Central/Sonotype, register and get "verified". They're in the middle of some kind of transition so everything is deprecated or unstable. 2. Figure out signing, because of course published packages must be signed. Now I have a secret to keep track of too, great. 3. Discover that there is no stable Gradle plugin for publishing to the "new" Maven Central, it's coming soon... Choose one of the handful of community plugins with a handful of stars on GitHub. 4. Spend a few hours troubleshooting a "Gradle build failed to end" error, which ended up being due to signing not finding a signing key. 3rd party plugin didn't handle errors properly, and a bug in Gradle meant that my secret wasn't picked up from local.properties. 4. Eventually discover that because Kotlin Multiplatform can't be cross-compiled, there is no way to actually publish a multiplatform library without spinning up a bunch of CI runners. And you can't just publish code -- JVM packages have to contain compiled artifacts. 5. Realise this now involves maintaining GitHub Actions and Gradle, which is an ongoing cost. 6. Give up. The harm that this kind of complexity must be causing to the ecosystem is immeasurable.
- iainmerrick 2y ago
  I’d just like to add, NPM gets a lot of flak (mostly deservedly) but it too is still vastly easier than anything in the JVM ecosystem. Even with all the headaches around modules versus CJS, and JS versus TypeScript, NPM is a lot easier than Gradle. Notably, you have a choice of alternate tools (eg pnpm, yarn, bun) that interoperate pretty well. I guess my point is, Gradle and Maven are specifically and outstandingly bad.
  - lihaoyi 2y ago
    If you think gradle and maven are bad, you should try Mill! There is more to build tooling than gradle or maven, the field has evolved significantly since those tools launched 15-20 years ago, and Mill tries to do things better
  - ackfoobar 2y ago
    I must be missing something here. Don't the tools you mentioned do a lot less than Gradle? Gradle knows test depends on compile, which depends on code generation (say protobuf) - with caching and change detection. Compare that to chaining up the commands in the `scripts` section of `package.json`. EDIT: another comment making this point: https://news.ycombinator.com/item?id=41969847 https://news.ycombinator.com/item?id=41969847
    - iainmerrick 2y ago
      I could be convinced if those features of Gradle actually worked well, or even worked properly, like dependency management does in e.g. Bazel. In practice, Gradle really seems to fall down on the basic task of just being able to build stuff in the first place. It feels like you’re constantly fighting version hell just to find a Gradle version and plugins that work together, let alone your actual code dependencies. And if you actually do need to do something slightly more complicated, like code generation, it’s very difficult to work with and the docs are really bad.
      - ackfoobar 2y ago

user1241320 2y ago

One note from having worked with both that I don’t see mentioned: Golang dependencies are sources you basically pull and compile with your own code. In JVM-land dependencies are precompliled packages (jars). This adds one little step.

iamcalledrob 2y ago
...or a big step, if cross-compiling is required (e.g. Kotlin Multiplatform) I'm surprised there is no source-only dependency solution for JVM -- it'd solve this issue. Pull down the source and build on the fly. Perhaps there is and I'm unaware?
- jitl 2y ago
  Well since the builds tend to be monstrously complicated for some reason, and there’s no standard build tool, maybe it’s more impossible than possible to consider source based distribution. Or it would be like JavaScript where you still need a build and publish step to turn “developer Java / other languages” into “vanilla source distributable Java”.
- pkolaczk 2y ago
  I'm afraid Java/Scala/Kotlin compilers are too slow to make that convenient. Even currently building pure Java projects can take minutes when it's compiling just like 300k lines. What if it had to compile millions of lines from all the dependencies?
  - kaba0 2y ago
    The actual compilation step is 100% not the bottleneck - it can go as fast as 10k-50k lines per second! (According to the Mill benchmark, but that’s the Mill-independent part). Comparatively, Go does “only” 16k lines per second based on some HN comments.
    - pkolaczk 2y ago
      But you’re likely comparing on different hardware though. Go compiling only 16k lines per second is hard to believe for me. Maybe they meant on single CPU core. Rustc compiles over 50k lines per second on my MBP in debug mode and Go must be definitely faster, as everyone knows rust is very slow to compile. But anyway, you may be right. I just ran mvn install for the second time with no source change on my current project. It took 57 seconds.

cryptos 2y ago

You speak from my soul! I'm in the Java world for a really long time now and I'm wondering for years why the build tools need to be so complicated an annoying. I know Go, Node.js and bit of Rust and all have more pleasant easier to use build tools! The JVM (or GraalVM) as an ecosystem is just fine and probably one of the best, but build tools might be achille's heel. Maybe it would be a good idea for Oracle to invest into that area ...

mike_hearn 2y ago
My experience of JS projects is that build tools are frequently ad-hoc. That is, there simply isn't a general build tool at all, but just a large pile of scripts calling under-documented libraries. Parallelization, caching and quite often even portability are just missing. To justify this statement consider this blog post I wrote a while ago about porting GitHub Desktop (an Electron app) from its prior build/deployment system to Conveyor [1]. Conveyor is a tool for shipping desktop apps and is implemented as a single-purpose build system. The relevant part is this commit: https://github.com/hydraulic-software/github-desktop/commit/5247d87871cf0ddc1d51e486c77ccc9896831bec https://github.com/hydraulic-software/github-desktop/commit/... The amount of code that can be deleted is huge! Some of it is in-process code that isn't needed with Conveyor (setting up Squirrel etc), but a lot is just shell scripts that happen to be written in JS. Replacing that with a real build system not only simplifies the codebase but means the build steps are fully parallelized, fully incremental, easier to debug, portable (the build can run on any platform), progress is reported in a uniform way and so on. So whilst the JS ecosystem's approach to build tools may be "simple" in some way, in the sense that there's no dominant build tool like Maven or Gradle, that simplicity does cost you in other ways. [1] https://hydraulic.dev/blog/8-packaging-electron-apps.html https://hydraulic.dev/blog/8-packaging-electron-apps.html (Disclosure: Conveyor is a commercial product made by my company)

dikei 2y ago

> The amount of time it takes to publish a multi-platform Kotlin library for the first time can be measured in days. I published my first Golang library in minutes, by comparison. It's a bit Apple & Orange comparison: publishing a JVM only Kotlin library is quite easy, it's the multiplatform part that takes time.

jitl 2y ago
Last time I published a JVM library I had to Open A Jira Ticket to request the rights to publish a package on the main package registry. Then I had to verify I owned the DNS name prefix for my package by fiddling the DNS records at my hosting provider. It took days just to get authorized! Not including the time needed to like, figure out how to make JARs happen. In go: `git push` to a public repo In js: `npm publish` after making an NPM account
- pkolaczk 2y ago
  In rust: `cargo publish` after making an account on crates.io
- mdaniel 2y ago
  Merely as a "for your consideration," GitLab ships with its own Maven repository (along with npm, docker, Nuget, and a bazillion others)[1] so you have total sovereignty over the publishing auth story. I can appreciate going with Central can be a DX win if you're distributing a library, since having folks add <repository> lines to their pom.xml or settings.xml is a hassle, but at least you get to decide which hassle you prefer :-D In fairness, GitHub also finally got on board the train, too: https://docs.github.com/en/actions/use-cases-and-examples/publishing-packages/publishing-java-packages-with-maven#publishing-packages-to-github-packages https://docs.github.com/en/actions/use-cases-and-examples/pu... 1: https://docs.gitlab.com/ee/user/packages/maven_repository/ https://docs.gitlab.com/ee/user/packages/maven_repository/
- vips7L 2y ago
  Sometimes barrier to entry is good. For example, both npm and cargo struggle with package name squatting and malicious packages that are miss spellings of common packages.

neeleshs 2y ago

Interesting. I spend nearly zero time with my maven setup and almost all the time is in coding. I am genuinely curious to know where that 30% time goes? Is it waiting for builds?

philipwhiuk 2y ago

> I published my first Golang library in minutes, by comparison. For what platform(s)? Or did you really just push the source code?

pkolaczk 2y ago
That's the trick. You publish the source code. And it's still faster to build all dependencies from source than maven / gradle manages to resolve and download the binary dependencies ;)
- brabel 2y ago
  That's true, Maven is ridiculously slow to resolve dependencies while Gradle only really works with reasonable speed if you allow it to hog your system with a deamon. I myself wrote a dependency resolver that matches Maven in functionality, and even a large project that uses Spring Boot and its dozens of dependencies can be resolved in a couple of seconds. About 10x faster than Maven or something like that. If you look at Maven's source code you'll see why. It's the worst kind of Java Enterprise overengineering you can imagine, complete with its own dependency injection framework, everything is pluggable (for no reason, really, do you really need to replace HTTPS for your protocols?? In Plexus you can), to the point that all the de-coupling results in lots of things duplicating functionality everywhere. I am not sure but I would bet Maven parses your POM at least 10 times to do anything due to the de-coupled nature of it.
  - lihaoyi 2y ago
    Maven is actually pretty behind in terms of JVM dependency resolution. Mill uses Coursier, same as my last company did, and when my last company switched from Maven to Coursier we saw a 2 order of magnitude speedup, with resolution commands that used to take 30min finish in a few seconds to give the exact same artifacts and versions. I actually have no idea why these other resolvers are so slow, or why Coursier is so fast, but this slowness is very much a "maven" or "gradle" thing rather than a "jvm" thing. And Mill using coursier does significantly better!

pkolaczk 2y ago

I've been working on Java-based systems for about 20 years now, and I fully relate to that. Same experience. This is so annoying that I prefer to use Rust over Java even in areas where things like better performance or better type system don't matter. But being able to start a fresh project with one `cargo init` and a few `cargo add` invocations to add any dependencies... well, this is priceless.

mdaniel 2y ago
Are you aware of Maven Archetypes[1]? I believe they were the "cookiecutter" before cookiecutter existed, although I am 10000000% on-board that their discovery story is total garbage :-( 1: https://maven.apache.org/archetype/index.html https://maven.apache.org/archetype/index.html and https://maven.apache.org/archetype/maven-archetype-plugin/usage.html https://maven.apache.org/archetype/maven-archetype-plugin/us...
- pkolaczk 2y ago
  But I don’t want to copy a full project with prepopulated list of dependencies chosen by someone else. I want to start small and add dependencies I need. It’s like LEGO vs Playmobil. I want LEGO. ;)
brabel 2y ago
Interesting that you ended up going all the way to Rust land instead of just using one of the multiple tools that have been created to help with this, like: * Spring Boot (it has a UI to create projects where you pick Java version, DB, build tool, some libs etc): https://spring.io/guides/gs/spring-boot https://spring.io/guides/gs/spring-boot * JHipster - the nuclear option, pick what you want a la carte: https://www.jhipster.tech/ https://www.jhipster.tech/ * JBang - a cute CLI for this: https://www.jbang.dev/ https://www.jbang.dev/ * Maven Archetypes - the old fashioned way (existed before "create-app" kind of tools appeared): https://maven.apache.org/guides/introduction/introduction-to-archetypes.html https://maven.apache.org/guides/introduction/introduction-to... And most IDEs also have "new project" wizzards.
rileymichael 2y ago

imtringued 2y ago

I'm in JVM land. I spend very little time dealing with the build system. It is actually insane how well it works. Also, why does it matter how long it takes to publish a library for the first time? It sounds like a non-issue to me. I have written dozens of libraries and published them to a local artifactory instance because it simply doesn't matter if your company specific code is accessible to the world or not.