4 ms·

Anyone have any experience using a Java treeshaker? I have a Clojure GUI/JFX App that's bloated and I've only managed to slim it down with manual package prunin

by geokon 5y ago

Anyone have any experience using a Java treeshaker? I have a Clojure GUI/JFX App that's bloated and I've only managed to slim it down with manual package pruning. I couldn't get Proguard to work with Clojure. Granted proguard is quite baroque so I probably did something wrong. But maybe there is some simpler solution out there for the JVM? Or would this simply be impossible b/c Clojure uses reflection? Though so does Javascript from what I understand..

setr 5y ago
Shouldn’t the JVM already be doing this, as part of dead code analysis during the compile step? I believe tree shaking in JavaScript also only handles static analysis — eg imports and function calls. I imagine though usage of something like exec throws DCE right out the window, poisoning the whole program
- geokon 5y ago
  No, the Java compiler does not do this. You can try it yourself and see. All dependencies are bundled into the JAR. If you have some large library as a dependency (like BoofCV or JavaFX for instance) then all of the classes and dependencies are bundled. Even if you don't use 99% of them. That's why a lot of Java libraries are broken up into many smaller sub-libraries which you can then manually pick and choose from. It's also why you sometimes see people re-implement smaller sections of larger libraries - b/c you don't want to be forced to drag in the whole mess of code As far as I understand, and I'm murky on the details, but if you're language supports reflection then you can call any dependency during runtime. And so this is not amenable to static analysis. So the Java compiler doesn't have the same guarantees as say a C++ compiler. Most people are running JVM on the server so they just don't care about executable size (and on Android people use proguard). But reflection in general is a source of headaches.. If you use something like GraalVM then it will try to prune dead code but it will not work well with reflections. That all being said, on the specifics, I'm actually not entirely sure how you'd use reflection to arbitrarily call library code at run time. If anyone knows, I'd be curious to see an example
  - Groxx 5y ago
    Java's reflection capabilities are pretty far down the "almost anything is possible" side of things, so yeah - unless you can rule out ALL reflection in the whole application, there's no way to safely shake a tree. Which is exactly why ProGuard and R8 have so many flags and options, and basically every use of it requires customization: https://developer.android.com/studio/build/shrink-code https://developer.android.com/studio/build/shrink-code
    - geokon 5y ago
      Crazy to think they choose an unshrinkable language as a mobile platform :) It doesn't seem like that powerful of a language feature (and more of a code smell) and it creates this mess down the line. I wonder what fraction of libraries even require it.. I'm guessing it's b/c of the flag soup i never got ProGuard working. I know the Clojure language/runtime itself relies on some reflection - unfortunately. That probably complicates things
      - Groxx 5y ago
        Java's is extremely powerful, especially when you include stuff like bytecode-weaving (e.g. AspectJ: https://www.lenar.io/logging-method-invocations-in-java-with-aspectj/ https://www.lenar.io/logging-method-invocations-in-java-with...). Which I mostly include because support for it effectively built into the language and all major IDEs, whether it's a run-time or compile-time construct (or a blend of both). And with a sufficiently-complex custom class loader, you can do darn near anything at runtime, as you have control over how the system loads new code. Android specifically though: an unbelievable number of libraries use reflection at least a little, if not deeply. E.g. the Android UI framework is absolutely riddled with it - ever used XML to inflate views, i.e. the default way to build a UI? That's all based on reflection. Many (many!) of the most-popular libraries use it as well, though the efficiency-minded ones generally use compile-time codegen (annotation processors) where at all possible.
        geokon 5y ago
        Yeah, I mean from a IDE/developer point of view reflection is fantastic - really can't argue with that. At dev time it's a life saver. But coming from C++ my gut reaction is that for compilation/distribution time this is mostly a crutch/code-smell with a huge price. I've for instance noticed there is seemingly a lot of library rewriteing and fragmentation. It's common for people to choose a smaller one-trick-pony library instead of a more mature large library - to avoid dragging in a enormous dependency. That ends up as a huge bummer for the ecosystem Thanks for the insight into Java internals and Android. It's a shame it's so tightly coupled at the roots but it's pure fantasy on my part to have a reflection-less JVM :) At the end of the day, there isn't really anything else like Clojure, so I just try to accept the warts and features. Out of curiosity I took a cursory look at Qt and they seem to handle QML without using RTTI - so think it's all solvable - at least in most cases (I'm guessing by enumerating your possible types at compile time?)
kaba0 5y ago
I recommend checking out the jdeps command line tool, that comes bundled with the JDK. Though you probably have to read up on modules a bit first. It can create very lean “JREs” that can be bundled with the program and only load the required classes.
- geokon 5y ago
  As I understood it, jdeps is only for shrinking the runtime and bundling it with the program. This part seems fine.. or at least they're doing the best they can :) I ended up skipping this step and distributing a JAR - which is a bit of no-no .. but i just ask user to install the Java runtime. (it's better than making executable for every platform and then spinning up VMs to test) However, jdeps or not, the rest of the executable still remains as bloated as before
  - kaba0 5y ago
    You are right in that it doesn’t do dead code elimination per se. But if my understanding is correct, java classes are loaded as needed. So other than JAR size, you don’t really need it as much as in the case of JS for example.
    - geokon 5y ago
      That's an important distinction. I guess my annoyance is mostly cosmetic. The app is pretty small and does a simple data processing task. Had I written the app in C++ it would have been a couple of megabytes (and it would have taken me 10 times longer to write..). But with Clojure/Java/JVM/jdeps it comes out as some monstrous 100MB installer. It just feels a bit gross ... End user apps are second class citizens on the JVM (except weirdly on Android.. but they're in a weird bubble of their own)
      - kaba0 5y ago
        Perhaps give native compilation a try? Not sure how well does it work with Clojure, but that might shrink the app size a bit as well?

by geokon 5y ago

setr 5y ago

Shouldn’t the JVM already be doing this, as part of dead code analysis during the compile step? I believe tree shaking in JavaScript also only handles static analysis — eg imports and function calls. I imagine though usage of something like exec throws DCE right out the window, poisoning the whole program

geokon 5y ago
No, the Java compiler does not do this. You can try it yourself and see. All dependencies are bundled into the JAR. If you have some large library as a dependency (like BoofCV or JavaFX for instance) then all of the classes and dependencies are bundled. Even if you don't use 99% of them. That's why a lot of Java libraries are broken up into many smaller sub-libraries which you can then manually pick and choose from. It's also why you sometimes see people re-implement smaller sections of larger libraries - b/c you don't want to be forced to drag in the whole mess of code As far as I understand, and I'm murky on the details, but if you're language supports reflection then you can call any dependency during runtime. And so this is not amenable to static analysis. So the Java compiler doesn't have the same guarantees as say a C++ compiler. Most people are running JVM on the server so they just don't care about executable size (and on Android people use proguard). But reflection in general is a source of headaches.. If you use something like GraalVM then it will try to prune dead code but it will not work well with reflections. That all being said, on the specifics, I'm actually not entirely sure how you'd use reflection to arbitrarily call library code at run time. If anyone knows, I'd be curious to see an example
- Groxx 5y ago
  Java's reflection capabilities are pretty far down the "almost anything is possible" side of things, so yeah - unless you can rule out ALL reflection in the whole application, there's no way to safely shake a tree. Which is exactly why ProGuard and R8 have so many flags and options, and basically every use of it requires customization: https://developer.android.com/studio/build/shrink-code https://developer.android.com/studio/build/shrink-code
  - geokon 5y ago
    Crazy to think they choose an unshrinkable language as a mobile platform :) It doesn't seem like that powerful of a language feature (and more of a code smell) and it creates this mess down the line. I wonder what fraction of libraries even require it.. I'm guessing it's b/c of the flag soup i never got ProGuard working. I know the Clojure language/runtime itself relies on some reflection - unfortunately. That probably complicates things
    - Groxx 5y ago
      Java's is extremely powerful, especially when you include stuff like bytecode-weaving (e.g. AspectJ: https://www.lenar.io/logging-method-invocations-in-java-with-aspectj/ https://www.lenar.io/logging-method-invocations-in-java-with...). Which I mostly include because support for it effectively built into the language and all major IDEs, whether it's a run-time or compile-time construct (or a blend of both). And with a sufficiently-complex custom class loader, you can do darn near anything at runtime, as you have control over how the system loads new code. Android specifically though: an unbelievable number of libraries use reflection at least a little, if not deeply. E.g. the Android UI framework is absolutely riddled with it - ever used XML to inflate views, i.e. the default way to build a UI? That's all based on reflection. Many (many!) of the most-popular libraries use it as well, though the efficiency-minded ones generally use compile-time codegen (annotation processors) where at all possible.
      - geokon 5y ago
        Yeah, I mean from a IDE/developer point of view reflection is fantastic - really can't argue with that. At dev time it's a life saver. But coming from C++ my gut reaction is that for compilation/distribution time this is mostly a crutch/code-smell with a huge price. I've for instance noticed there is seemingly a lot of library rewriteing and fragmentation. It's common for people to choose a smaller one-trick-pony library instead of a more mature large library - to avoid dragging in a enormous dependency. That ends up as a huge bummer for the ecosystem Thanks for the insight into Java internals and Android. It's a shame it's so tightly coupled at the roots but it's pure fantasy on my part to have a reflection-less JVM :) At the end of the day, there isn't really anything else like Clojure, so I just try to accept the warts and features. Out of curiosity I took a cursory look at Qt and they seem to handle QML without using RTTI - so think it's all solvable - at least in most cases (I'm guessing by enumerating your possible types at compile time?)

kaba0 5y ago

I recommend checking out the jdeps command line tool, that comes bundled with the JDK. Though you probably have to read up on modules a bit first. It can create very lean “JREs” that can be bundled with the program and only load the required classes.

geokon 5y ago
As I understood it, jdeps is only for shrinking the runtime and bundling it with the program. This part seems fine.. or at least they're doing the best they can :) I ended up skipping this step and distributing a JAR - which is a bit of no-no .. but i just ask user to install the Java runtime. (it's better than making executable for every platform and then spinning up VMs to test) However, jdeps or not, the rest of the executable still remains as bloated as before
- kaba0 5y ago
  You are right in that it doesn’t do dead code elimination per se. But if my understanding is correct, java classes are loaded as needed. So other than JAR size, you don’t really need it as much as in the case of JS for example.
  - geokon 5y ago
    That's an important distinction. I guess my annoyance is mostly cosmetic. The app is pretty small and does a simple data processing task. Had I written the app in C++ it would have been a couple of megabytes (and it would have taken me 10 times longer to write..). But with Clojure/Java/JVM/jdeps it comes out as some monstrous 100MB installer. It just feels a bit gross ... End user apps are second class citizens on the JVM (except weirdly on Android.. but they're in a weird bubble of their own)
    - kaba0 5y ago
      Perhaps give native compilation a try? Not sure how well does it work with Clojure, but that might shrink the app size a bit as well?