6 ms·

SD4J – Stable Diffusion pipeline in Java using ONNX Runtime

77 points by simonpure 3y ago

vinni2 3y ago
[flagged]
- tormeh 3y ago
  I dislike Java, but some poor coders are writing code in MUMPS for their poor customers. Java is not a particularly good language, but it's not the worst thing out there.
- ltbarcly3 3y ago
  There are a lot of relatively inexpensive, relatively fungible engineers, especially offshore. (not really but this is what idiot middle managers and VPs will tell you). Lots if not most of the biggest and most profitable businesses in the world are built primarily using Java. (this is true).
- asylteltine 3y ago
  I agree. Every single time I a new tool that’s written in Java, I sigh. Just… why? I understand old legacy enterprise stuff, but new things??
  - gbear605 3y ago
    I’ve tried lots of different languages, and for building actual large applications I just prefer Java. The language has changed a lot in the last ten years to be more agile and enjoyable to use.
- mschuster91 3y ago
  Assuming you do get cross-compilation sorted out once and get your libraries compiled to JNI JARs, it's easier to get running than the dependency hell associated with shipping C++ stuff around.
- ramijames 3y ago
  "Just because it isn't useful for me specifically, then nobody else should use it."
- smusamashah 3y ago
  As a Java dev now, this comment does not make sense. Doing Java for 4 years now and I only appreciate how much it can do (working on MMO which is written in Java both backend and fronted).
- 62951413 3y ago
  It's a real mystery. Some popular conspiracy theories: * Scala projects the Haskell-on-the-JVM image too much to scare off the practically-minded. It has no ORCL or GOOG behind it (its cousin F# has had official MSFT support for a decade at least). * Kotlin is strongly associated with the frontend. Just try to search for a backend Kotlin role in the Bay Area to see. * Java 17-21 is catching up with Kotlin fast enough to make many people doubt the upskilling effort.
  - pjmlp 3y ago
    Kotlin only matters on Android, because Google says so, and even them had to accept Kotlin without Maven ecosystem is meaningless, increasingly moving into modern Java, thus Android updates to Java 11, followed by Java 17.
- pjmlp 3y ago
  Because Java, alongside .NET, are the heirs of Xerox PARC tooling and development school.
rvz 3y ago
[flagged]
- lbalazscs 3y ago
  The ONNX Runtime is written in C++ and it uses the GPU. And Java is certainly much faster than Python.
  - rvz 3y ago
    That means nothing if one uses an inefficient language for both inference and training and the slower language will still bottleneck the ONNX runtime FFI calls and on top of Java's own garbage collector. For those that care about performance know that the worst case is taken into account regardless if it is written in C++ for both memory and runtime performance.
- pjmlp 3y ago
  At least Java cares about JIT and AOT compilers, unlike Python that needs to ping back into C, C++ or Fortran for anything beyond UNIX scripting.
ShamelessC 3y ago
[flagged]
- refulgentis 3y ago
  ?
  - ShamelessC 3y ago
    Sarcasm. It would be great satire, if it actually were.
    - refulgentis 3y ago
      Of what?
deleted 3y ago
[deleted]
bmc7505 3y ago
Old enough to remember the days when Java reached near AI-level hype. Most of the advanced placement and undergraduate CS courses in the U.S. were taught in Java. There was an annual conference in San Fransisco called JavaOne/Oracle OpenWorld that rivaled the size of even NeurIPS, drawing tens of thousands of attendees each year. According to some statistics, there were 10 million Java developers and 3 billion devices running Java worldwide. I wonder what happened? Why does HN dislike Java so much? edit: Someone should really take the time to highlight all the innovative machine learning research happening in Java. The original implementation of t-SNE [1] was written in Java. Most NLP researchers have heard of CoreNLP [2], also Java. One of the earliest ML libraries, Weka [3], was written in Java and is still actively developed at the University of Waikato. Noteworthy research on ML4Code specifically targets the Java language [4]. There's a bunch of published sketching algorithms for Java (e.g., DataSketches [5], t-digests, ddsketch et al.), featured in an invited talk [6] to NeurIPS this year. Just to name a few off the top of my head. [1]: https://github.com/lejon/T-SNE-Java https://github.com/lejon/T-SNE-Java [2]: https://github.com/stanfordnlp/CoreNLP https://github.com/stanfordnlp/CoreNLP [3]: https://www.cs.waikato.ac.nz/ml/weka/ https://www.cs.waikato.ac.nz/ml/weka/ [4]: https://openreview.net/pdf?id=bUDmRzeh3PT https://openreview.net/pdf?id=bUDmRzeh3PT [5]: https://github.com/apache/datasketches-java https://github.com/apache/datasketches-java [6]: https://nips.cc/virtual/2023/events/Invited%20Talk https://nips.cc/virtual/2023/events/Invited%20Talk
- ivan_gammel 3y ago
  HN doesn't really dislike it, looking at the past threads. There are usually a few people quickly jumping to comment negatively, but in most cases they do not have enough experience with it or have not touched it since Java 8 or earlier version. Also, it is not a small thing to learn and it seems to be unpopular among the bootcamps - I have seen a lot of Python course ads, but hardly a single one on Java.
  - deleted 3y ago
    [deleted]
- 62951413 3y ago
  There was a _long_ hiatus in Java progress between 1.5 (release in September 2004) and 1.8 (released in March 2014). Before Java 8 the language was, basically, the previous generation's golang. in that timeframe there was no Kotlin, Scala had little toolchain support and some sharp edges. So many smart people left the JVM altogether.
genman 3y ago
[flagged]
- craigacp 3y ago
  Both GPU support and SIMD CPU support are being worked on in OpenJDK projects, there are talks on them from last year's JVM language summit - https://youtube.com/playlist?list=PLX8CzqL3ArzW90jKUCf4H6xCKpStxsOzp&si=T8-cgm7Njn0y9T41 https://youtube.com/playlist?list=PLX8CzqL3ArzW90jKUCf4H6xCK...
  - genman 3y ago
    Thank you. It looks like there is still a lot of work to do.
- ShamelessC 3y ago
  [flagged]
  - MaxBarraclough 3y ago
    > We were forced to use Java and then switched languages and saw the light. Other languages will do a better job in various ways, certainly, but in my opinion Java remains a pretty good middle-of-the-road language. If you want to go deep on high-performance computing, or functional programming, or type-safety, or low-level programming, then Java isn't going to win at any one of these. It's still broadly successful in what it's trying to achieve: a fairly fast, very portable language and virtual machine with various features to help the programmer deal with code-defects and achieve correctness, generally preferencing these (and portability) over performance where necessary. It also generally commits to backward compatibility. There is no one language to rule them all. It's more a matter of Pareto efficiency.
  - severino 3y ago
    Who forced you to use Java and why?
  - belter 3y ago
gfodor 3y ago
If you liked this you might like SDXL Turbo in the browser, along with some custom pixel art LoRAs: https://gfodor.github.io/image-gen/index.html https://gfodor.github.io/image-gen/index.html
craigacp 3y ago
ONNX Runtime needed a good demo application in Java and this was fun to do (I maintain the Java API for ONNX Runtime and wrote this SD implementation). I've added SDv2 and SDXL support (and the turbo variants thereof) after the initial release, and I'll upgrade it to the latest ONNX Runtime when that comes out to get FP16 support among other things.
- deleted 3y ago
  [deleted]
- xrd 3y ago
  Did you write this code? Any good tutorials for ONNX you recommend?
  - craigacp 3y ago
    I did. It depends what you want, for an overview of how ONNX Runtime works then Microsoft have a bunch of things on https://onnxruntime.ai https://onnxruntime.ai, but the Java content is a bit lacking on there as I've not had time to write much. Eventually I'll probably write something similar to the C# SD tutorial they have on there but for the Java API. For writing ONNX models from Java we added an ONNX export system to Tribuo in 2022 which can be used by anything on the JVM to export ONNX models in an easier way than writing a protobuf directly. Tribuo doesn't have full coverage of the ONNX spec, but we're happy to accept PRs to expand it, otherwise it'll fill out as we need it.
    - galangalalgol 3y ago
      I have been very impressed at how performant the runtime's cpu inference is. It beat out hand written avx intrinsics by almost an order of magnitude. I had to go find a machine with no discrete gpu to convince myself it wasn't using one despite it saying it wasn't.
neonsunset 3y ago
[flagged]
- deleted 3y ago
  [deleted]
neonsunset 3y ago
[flagged]
- deleted 3y ago
  [deleted]

77 points by simonpure 3y ago

vinni2 3y ago

[flagged]

tormeh 3y ago
I dislike Java, but some poor coders are writing code in MUMPS for their poor customers. Java is not a particularly good language, but it's not the worst thing out there.
ltbarcly3 3y ago
There are a lot of relatively inexpensive, relatively fungible engineers, especially offshore. (not really but this is what idiot middle managers and VPs will tell you). Lots if not most of the biggest and most profitable businesses in the world are built primarily using Java. (this is true).
asylteltine 3y ago
I agree. Every single time I a new tool that’s written in Java, I sigh. Just… why? I understand old legacy enterprise stuff, but new things??
- gbear605 3y ago
  I’ve tried lots of different languages, and for building actual large applications I just prefer Java. The language has changed a lot in the last ten years to be more agile and enjoyable to use.
mschuster91 3y ago
Assuming you do get cross-compilation sorted out once and get your libraries compiled to JNI JARs, it's easier to get running than the dependency hell associated with shipping C++ stuff around.
ramijames 3y ago
"Just because it isn't useful for me specifically, then nobody else should use it."
smusamashah 3y ago
As a Java dev now, this comment does not make sense. Doing Java for 4 years now and I only appreciate how much it can do (working on MMO which is written in Java both backend and fronted).
62951413 3y ago
It's a real mystery. Some popular conspiracy theories: * Scala projects the Haskell-on-the-JVM image too much to scare off the practically-minded. It has no ORCL or GOOG behind it (its cousin F# has had official MSFT support for a decade at least). * Kotlin is strongly associated with the frontend. Just try to search for a backend Kotlin role in the Bay Area to see. * Java 17-21 is catching up with Kotlin fast enough to make many people doubt the upskilling effort.
- pjmlp 3y ago
  Kotlin only matters on Android, because Google says so, and even them had to accept Kotlin without Maven ecosystem is meaningless, increasingly moving into modern Java, thus Android updates to Java 11, followed by Java 17.
pjmlp 3y ago
Because Java, alongside .NET, are the heirs of Xerox PARC tooling and development school.

rvz 3y ago

[flagged]

lbalazscs 3y ago
The ONNX Runtime is written in C++ and it uses the GPU. And Java is certainly much faster than Python.
- rvz 3y ago
  That means nothing if one uses an inefficient language for both inference and training and the slower language will still bottleneck the ONNX runtime FFI calls and on top of Java's own garbage collector. For those that care about performance know that the worst case is taken into account regardless if it is written in C++ for both memory and runtime performance.
pjmlp 3y ago
At least Java cares about JIT and AOT compilers, unlike Python that needs to ping back into C, C++ or Fortran for anything beyond UNIX scripting.

ShamelessC 3y ago

[flagged]

refulgentis 3y ago
?
- ShamelessC 3y ago
  Sarcasm. It would be great satire, if it actually were.
  - refulgentis 3y ago
    Of what?

deleted 3y ago

[deleted]

bmc7505 3y ago

Old enough to remember the days when Java reached near AI-level hype. Most of the advanced placement and undergraduate CS courses in the U.S. were taught in Java. There was an annual conference in San Fransisco called JavaOne/Oracle OpenWorld that rivaled the size of even NeurIPS, drawing tens of thousands of attendees each year. According to some statistics, there were 10 million Java developers and 3 billion devices running Java worldwide. I wonder what happened? Why does HN dislike Java so much? edit: Someone should really take the time to highlight all the innovative machine learning research happening in Java. The original implementation of t-SNE [1] was written in Java. Most NLP researchers have heard of CoreNLP [2], also Java. One of the earliest ML libraries, Weka [3], was written in Java and is still actively developed at the University of Waikato. Noteworthy research on ML4Code specifically targets the Java language [4]. There's a bunch of published sketching algorithms for Java (e.g., DataSketches [5], t-digests, ddsketch et al.), featured in an invited talk [6] to NeurIPS this year. Just to name a few off the top of my head. [1]: https://github.com/lejon/T-SNE-Java https://github.com/lejon/T-SNE-Java [2]: https://github.com/stanfordnlp/CoreNLP https://github.com/stanfordnlp/CoreNLP [3]: https://www.cs.waikato.ac.nz/ml/weka/ https://www.cs.waikato.ac.nz/ml/weka/ [4]: https://openreview.net/pdf?id=bUDmRzeh3PT https://openreview.net/pdf?id=bUDmRzeh3PT [5]: https://github.com/apache/datasketches-java https://github.com/apache/datasketches-java [6]: https://nips.cc/virtual/2023/events/Invited%20Talk https://nips.cc/virtual/2023/events/Invited%20Talk

ivan_gammel 3y ago
HN doesn't really dislike it, looking at the past threads. There are usually a few people quickly jumping to comment negatively, but in most cases they do not have enough experience with it or have not touched it since Java 8 or earlier version. Also, it is not a small thing to learn and it seems to be unpopular among the bootcamps - I have seen a lot of Python course ads, but hardly a single one on Java.
- deleted 3y ago
  [deleted]
62951413 3y ago
There was a _long_ hiatus in Java progress between 1.5 (release in September 2004) and 1.8 (released in March 2014). Before Java 8 the language was, basically, the previous generation's golang. in that timeframe there was no Kotlin, Scala had little toolchain support and some sharp edges. So many smart people left the JVM altogether.

genman 3y ago

[flagged]

craigacp 3y ago
Both GPU support and SIMD CPU support are being worked on in OpenJDK projects, there are talks on them from last year's JVM language summit - https://youtube.com/playlist?list=PLX8CzqL3ArzW90jKUCf4H6xCKpStxsOzp&si=T8-cgm7Njn0y9T41 https://youtube.com/playlist?list=PLX8CzqL3ArzW90jKUCf4H6xCK...
- genman 3y ago
  Thank you. It looks like there is still a lot of work to do.
ShamelessC 3y ago
[flagged]
- MaxBarraclough 3y ago
  > We were forced to use Java and then switched languages and saw the light. Other languages will do a better job in various ways, certainly, but in my opinion Java remains a pretty good middle-of-the-road language. If you want to go deep on high-performance computing, or functional programming, or type-safety, or low-level programming, then Java isn't going to win at any one of these. It's still broadly successful in what it's trying to achieve: a fairly fast, very portable language and virtual machine with various features to help the programmer deal with code-defects and achieve correctness, generally preferencing these (and portability) over performance where necessary. It also generally commits to backward compatibility. There is no one language to rule them all. It's more a matter of Pareto efficiency.
- severino 3y ago
  Who forced you to use Java and why?
- belter 3y ago

gfodor 3y ago

If you liked this you might like SDXL Turbo in the browser, along with some custom pixel art LoRAs: https://gfodor.github.io/image-gen/index.html https://gfodor.github.io/image-gen/index.html

craigacp 3y ago

ONNX Runtime needed a good demo application in Java and this was fun to do (I maintain the Java API for ONNX Runtime and wrote this SD implementation). I've added SDv2 and SDXL support (and the turbo variants thereof) after the initial release, and I'll upgrade it to the latest ONNX Runtime when that comes out to get FP16 support among other things.

deleted 3y ago
[deleted]
xrd 3y ago
Did you write this code? Any good tutorials for ONNX you recommend?
- craigacp 3y ago
  I did. It depends what you want, for an overview of how ONNX Runtime works then Microsoft have a bunch of things on https://onnxruntime.ai https://onnxruntime.ai, but the Java content is a bit lacking on there as I've not had time to write much. Eventually I'll probably write something similar to the C# SD tutorial they have on there but for the Java API. For writing ONNX models from Java we added an ONNX export system to Tribuo in 2022 which can be used by anything on the JVM to export ONNX models in an easier way than writing a protobuf directly. Tribuo doesn't have full coverage of the ONNX spec, but we're happy to accept PRs to expand it, otherwise it'll fill out as we need it.
  - galangalalgol 3y ago
    I have been very impressed at how performant the runtime's cpu inference is. It beat out hand written avx intrinsics by almost an order of magnitude. I had to go find a machine with no discrete gpu to convince myself it wasn't using one despite it saying it wasn't.

neonsunset 3y ago

[flagged]

deleted 3y ago
[deleted]

neonsunset 3y ago

[flagged]

deleted 3y ago
[deleted]