4 ms·

Another anecdotal factoid ... Not a single LLM based example on GitHub seems to be using Java.

Onawa 3y ago
How well can you access graphics drivers and cuda cores for doing any kind of local large language model stuff using java? I know that there is a project called MLC for machine learning compilation that is meant to be written in Python and then transcribed using machine learning to a native language for the platform so that you can use mobile processors and laptops and stuff more efficiently for small language models.
- water9 3y ago
  Java has the Java native interface where you can interface with c/c++ written code
  - nradov 3y ago
    JNI is kind of a hassle. If most of the heavy lifting is being done by native code then using Java becomes pointless. Might as well just go with Python which makes calling native code a bit easier.
    - cempaka 3y ago
      Project Panama is meant to make this a lot nicer, and they often explicitly mention trying to stay competitive in machine learning as a key impetus for it. But yeah, JNI is so awful even the language architects are compelled to trash it and apologize for how bad it is.
    - water9 3y ago
      I cant argue with you there, but Java isn’t that much slower than c++ now a-days. especially on an X86 platform where there’s been lots of time to tune the JVM.
      - nradov 3y ago
        Sure, Java performance is good but again if most of the real work is being done in native code then it doesn't really matter.
graypegg 3y ago
This feels hard to verify. The LLM tag, filtered by Java repos [0] returns some results. This only includes repos that tagged their repos which definitely is not all of them. How are you measuring this? Anecdotal factoid is a bit of an oxymoron [0] https://github.com/topics/llm?l=java https://github.com/topics/llm?l=java
- nwiswell 3y ago
  > Anecdotal factoid is a bit of an oxymoron Obnoxious pedant mode: it's actually not, because GP misused the word "factoid". From Wikipedia[1]: "A factoid is either an invented or assumed statement presented as a fact,[1][2] or a true but brief or trivial item of news or information." [1]: https://en.wikipedia.org/wiki/Factoid https://en.wikipedia.org/wiki/Factoid
  - graypegg 3y ago
    Heheh, I’ll take obnoxious pedantry over an anecdotal factoid any day!

How well can you access graphics drivers and cuda cores for doing any kind of local large language model stuff using java? I know that there is a project called MLC for machine learning compilation that is meant to be written in Python and then transcribed using machine learning to a native language for the platform so that you can use mobile processors and laptops and stuff more efficiently for small language models.

water9 3y ago
Java has the Java native interface where you can interface with c/c++ written code
- nradov 3y ago
  JNI is kind of a hassle. If most of the heavy lifting is being done by native code then using Java becomes pointless. Might as well just go with Python which makes calling native code a bit easier.
  - cempaka 3y ago
    Project Panama is meant to make this a lot nicer, and they often explicitly mention trying to stay competitive in machine learning as a key impetus for it. But yeah, JNI is so awful even the language architects are compelled to trash it and apologize for how bad it is.
  - water9 3y ago
    I cant argue with you there, but Java isn’t that much slower than c++ now a-days. especially on an X86 platform where there’s been lots of time to tune the JVM.
    - nradov 3y ago
      Sure, Java performance is good but again if most of the real work is being done in native code then it doesn't really matter.

graypegg 3y ago

This feels hard to verify. The LLM tag, filtered by Java repos [0] returns some results. This only includes repos that tagged their repos which definitely is not all of them. How are you measuring this? Anecdotal factoid is a bit of an oxymoron [0] https://github.com/topics/llm?l=java https://github.com/topics/llm?l=java

nwiswell 3y ago
> Anecdotal factoid is a bit of an oxymoron Obnoxious pedant mode: it's actually not, because GP misused the word "factoid". From Wikipedia[1]: "A factoid is either an invented or assumed statement presented as a fact,[1][2] or a true but brief or trivial item of news or information." [1]: https://en.wikipedia.org/wiki/Factoid https://en.wikipedia.org/wiki/Factoid
- graypegg 3y ago
  Heheh, I’ll take obnoxious pedantry over an anecdotal factoid any day!