4 ms·

The title is less clear than it could be IMO. When I saw "no dependency" I thought maybe it could be no_std (llama.c is relatively lightweight in this regard).

The title is less clear than it could be IMO. When I saw "no dependency" I thought maybe it could be no_std (llama.c is relatively lightweight in this regard). But it's definitely not `no_std` and in fact seems like it has several dependencies. Perhaps all of them are rust dependencies?

saghm 2y ago
The readme seems to indicate that it expects pytorch alongside several other Python dependencies in a requirements.txt file (which is the only place I can find any form of the word "dependency" on the page). I'm very confused by the characterization in the title here given that it doesn't seem to be claimed at all by the project itself (which simple has the subtitle "Minimal LLM inference in Rust"). From the git history, it looks like the username of the person who posted this here is someone who has contributed to the project but isn't the primary author. If they could elaborate on what exactly they mean by saying this has "zero dependencies", that might be helpful.
- littlestymaar 2y ago
  > The readme seems to indicate that it expects pytorch alongside several other Python dependencies in a requirements.txt file That's only if you want to convert the model yourself, you don't need that if you use the converted weights on the author's huggingface page (in “prepared-models” table of the README). > From the git history, it looks like the username of the person who posted this here is someone who has contributed to the project but isn't the primary author. Yup that's correct, so far I've only authored the dioxus GUI app. > If they could elaborate on what exactly they mean by saying this has "zero dependencies", that might be helpful. See my other response: https://news.ycombinator.com/item?id=41812665 https://news.ycombinator.com/item?id=41812665
  - J_Shelby_J 2y ago
    What do you think about implementing your gui for other rust LLM projects? I’m looking for a front end for my project: https://github.com/ShelbyJenkins/llm_client https://github.com/ShelbyJenkins/llm_client
vitaminka 2y ago
is rust cargo basically like npm at this point? like how on earth is sixteen dependencies means no dependencies lol
- littlestymaar 2y ago
  > like how on earth is sixteen dependencies means no dependencies lol You're counting optional dependencies used in the binaries which isn't fair (obviously the GUI app or the backend of the webui are going to have dependencies!). But yes 3 dependencies isn't literally no dependency.
- tormeh 2y ago
  Yes, basically. Someone who is a dependency maximalist (never write any code that can be replaced by a dependency) then you can easily end up with a thousand dependencies. I don't like things being that way, but others do. It's worth noting that Rust's std library is really small, and you therefore need more dependencies in Rust than in some other languages like Python. There are some "blessed" crates though, like the ones maintained by the rust-lang team themselves (https://crates.io/teams/github:rust-lang:libs https://crates.io/teams/github:rust-lang:libs and https://crates.io/teams/github:rust-lang-nursery:libs https://crates.io/teams/github:rust-lang-nursery:libs). Also, when you add a dependency like Tokio, Axum, or Polars, these are often ecosystems of crates rather than singular crates. Tl;dr: Good package managers end up encouraging micro-dependencies and dependency bloat because these things are now painless. Cargo is one of these good package managers.
  - jll29 2y ago
    How about designing a "proper" standard library for Rust (comparable to Java's or CommonLISP's), to ensure a richer experience, avoiding dependency explosions, and also to ensure things are written in a uniform interface style? Is that something the Rust folks are considering or actively working on? EDIT: nobody is helped by 46 regex libraries, none of which implements Unicode fully, for example (not an example taken from the Rust community).
    - 2y ago
littlestymaar 2y ago
Titles are hard. What I wanted to express is that it doesn't have any pytorch or Cuda or onnx or whatever deep learning dependency and that all the logic is self contained. To be totally transparent it has 5 Rust dependencies by default, two of them should be feature gated for the chat (chrono and clap), and then there are 3 utility crates that are used to get a little bit more performance out of the hardware (`rayon` for easier parallelization, `wide` for helping with SIMD, and `memmap2` for memory mapping of the model file).
- J_Shelby_J 2y ago
  Yeah, hard to not be overly verbose. “No massive dependencies with long build times and deep abstractions!” Is not as catchy.
  - 0x457 2y ago
    No dependencies in this case (and pretty much any rust project) means: to build you need rustc+cargo and to use you just need resulting binary. As in you don't need to have C compiler, python, dynamic libraries. "pure rust" would be a better way to describe it.
    - littlestymaar 2y ago
      It's a little bit more than pure Rust: to build the library there's basically only two dependencies (rayon and wide) which bring only 14 transitive dependencies (anyone who's built even simple Rust program knows that this is a very small number). And there's more, Rayon and wide are only needed for performance and we could trivially put them behind a feature flag and have zero dependency and have the library work in a no-std context actually, but it would be so slow it would have no use at all so I don't really think that makes sense to do except in order to win an argument…
ctz 2y ago
The original may have made sense, eg "no hardware dependency", or "no GPU dependency". Unfortunately HN deletes words from titles with no rhyme or reason, and no transparency.

saghm 2y ago

The readme seems to indicate that it expects pytorch alongside several other Python dependencies in a requirements.txt file (which is the only place I can find any form of the word "dependency" on the page). I'm very confused by the characterization in the title here given that it doesn't seem to be claimed at all by the project itself (which simple has the subtitle "Minimal LLM inference in Rust"). From the git history, it looks like the username of the person who posted this here is someone who has contributed to the project but isn't the primary author. If they could elaborate on what exactly they mean by saying this has "zero dependencies", that might be helpful.

littlestymaar 2y ago
> The readme seems to indicate that it expects pytorch alongside several other Python dependencies in a requirements.txt file That's only if you want to convert the model yourself, you don't need that if you use the converted weights on the author's huggingface page (in “prepared-models” table of the README). > From the git history, it looks like the username of the person who posted this here is someone who has contributed to the project but isn't the primary author. Yup that's correct, so far I've only authored the dioxus GUI app. > If they could elaborate on what exactly they mean by saying this has "zero dependencies", that might be helpful. See my other response: https://news.ycombinator.com/item?id=41812665 https://news.ycombinator.com/item?id=41812665
- J_Shelby_J 2y ago
  What do you think about implementing your gui for other rust LLM projects? I’m looking for a front end for my project: https://github.com/ShelbyJenkins/llm_client https://github.com/ShelbyJenkins/llm_client

vitaminka 2y ago

is rust cargo basically like npm at this point? like how on earth is sixteen dependencies means no dependencies lol

littlestymaar 2y ago
> like how on earth is sixteen dependencies means no dependencies lol You're counting optional dependencies used in the binaries which isn't fair (obviously the GUI app or the backend of the webui are going to have dependencies!). But yes 3 dependencies isn't literally no dependency.
tormeh 2y ago
Yes, basically. Someone who is a dependency maximalist (never write any code that can be replaced by a dependency) then you can easily end up with a thousand dependencies. I don't like things being that way, but others do. It's worth noting that Rust's std library is really small, and you therefore need more dependencies in Rust than in some other languages like Python. There are some "blessed" crates though, like the ones maintained by the rust-lang team themselves (https://crates.io/teams/github:rust-lang:libs https://crates.io/teams/github:rust-lang:libs and https://crates.io/teams/github:rust-lang-nursery:libs https://crates.io/teams/github:rust-lang-nursery:libs). Also, when you add a dependency like Tokio, Axum, or Polars, these are often ecosystems of crates rather than singular crates. Tl;dr: Good package managers end up encouraging micro-dependencies and dependency bloat because these things are now painless. Cargo is one of these good package managers.
- jll29 2y ago
  How about designing a "proper" standard library for Rust (comparable to Java's or CommonLISP's), to ensure a richer experience, avoiding dependency explosions, and also to ensure things are written in a uniform interface style? Is that something the Rust folks are considering or actively working on? EDIT: nobody is helped by 46 regex libraries, none of which implements Unicode fully, for example (not an example taken from the Rust community).
  - 2y ago

littlestymaar 2y ago

Titles are hard. What I wanted to express is that it doesn't have any pytorch or Cuda or onnx or whatever deep learning dependency and that all the logic is self contained. To be totally transparent it has 5 Rust dependencies by default, two of them should be feature gated for the chat (chrono and clap), and then there are 3 utility crates that are used to get a little bit more performance out of the hardware (`rayon` for easier parallelization, `wide` for helping with SIMD, and `memmap2` for memory mapping of the model file).

J_Shelby_J 2y ago
Yeah, hard to not be overly verbose. “No massive dependencies with long build times and deep abstractions!” Is not as catchy.
- 0x457 2y ago
  No dependencies in this case (and pretty much any rust project) means: to build you need rustc+cargo and to use you just need resulting binary. As in you don't need to have C compiler, python, dynamic libraries. "pure rust" would be a better way to describe it.
  - littlestymaar 2y ago
    It's a little bit more than pure Rust: to build the library there's basically only two dependencies (rayon and wide) which bring only 14 transitive dependencies (anyone who's built even simple Rust program knows that this is a very small number). And there's more, Rayon and wide are only needed for performance and we could trivially put them behind a feature flag and have zero dependency and have the library work in a no-std context actually, but it would be so slow it would have no use at all so I don't really think that makes sense to do except in order to win an argument…

ctz 2y ago

The original may have made sense, eg "no hardware dependency", or "no GPU dependency". Unfortunately HN deletes words from titles with no rhyme or reason, and no transparency.