4 ms·

In terms of pickled files, we may ask them to release the models in safetensors. That would be a big improvement. There's not much we can do about dependencies

by jxy 3y ago

In terms of pickled files, we may ask them to release the models in safetensors. That would be a big improvement. There's not much we can do about dependencies on pytorch or other python libraries. Perhaps people can make more independent implementations. The redundancy in implementations would help.

substation13 3y ago
I'm surprised that some of the cutting edge AI work is not being done in pure Rust. Why even have a binding layer?
- ElFitz 3y ago
  Python seems to have won over the Data / ML world. So I’d wager they use what they and their intended audience know.
  - tough 3y ago
    Someone implemented llama on Go recently, https://github.com/gotzmann/llama.go https://github.com/gotzmann/llama.go Wouldn't discard a rust implementation of some LLM architecture at some point
    - ElFitz 3y ago
      There’s also ggml, which is quite impressive considering they get whisper and other models to run on a phone.
    - Philpax 3y ago
      Hi! I'm a maintainer of https://github.com/rustformers/llama-rs https://github.com/rustformers/llama-rs. We're planning to expand our model support soon.
- MacsHeadroom 3y ago
  https://github.com/rustformers/llama-rs https://github.com/rustformers/llama-rs
sdenton4 3y ago
The pickles drive me crazy. Tensorflow saved models are a great way to solve the problem... Save the computation graph and weights, and drop all the crusty code dependencies. I think ONNX models are similar. I expect there should be a Jax equivalent at some point, as Jax is basically perfectly designed for this (everything is expressed in lax operations, which allows changing implementations for cpu/gpu/tpu freely... So just save the list of lax ops).