4 ms·

Candle is a minimalist ML framework for Rust with a focus on performance (including GPU support) and ease of use https://github.com/huggingface/candle https://

Candle is a minimalist ML framework for Rust with a focus on performance (including GPU support) and ease of use https://github.com/huggingface/candle https://github.com/huggingface/candle

imjonse 2y ago
Candle focuses on inference though.
- l-m-z 2y ago
  Candle dev here, we also support training/backdrop! We certainly focus on optimizing inference performance but hopefully that should improve the training efficiency too.
- revskill 2y ago
  What is referecing ?
  - HarHarVeryFunny 2y ago
    Inference means using the neural net, as opposed to training it. During inference you feed an input into the NN and it passes through it in "forwards" direction (i.e. from input to output), being modified according to the "weights" that were learnt during training, to derive the output. During training, each training sample is first fed forwards through the NN, the same way as for inference, but then the output of the model (which at the beginning of training will be random/wrong) is compared to the correct/desired output for that training sample, and a corresponding error value will then be fed backwards (from output to input) through the NN according to the "backpropagation" mechanism to update the weights. Training is a lot more involved than inference since it involves this backpropagation step.
basbuller 2y ago
Not barely as minimal as Karpathy his implementation
jeroenvlek 2y ago
Love Candle! I actually ported Karpathy's previous GPT tutorial to candle, including training [0] [0] https://www.perceptivebits.com/building-gpt-from-scratch-in-rust-and-candle/ https://www.perceptivebits.com/building-gpt-from-scratch-in-...
0xfedbee 2y ago
I wouldn't call in "minimalist" after seeing Karpathy's code.
r-k-jo 2y ago
You can also target WASM. Depending on the model size, it can be really good. Here are some examples of quantized models. Vision Model https://huggingface.co/spaces/radames/Candle-Moondream-2 https://huggingface.co/spaces/radames/Candle-Moondream-2 Blip Image Captioning https://huggingface.co/spaces/radames/Candle-BLIP-Image-Captioning https://huggingface.co/spaces/radames/Candle-BLIP-Image-Capt... Microsoft Phi 2 https://huggingface.co/spaces/radames/Candle-phi1-phi2-wasm-demo?model=phi_2_0_q4k https://huggingface.co/spaces/radames/Candle-phi1-phi2-wasm-...

Candle is a minimalist ML framework for Rust with a focus on performance (including GPU support) and ease of use https://github.com/huggingface/candle https://github.com/huggingface/candle

imjonse 2y ago

Candle focuses on inference though.

l-m-z 2y ago
Candle dev here, we also support training/backdrop! We certainly focus on optimizing inference performance but hopefully that should improve the training efficiency too.
revskill 2y ago
What is referecing ?
- HarHarVeryFunny 2y ago
  Inference means using the neural net, as opposed to training it. During inference you feed an input into the NN and it passes through it in "forwards" direction (i.e. from input to output), being modified according to the "weights" that were learnt during training, to derive the output. During training, each training sample is first fed forwards through the NN, the same way as for inference, but then the output of the model (which at the beginning of training will be random/wrong) is compared to the correct/desired output for that training sample, and a corresponding error value will then be fed backwards (from output to input) through the NN according to the "backpropagation" mechanism to update the weights. Training is a lot more involved than inference since it involves this backpropagation step.

basbuller 2y ago

Not barely as minimal as Karpathy his implementation

jeroenvlek 2y ago

Love Candle! I actually ported Karpathy's previous GPT tutorial to candle, including training [0] [0] https://www.perceptivebits.com/building-gpt-from-scratch-in-rust-and-candle/ https://www.perceptivebits.com/building-gpt-from-scratch-in-...

0xfedbee 2y ago

I wouldn't call in "minimalist" after seeing Karpathy's code.

r-k-jo 2y ago

You can also target WASM. Depending on the model size, it can be really good. Here are some examples of quantized models. Vision Model https://huggingface.co/spaces/radames/Candle-Moondream-2 https://huggingface.co/spaces/radames/Candle-Moondream-2 Blip Image Captioning https://huggingface.co/spaces/radames/Candle-BLIP-Image-Captioning https://huggingface.co/spaces/radames/Candle-BLIP-Image-Capt... Microsoft Phi 2 https://huggingface.co/spaces/radames/Candle-phi1-phi2-wasm-demo?model=phi_2_0_q4k https://huggingface.co/spaces/radames/Candle-phi1-phi2-wasm-...