3 ms·
Show HN: Red Candle – Run LLMs Natively in Ruby with Rust
I've been working on Red Candle, a Ruby gem that runs LLMs (Llama, Mistral, Gemma, Phi) directly in your Ruby process through Rust bindings (based on the candle crate from Hugging Face). No Python, no servers - just FFI with Metal/CUDA acceleration.
It's been useful for adding AI features to Rails apps without the complexity of managing separate services. Would love feedback from anyone working with LLMs in Ruby.
- petercooper 1y agoI tried it and it worked, good effort. That TinyLlama 1.1B model was quite the shock though after being used to bigger models! I also got an angry reader email when I shared it in my newsletter saying they were "judging me" for posting AI stuff, lol.