2 ms·
I really like the post that they mention (https://www.secondstate.io/articles/fast-llm-inference/ https://www.secondstate.io/articles/fast-llm-inference/). The
by xrd 3y ago
I really like the post that they mention (https://www.secondstate.io/articles/fast-llm-inference/ https://www.secondstate.io/articles/fast-llm-inference/). The reasons for avoiding python all resonate with me. I'm excited to play with WASI-NN (https://github.com/WebAssembly/wasi-nn https://github.com/WebAssembly/wasi-nn) and that rust code is very readable to load up a GGUL model.
- euazOn 3y agoRelated discussion: https://news.ycombinator.com/item?id=38246668 https://news.ycombinator.com/item?id=38246668