4 ms·
Demo: Use WebAssembly to Run LLMs on Your Own Device with WasmEdge
- 3Sophons 3y agoLlamaEdge revolutionizes AI/LLM runtime with lightweight (<5MB), portable, and secure applications for diverse CPUs/GPUs across different OSes, simplifying development and deployment from local to edge and cloud. https://www.secondstate.io/LlamaEdge/ https://www.secondstate.io/LlamaEdge/
- illuminant 3y agoSuper cool idea, I've been hoping for such a thing! However, terrible interface! Firstly I hate blindly running shell scripts from third part URLs. Secondly yours is awful! It does too much. It would be better to instruct how to set up the wasm environment and download the LLM separately from running the dang thing. I'm laboriously separating these parts now to manually troubleshoot those that are failing. Thanks for your efforts however. If I get tinyllamma working I'll be piping it through piper for some speech output!
- kjqgqkejbfefn 3y agoYo, can you share your email ? We should speak. You will be interested in this: https://arxiv.org/abs/2110.04913 https://arxiv.org/abs/2110.04913
- illuminant 3y agothat's promethean.way /at/ protonmail /dot/ com
- 3Sophons 3y agoPlease let us know if you have other concerns or bump into any issues? There is a different LLM based article for each model, like the Mixtral MoE https://www.secondstate.io/articles/mixtral-8-7b/ https://www.secondstate.io/articles/mixtral-8-7b/
- 3Sophons 3y agoPlease look at this separated steps as requested! https://www.secondstate.io/articles/wasm-runtime-agi/ https://www.secondstate.io/articles/wasm-runtime-agi/