3 ms·

I like your site. WebAssembly has opened up client side ML inference and it's a really convenient way for devs to serve certain models now. Out of curiosity, wh

• sevagh 3y ago
  Good question! So, I wasn't even thinking about WASM to begin with. When I saw llama.cpp and whisper.cpp on the front page of HN, I found the idea exciting - instead of neural networks being magic, I wanted to copy the ggml idea of parsing the PyTorch weights file myself and rewriting the inference code in a lower-level language than Python (or, it's even more accurate to say PyTorch, since there is so much matrix heavy lifting e.g. broadcasting or reshaping that is done for you automatically). That's when I wrote umx.cpp [1] (which is what this site is based on). On an unrelated project, a friend of mine mentioned WASM, and as I looked into it a bit more I thought trying to compile umx.cpp to WASM would be a great idea, since I only use Eigen (which is a header-only library that only depends on std). 1: https://github.com/sevagh/umx.cpp https://github.com/sevagh/umx.cpp