3 ms·
It seems the only code that runs in the browser here is the code that talks to LLMs on servers. Why would you need WASM for this?
by TekMol 1y ago
It seems the only code that runs in the browser here is the code that talks to LLMs on servers.
Why would you need WASM for this?
- politelemon 1y agoThey're using some python libraries like openai-agents so presumably it's to save on development efforts of calling/prompting/managing the HTTP endpoints. But yes this could just be done in regular JS in the browser, they'd have to write a lot of boilerplate for an ecosystem which is mainly Python.
- yjftsjthsd-h 1y ago> But yes this could just be done in regular JS in the browser, they'd have to write a lot of boilerplate for an ecosystem which is mainly Python. Surely that's a prime use for AI?
- m13rar 1y agoFrom a quick gander. WASM is not to talk to the servers. WASM can be utilized to run AI Agents to talk to local LLMs from a sandboxed environment through the browser. For example in the next few years if Operating System companies and PC producers make small local models stock standards to improve the operating system functions and other services. This local LLM engine layer can be used by browser applications too and that being done through WASM without having to write Javascript and using WASM sandboxed layer to safely expose the this system LLM Engine Layer.
- TekMol 1y agoNo matter if the LLM is on the same machine or elsewhere, why would you need WASM to talk to it and not just JS?
- lgas 1y agoYou never need WASM (or any other language, bytecode format, etc) to talk to LLMs. But WASM provides things people might like for agents, eg. strict sandboxing by default.