3 ms·
LLMs in the Browser
- deleted 2y ago[deleted]
- chomp 2y agoMost of this looks Chrome-only, so relatively not useful except to deepen their ad-hold on the wider ecosystem.
- ldenoue 2y agoActually we can run these networks today on any WebGPU enabled browser. If no WebGPU then transformersjs will still work (but slower). I use for example a DistilBert token classifier to restore punctuation in YouTube transcripts in Scribe: everything happens locally. Demo https://www.appblit.com/scribe https://www.appblit.com/scribe
- halJordan 2y agoAnd yet, of the three majors, who has general availability of webgpu? So not only has Google gotten to whatever even playing field you say you want before the others, they're pushing ahead with a more bespoke tool (seen here). At some point the egregious cynicism of "Well i hate evil ad-bringers so im dismissing everything else" has got to be tempered with some semblance of rationality.
- ssahoo 2y agoWith Webgpu users should be able to run other models as well. I have built chatllm.pro cheome extension that can run all sort of small models incliding the phi2 natively using a wasm library. You will be able to chat with the page as well