4 ms·
Web LLM
- ccozan 4y agoI find theses affirmations a bit contradictory: "This opens up a lot of fun opportunities to build AI assistants for everyone" "We have tested it on windows and mac, you will need a gpu with about 6.4G memory."
- capableweb 4y agoYeah, I don't think it's super common to have that available, in most segments. Although I think Apple doesn't sell computers with less than 8GB RAM, which is used by the GPU as well as the memory is unified. And latest Valve survey says that at least ~54% of people using Steam who participate in the survey, has a GPU with 8GB or more. According to https://web3dsurvey.com/webgpu https://web3dsurvey.com/webgpu WebGPU enjoys a whopping ~6% browser support. But the previous two segments are very small ones, if you compare to everywhere computers are being used. Coupled together with that WebGPU is hardly enabled anywhere, I wouldn't be surprised if maybe only ~1% of everyone using their own computer have access to 8GB VRAM + can use WebGPU. So yeah, bit weird to say everyone when looking at the requirements of 8GB VRAM + WebGPU enabled browser.
- flohofwoe 4y ago> WebGPU enjoys a whopping ~6% browser support. That's actually not too bad considering that WebGPU hasn't been released yet (the number will drastically go up when Chrome 113 is out of beta)
- capableweb 4y ago> the number will drastically go up when Chrome 113 is out of beta Which coincidentally is happening in about 5 hours (but it'll take many days for the update to actually go everywhere) according to https://chromestatus.com/roadmap https://chromestatus.com/roadmap, exciting :)
- youssefabdelm 4y agoCan someone explain why these types of projects are compelling (running an LLM in the user's browser?) I guess privacy and ease-of-use? So you don't need to download llama.cpp, and run it in the terminal or something?
- Veen 4y agoYeah, ease of use. I haven't wanted to bother with running local LLMs until now because I expected it would be complicated and time consuming (and I'm technically competent). I managed to get this up-and-running in about five minutes.
- sourcecodeplz 4y agoI thought the same but llama.cpp is literally two downloads in the same folder and then you double click an .exe
- EGreg 4y agoYou lost me at .exe
- woah 4y agoReminds me of the HN comments on the original dropbox announcement: https://news.ycombinator.com/item?id=8863 https://news.ycombinator.com/item?id=8863
- AmericanChopper 4y agoI know this comment is a bit of a meme, but in my experience “well actually you can already do this on Linux” is almost universally a calling card of an impending bad opinion.
- tpoacher 4y agoWhy bad? OP from 2007 still has a point. Actually, given dropbox's deterioration over time, FTP+SVN is sounding pretty good to me right now.
- radiojasper 4y agoIt doesn't run on Firefox, so I can't see what it's about. It also doesn't run on Edge and I don't have Chrome installed, obviously.