4 ms·
A future version of this might just be a local, offline LLM
by qgin 3y ago
A future version of this might just be a local, offline LLM
- cush 3y agoA future version of an ebook library would be an offline large language model? So like, instead of downloading Moby Dick, you can ask for it to be generated?
- n8fr8too 3y agoThere is definitely interest in providing off-grid communities easy access to lightweight LLMs... one idea is to use something like: https://github.com/mlc-ai/web-llm https://github.com/mlc-ai/web-llm to deliver it in a browser
- cush 3y agoOh I see. Was confused. The “this” in OP’s comment is an offline library. An offline LLM would be nice too.
- qgin 3y agoYeah, to clarify, I didn't mean it would be a direct replacement for literature, just that on the same theme of of providing a resource in places where connectivity is an issue, an LLM is a reasonable way to compress a whole lot of human knowledge into a relatively small space in a usable form.
- butterbox 3y agoI am a techno-skeptic most of the time, but I don't think this idea is particularly crazy. Usability-wise, something you can talk to (in your own "low resource" language) is a winner. The catch is that you have users without either the sophistication to size up how trustworthy this model is or to do out-of-band verification (remember: no internet). I think a RAG model serving answers from Wikipedia could be suitably safe, but we'd have to do some user research to know more.