4 ms·

Can you explain your use case? I tried to get into offline llms, on my machine and even android but without discrete graphics, its a slow hog so I didnt enjoy i

Can you explain your use case? I tried to get into offline llms, on my machine and even android but without discrete graphics, its a slow hog so I didnt enjoy it but suppose I buy one, what then ?

andai 3y ago
I run Mistral-7B on an old laptop. It's not very fast and it's not very good, but it's just good enough to be useful. My use case is that I'm more productive working with a LLM but being online is a constant temptation and distraction. Most of the time I'll reach for offline docs to verify. So the LLM just points me in the right direction. I also miss Google offline, so I'm working on a search engine. I thought I could skip crawling by just downloading common crawl, but unfortnately it's enormous and mostly junk or unsuitable for my needs. So my next project is how to data-mine common crawl to extract just the interesting (to me) bits... When I have a search engine and a LLM I'll be able to run my own Phind, which will be really cool.
- luke-stanley 3y ago
  Presumably you could run things like PageRank, I'm sure people do this sort of thing with CommonCrawl. There are lots of variants of graph connectivity scoring methods and classifiers. What a time to be alive eh?
noman-land 3y ago
Yes, I have a side project that uses local whisper.cpp to transcribe a podcast I love and shows a nice UI to search and filter the contents. I use Mixtral 8x7b in chat interface via llamafile primarily to help me write python and sqlite code and as a general Q&A agent. I ask it all sorts of technical questions, learn about common tools, libraries, and idioms in an ecosystem I'm not familiar with, and then I can go to official documentation and dig in. It has been a huge force multiplier for me and most importantly of all, it removes the dread of not knowing where to start and the dread of sending your inner monologue to someone's stupid cloud. If you're curious: https://github.com/noman-land/transcript.fish/ https://github.com/noman-land/transcript.fish/ though this doesn't include any Mixtral stuff because I don't use it programmatically (yet). I soon hope to use it to answer questions about the episodes like who the special guest is and whatnot, which is something I do manually right now.
popf1 3y ago
> Can you explain your use case? pretty sure you can run it un-censored... that would be my use case

Can you explain your use case? I tried to get into offline llms, on my machine and even android but without discrete graphics, its a slow hog so I didnt enjoy it but suppose I buy one, what then ?

andai 3y ago

I run Mistral-7B on an old laptop. It's not very fast and it's not very good, but it's just good enough to be useful. My use case is that I'm more productive working with a LLM but being online is a constant temptation and distraction. Most of the time I'll reach for offline docs to verify. So the LLM just points me in the right direction. I also miss Google offline, so I'm working on a search engine. I thought I could skip crawling by just downloading common crawl, but unfortnately it's enormous and mostly junk or unsuitable for my needs. So my next project is how to data-mine common crawl to extract just the interesting (to me) bits... When I have a search engine and a LLM I'll be able to run my own Phind, which will be really cool.

luke-stanley 3y ago
Presumably you could run things like PageRank, I'm sure people do this sort of thing with CommonCrawl. There are lots of variants of graph connectivity scoring methods and classifiers. What a time to be alive eh?

noman-land 3y ago

Yes, I have a side project that uses local whisper.cpp to transcribe a podcast I love and shows a nice UI to search and filter the contents. I use Mixtral 8x7b in chat interface via llamafile primarily to help me write python and sqlite code and as a general Q&A agent. I ask it all sorts of technical questions, learn about common tools, libraries, and idioms in an ecosystem I'm not familiar with, and then I can go to official documentation and dig in. It has been a huge force multiplier for me and most importantly of all, it removes the dread of not knowing where to start and the dread of sending your inner monologue to someone's stupid cloud. If you're curious: https://github.com/noman-land/transcript.fish/ https://github.com/noman-land/transcript.fish/ though this doesn't include any Mixtral stuff because I don't use it programmatically (yet). I soon hope to use it to answer questions about the episodes like who the special guest is and whatnot, which is something I do manually right now.

popf1 3y ago

> Can you explain your use case? pretty sure you can run it un-censored... that would be my use case