5 ms·
I've been wanting to try out local LLMs for a while but have always had the feeling that I would end up wasting my time. I imagined that I would have to spend a
by iamdamian 3y ago
I've been wanting to try out local LLMs for a while but have always had the feeling that I would end up wasting my time. I imagined that I would have to spend a perfectly good Saturday futzing around with lots of settings and environments and trying to free up hundreds of GB of hard drive space for a model or wouldn't have the right hardware to properly run one locally. I also imagined that this probably wouldn't work out and that I'd end up giving up, so I've kept this idea on the backburner.
Ollama's promise of a simple setup was enough to get me out of that rut. Within a minute of seeing your README, I decided that this would be easy enough to experiment with, and if I didn't get it right quickly, I would give up and be no worse off.
I'm glad I gave it a shot. I now have llama2 working locally on my laptop and am already thinking about what I can do with this.
All this is to say: I hope everyone working in this area recognizes that not only is it important to make local LLMs as effective as possible, but it's also critical to make them approachable and easy to set up. By doing this, you'll get more eyes on this tech and more passion for local and open LLMs.
- poyu 3y agoI’ve been having a blast with Koboldcpp with WizardLM and Silly Tavern + Stable Diffusion Webui on my M2 Max, it really is valid form of text based entertainment for me!
- dataexporter 3y agoAlso take a look at the Python modules that Simon Willison has made available for interfacing with local LLMs. They make it much easier and effective.
- brianjking 3y agohttps://gpt4all.io/index.html https://gpt4all.io/index.html is another great and simple way to get started.