3 ms·
I’ve got a local llama 3.2 3B running on my macOS. I can query it for recipes, autocomplete obvious code (this is the only thing GitHub Copilot was useful for).
by henry2023 2y ago
I’ve got a local llama 3.2 3B running on my macOS. I can query it for recipes, autocomplete obvious code (this is the only thing GitHub Copilot was useful for). And answer simple questions when provided with a little bit of context.
All with much lower latency than an HTTP request to a random place, knowing that my data can’t be used to trading anything, and it’s free.
It’s absolutely insane this is the real world now.
- nicolas_t 2y agoWhat do you use to interface with llama for autocomplete? and what editor do you use? Not wanting my data to be sent to random places is what has limited my use of tools like copilot (so I'd only use it very sparingly after thinking if sending the data would be a breach of nda or not)
- grahamj 2y agoollama + VSCode + Continue extension
- mark_l_watson 2y ago+1 for sure running LLMs 3.2 3B is super fast and useful. I have been pushing it for local RAG and code completion also. I bought a 32B memory Mac six months ago, which I now regret because the small local models are now extremely useful and run fine on old 8B memory Macs, and support all the fun experiments I want to do.