4 ms·

Or https://ollama.com/ https://ollama.com/

• fortyseven 2y ago
  This has been my go-to for all of my local LLM interaction: it easy to get going, manages all of the models easily. Nice clean API for projects. Updated regularly; works across Windows, Mac, Linux. It's a wrapper around LlamaCpp, but it's a damned good one.
  • brewtide 2y ago
    Same here, however minimal. I've also installed openwebui so the instance has a local web interface, and then use tailscale to access my at home LAN when put and about on the cellphone. (Goes16 weather data, ollama, a speed cam setup, and esphome temp sensors around the home / property). It's been pretty flawless, and honestly pretty darn useful here and there. The big guns go faster and do more, but I'd prefer not having every interaction logged etc. 6core 8th gen i7 I think, with a 1050ti. Old stuff. And it's quick enough on the smaller 7/8b models for sure.