5 ms·
I'm a big ole noob when it comes to local AI. What are you using for a harness? Or platform to interact with it?
by westpfelia 2mo ago
I'm a big ole noob when it comes to local AI. What are you using for a harness? Or platform to interact with it?
- ch_sm 2mo agoyou can try ollama, omlx or llama.cpp for instance to download a model and get an inference server running locally. They expose „open ai compatible“ endpoints, so you can configure almost any harness to use them.
- jlkuester7 2mo agoI recommend trying pi.dev as your agent harness for local models. In my experience it has been the sweet spot of functionality (which you can and should extend with plugins) vs performance (OpenCode just swamps local models on my hardware).