2 ms·
What software did you use to set all this up? Kindof interested in giving this a shot myself.
by racked 2y ago
What software did you use to set all this up? Kindof interested in giving this a shot myself.
- azeirah 2y agoYou can use llama.cpp, it runs on almost all hardware. Whisper.cpp is similar, but unless you have a mid or high end nvidia card it will be a bit slower. Still very reasonable on modern hardware.
- bobbylarrybobby 2y agoIf you build locally for Apple hardware (instructions in the whisper.cpp readme) then it performs quite admirably on Apple computers as well.
- navbaker 2y agoDefinitely try it with Ollama, it is by far the simplest local LLM tool to get up and running with minimal fuss!