4 ms·
Can you explain further? Why would it not be free if it's running locally
by FezzikTheGiant 2y ago
Can you explain further? Why would it not be free if it's running locally
- talldayo 2y agoI thought you meant "free" in terms of computational cost; running local is technically free of charge, but also requires a lot of processing power. Inferecing a properly-sized LLM will potentially starve the rest of your software from GPU/CPU/memory access, so you have to plan accordingly.