3 ms·
If you're working with local LLMs, why do you care about cost?
by kbelder 1y ago
If you're working with local LLMs, why do you care about cost?
- jsemrau 1y agoYou can use a lower-end GPU (like the RTX 3060), which also uses less energy. But you are right, you won't be encountering model API costs when running it locally.