3 ms·
How much does it cost to run these? I see mentions of Claude and I assume all of these tools connect to a third party LLM api. I wish these could be run locall
by thih9 8mo ago
How much does it cost to run these?
I see mentions of Claude and I assume all of these tools connect to a third party LLM api. I wish these could be run locally too.
- zozbot234 8mo agoYou need very high-end hardware to run the largest SOTA open models at reasonable latency for real-time use. The minimum requirements are quite low, but then responses will be much slower and your agent won't be able to browse the web or use many external services.
- hu3 8mo ago$3k Ryzen ai-max PCs with 128GB of unified ram is said to run this reasonably well. But don't quote me on it.
- kube-system 8mo agoYou can run openclaw locally against ollama if you want. But the models that are distilled/quantized enough to run on consumer hardware can have considerably poorer quality than full models.
- Veen 8mo agoAlso more vulnerable to prompt injection than the frontier models, which are still vulnerable, but less so.