4 ms·

With OpenClaw and powerful local models like Kimi 2.5, these specs make a lot of sense.

by jordhy 7mo ago

jbellis 7mo ago
K2.5 isn't remotely a local model
- oofbey 7mo ago
  You can totally run it locally. If you have 500GB of RAM.
- deleted 7mo ago
  [deleted]
- razster 7mo ago
  Oh sure it is! I’ve helped set up an AI cluster rack with four K2.5s. With some custom tooling, we built our own local enterprise setup: Support ticketing system Custom chat support powered by our trained software-support model Resolved repository with detailed step-by-step instructions User-created reports and queries Natural language-driven report generation (my favorite — no more dragging filters into the builder; our (Secret) local model handles it for clients) In-application tools (C#/SQL/ASP.NET) to support users directly, since our software runs on-site and offline due to PPI A cool repair tool: import/export “support file packet patcher” that lets us push fixes live to all clients or target niche cases Qwen3 with LoRA fine-tuning is also incredible — we’re already seeing great results training our own models. There’s a growing group pushing K2.5s to run on consumer PCs (with 32GB RAM + at least 9GB VRAM) — and it’s looking very promising. If this works, we’ll be retooling everything: our apps and in-house programs. Exciting times ahead!
- zozbot234 7mo ago
  Technically you can get most MoE models to execute locally because RAM requirements are limited to the active experts' activations (which are on the order of active param size), everything else can be either mmap'd in (the read-only params) or cheaply swapped out (the KV cache, which grows linearly per generated token and is usually small). But that gives you absolutely terrible performance because almost everything is being bottlenecked by storage transfer bandwidth. So good performance is really a matter of "how much more do you have than just that bare minimum?"
- KPGv2 7mo ago
  of course it's not remotely local: remote and local are literally antonyms
bdavbdav 7mo ago
I’m not sure what model I’d trust locally with anything meaningful in Openclaw. The smaller/simpler the model is, the greater the chance of fluff answers is.
- john_alan 7mo ago
  GPT-OSS-120 works well.

With OpenClaw and powerful local models like Kimi 2.5, these specs make a lot of sense.

by jordhy 7mo ago

With OpenClaw and powerful local models like Kimi 2.5, these specs make a lot of sense.

jbellis 7mo ago

K2.5 isn't remotely a local model

oofbey 7mo ago
You can totally run it locally. If you have 500GB of RAM.
deleted 7mo ago
[deleted]
razster 7mo ago
Oh sure it is! I’ve helped set up an AI cluster rack with four K2.5s. With some custom tooling, we built our own local enterprise setup: Support ticketing system Custom chat support powered by our trained software-support model Resolved repository with detailed step-by-step instructions User-created reports and queries Natural language-driven report generation (my favorite — no more dragging filters into the builder; our (Secret) local model handles it for clients) In-application tools (C#/SQL/ASP.NET) to support users directly, since our software runs on-site and offline due to PPI A cool repair tool: import/export “support file packet patcher” that lets us push fixes live to all clients or target niche cases Qwen3 with LoRA fine-tuning is also incredible — we’re already seeing great results training our own models. There’s a growing group pushing K2.5s to run on consumer PCs (with 32GB RAM + at least 9GB VRAM) — and it’s looking very promising. If this works, we’ll be retooling everything: our apps and in-house programs. Exciting times ahead!
zozbot234 7mo ago
Technically you can get most MoE models to execute locally because RAM requirements are limited to the active experts' activations (which are on the order of active param size), everything else can be either mmap'd in (the read-only params) or cheaply swapped out (the KV cache, which grows linearly per generated token and is usually small). But that gives you absolutely terrible performance because almost everything is being bottlenecked by storage transfer bandwidth. So good performance is really a matter of "how much more do you have than just that bare minimum?"
KPGv2 7mo ago
of course it's not remotely local: remote and local are literally antonyms

bdavbdav 7mo ago

I’m not sure what model I’d trust locally with anything meaningful in Openclaw. The smaller/simpler the model is, the greater the chance of fluff answers is.

john_alan 7mo ago
GPT-OSS-120 works well.