2 ms·

How are folks running 8x22B (or MoE in general) in an affordable way?

• dartos 3y ago
  For personal inference, llama.cpp can run some MoEs on CPU. So like that.