3 ms·
Not exactly. MoE uses a router model to select a subset of layers per token. This makes them faster but still requires the same amount of RAM.
by irthomasthomas 10mo ago
Not exactly. MoE uses a router model to select a subset of layers per token. This makes them faster but still requires the same amount of RAM.