3 ms·
> [June 2024] v6 MoE model How does that work for RWKV architecture? Wouldn't you have to feed the same data through all the experts regardless if they're curr
by viraptor 3y ago
> [June 2024] v6 MoE model
How does that work for RWKV architecture? Wouldn't you have to feed the same data through all the experts regardless if they're currently active or not to keep the rolling state consistent? Or am I misunderstanding that architecture?