3 ms·
In any batch inference environment that includes experts, expert routing may vary depending on what else is in the batch. For one thing.
by vessenes 5mo ago
In any batch inference environment that includes experts, expert routing may vary depending on what else is in the batch. For one thing.