4 ms·
Have you ever generated access frequency statistics for the experts in these models, something like a histogram?
by WithinReason 6mo ago
Have you ever generated access frequency statistics for the experts in these models, something like a histogram?
- Gracana 6mo agoktransformers can do dynamic placement of experts and could presumably produce such a histogram, though currently its activation statistics are just a ".pt" file. https://github.com/kvcache-ai/ktransformers/blob/main/doc/en/kt-kernel/experts-sched-Tutorial.md https://github.com/kvcache-ai/ktransformers/blob/main/doc/en... FWIW I never got it to work and did not dig into it much.