3 ms·
yes I read that. do you think it's reasonable to assume that the same expert will be selected so consistently that model swapping times won't dominate total run
by read_if_gay_ 3y ago
yes I read that. do you think it's reasonable to assume that the same expert will be selected so consistently that model swapping times won't dominate total runtime?
- tarruda 3y agoNo idea TBH, we'll have to wait and see. Some say it might be possible to efficiently swap the expert weights if you can fit everything in RAM: https://x.com/brandnarb/status/1733163321036075368?s=20 https://x.com/brandnarb/status/1733163321036075368?s=20