3 ms·
At least for transformers, it can be kind of fixed with MOE + NVFP4 for small working set despite large resident size.
by cat_plus_plus 9mo ago
At least for transformers, it can be kind of fixed with MOE + NVFP4 for small working set despite large resident size.