4 ms·
The lmsys had a package (flexgen) that did a lot of this similar work (swap GPU to ram to disk) not sure if it's still being maintained
by htrp 2y ago
The lmsys had a package (flexgen) that did a lot of this similar work (swap GPU to ram to disk)
not sure if it's still being maintained