2 ms·
What are the speeds of other existing methods?
by KaoruAoiShiho 4y ago
What are the speeds of other existing methods?
- borzunov 4y agoTheoretical best-case for RAM offloading is 5.5 sec/token, for SSD offloading - 22 sec/token. Implementations we've tested are not faster than 10 sec/token though. See details in our paper: https://arxiv.org/pdf/2209.01188.pdf https://arxiv.org/pdf/2209.01188.pdf