3 ms·
"usable" is not the same as practical. Even running a quantized and optimized LLM on a smartphone would kill battery life at minimum.
by minimaxir 3y ago
"usable" is not the same as practical.
Even running a quantized and optimized LLM on a smartphone would kill battery life at minimum.
- brucethemoose2 3y agoTry MLC-LLM. Its not as bad as you'd think. In the future(?), they will probably use the AI blocks instead of the GPU, which are very low power.