4 ms·
Gemma-3n-E4B-it on my 2022 Galaxy Z Fold 4. CPU: 7.37 seconds to first token 35.55 tokens/second prefill speed 7.09 tokens/second decode speed 27.97 second
by nolist_policy 1y ago
Gemma-3n-E4B-it on my 2022 Galaxy Z Fold 4.
CPU:
7.37 seconds to first token
35.55 tokens/second prefill speed
7.09 tokens/second decode speed
27.97 seconds to complete the answer
GPU:
1.96 seconds to first token
133.40 tokens/second prefill speed
7.95 tokens/second decode speed
14.80 seconds to complete the answer
- cubefox 1y agoSo a apparently the NPU can't be used for models like this. I wonder what it is even good for.