4 ms·
The claim of 70% of 4090 is very strange, my 4090 runs a 30b at roughly 25 tokens/second compared to the 1token/second claimed by the p40 user here: https://new
by KaoruAoiShiho 3y ago
The claim of 70% of 4090 is very strange, my 4090 runs a 30b at roughly 25 tokens/second compared to the 1token/second claimed by the p40 user here: https://news.ycombinator.com/item?id=35861360 https://news.ycombinator.com/item?id=35861360
- MacsHeadroom 3y ago>compared to the 1token/second claimed by the p40 user That user is doing something wrong. They may not be cooling it and are getting thermal throttled. That would be my guess. The P40 is capable of upwards of 10 tokens/second with 30b.