3 ms·
3.7tok/s decode ain't bad at all for that setup i mean
by spottedmarley 2mo ago
3.7tok/s decode ain't bad at all
for that setup i mean
- peppe200175 2mo agoYes it's the best performance i can reach with a lot of caching between network layers of experts. But is incredible to make it run at all !!!