2 ms·
Thanks for your answers! While it is seemingly hard to calculate it, maybe one should just make a database website that tracks specific setups (model, exact va
by lagrange77 1y ago
Thanks for your answers!
While it is seemingly hard to calculate it, maybe one should just make a database website that tracks specific setups (model, exact variant / quantisation, runner, hardware) where users can report, which combination they got running (or not) along with metrics like tokens/s.
Visitors could then specify their runner and hardware and filter for a list of models that would run on that.
- diggan 1y agoYeah, what you're suggesting sounds like it could be more useful than the "generalized calculators" people are currently publishing and using.