2 ms·

Strix Point isn't even out yet. Sure that is a laptop form factor and you won't be able to run those 130b models because of RAM constraints, but things like mix

Strix Point isn't even out yet. Sure that is a laptop form factor and you won't be able to run those 130b models because of RAM constraints, but things like mixtral 8x7b will easily run at 20 tokens per second locally on your laptop within this year.

dagmx 3y ago
And what will 20t/s give the user as a user experience? I don’t mean to sound combative, but my point is that the disconnect between what customers will get and what they see is very high. Will your metric meaningfully change that aspect?
- exe34 3y ago
  That's faster than you can read, isn't it? Or at least fast enough to deliver through voice (which is even slower than reading).