4 ms·

Care to say how many tokens per second you're getting?

• delusional 2y ago
  For the "How many r's in the word strawberry" total duration: 15.278476756s load duration: 14.982999ms prompt eval count: 47 token(s) prompt eval duration: 5ms prompt eval rate: 9400.00 tokens/s eval count: 377 token(s) eval duration: 15.257s eval rate: 24.71 tokens/s
  • pton_xd 2y ago
    15 seconds to count the number of r's in a string using 120 TFlops... Really puts into perspective that we've got a long way yet to go.
    • sangnoir 2y ago
      This specific model is designed to second-guess itself/ chack it's work. So the slowness is part of the point.