3 ms·
People can read at a rate around 10 token/sec. So faster than that is pretty good, but it depends how wordy the response is (including chain of thought) and whe
by HPsquared 1y ago
People can read at a rate around 10 token/sec. So faster than that is pretty good, but it depends how wordy the response is (including chain of thought) and whether you'll be reading it all verbatim or just skimming.
- littlestymaar 1y ago> People can read at a rate around 10 token/sec. It really depends on the type of content you're generating: 10tk/s feels very slow for code but ok-ish for text.
- gtirloni 1y agoReading while words are flying by is really distracting. I believe it was mentioned at some point that 50t/s feels comfortable and ChatGPT aims for that (no source, sorry).