4 ms·
The slow spell out is really annoying. If they want to rate limit fine. But give me the answer faster and then lock queries for a few seconds instead.
by baxtr 4y ago
The slow spell out is really annoying. If they want to rate limit fine. But give me the answer faster and then lock queries for a few seconds instead.
- cloudking 4y agoThe spell out is a limitation of the way the model works, it predicts the text word by word.
- baxtr 4y agoYes I get that. But just speed the god damn thing up.
- visarga 4y agoThe god damn thing has to do 175B multiply and add operations for each one of your words.
- baxtr 4y agoWho cares? OP was asking about UX improvements. Maybe I missed the part, but IMHO no one was asking only for UX improvements that come cheap.
- visarga 4y agoThey said "just speed the god damn thing up" - this is related to model inference speed. If your answer is one page long (about 2000 tokens), be prepared to wait for a whole minute.
- ryanSrich 4y agoI’d much prefer the type be hidden until it’s ready entirely.