4 ms·
I think your demo need more realistic thinking logs because thinking usually burns at least 2x to 3x of tokens of the code and for harder tasks much more.
by SXX 5mo ago
I think your demo need more realistic thinking logs because thinking usually burns at least 2x to 3x of tokens of the code and for harder tasks much more.
- redox99 5mo agoYes, it should use actual output from some of the open models.
- unglaublich 5mo agoIndeed, at 30tok/s make it pause for 20 seconds while "thinking" is streaming (and hidden); that's the real experience.
- deleted 5mo ago[deleted]
- sig_kill 5mo ago[dead]
- calmingsolitude 5mo agoYou've advertised twice on this thread about your site, but I personally find the UX to be worse, not to mention the phantom scroll issues and (imo) the tasteless animations.