3 ms·
Because that's where the compute happens, in those "verbose" tokens. A transformer has a size, it can only do so many math operations in one pass. If your probl
by dist-epoch 8mo ago
Because that's where the compute happens, in those "verbose" tokens. A transformer has a size, it can only do so many math operations in one pass. If your problem is hard, you need more passes.
Asking it to be shorter is like doing fewer iteration of numerical integral solving algorithm.
- sambaumann 8mo agoYeah, but all the models live in chatGPT have reasoning (iirc) - they could use reasoning tokens to do the 'compute', and still show the user a succinct response that directly answers the query