3 ms·
> Minimizing tokens out, or generally "token efficiency," is part of the objective function for these systems. First time I heard that, and I doubt it. Don’t c
by Someone 2mo ago
> Minimizing tokens out, or generally "token efficiency," is part of the objective function for these systems.
First time I heard that, and I doubt it. Don’t customers pay for output tokens? If so, why would a company specifically spend time training their LLM to generate fewer?
- yreg 2mo agoSo they can charge more per token and decrease the pressure on their infra.
- Someone 2mo agoEven if you specifically train the model on producing shorter answers, I would think producing good short answers would require more resources just as it does for humans (https://quoteinvestigator.com/2012/04/28/shorter-letter/ https://quoteinvestigator.com/2012/04/28/shorter-letter/: “If I Had More Time, I Would Have Written a Shorter Letter”) If so, charging per output token is the wrong incentive.
- yreg 2mo agoWhat billing system would you suggest?