tldr: Language is bulky and LLMs can infer a lot of missing tokens. A simple optimization such as removing stop words with a probability `p` could lead saving a

tldr: Language is bulky and LLMs can infer a lot of missing tokens. A simple optimization such as removing stop words with a probability `p` could lead saving around a lot of token costs, especially over a large number of API calls. There's a lot that can be done but for now we provide a set of preliminary optimizers and their performance evaluation here: https://github.com/vaibkumr/prompt-optimizer#evaluations https://github.com/vaibkumr/prompt-optimizer#evaluations There's a lot that can't be done because we don't assume access to model weights, logits or decoding process. We are stuck with optimizing the number of tokens in input prompts. No free lunch: The reduction in cost often comes with a loss in LLM performance as can be observed here: https://github.com/vaibkumr/prompt-optimizer#cost-performance-tradeoff https://github.com/vaibkumr/prompt-optimizer#cost-performanc...