4 ms·
This looks interesting if it works. A side benefit of prompt hacking like this appears when you transition from OpenAI to local, cost-static models, too. Lossle
by drusepth 4y ago
This looks interesting if it works. A side benefit of prompt hacking like this appears when you transition from OpenAI to local, cost-static models, too. Losslessly minimizing the number of tokens in a prompt should also (somewhat) reduce run times on local LLMs (especially on lower-end hardware, where token processing takes much longer), which might be important in some use-cases.