It took us quite a bit of work to reduce the static part of our prompt from ~2300 tokens to ~500 tokens. I wish there was a good playbook for this. I used gpt-4

It took us quite a bit of work to reduce the static part of our prompt from ~2300 tokens to ~500 tokens. I wish there was a good playbook for this. I used gpt-4 a bit for some ideas, but almost all of it was just getting extremely creative with terse ways to represent info and cut cruft. Additionally, we've learned that LLMs lose a lot of context in the middle of a prompt, and so it might also be that a bunch of data not near the beginning or end of a prompt could just get removed entirely, or at least much more tersely summarized.