2 ms·

The trick is managing the context (number of tokens). It’s tempting to have a long conversation about tweaking some code, but a shorter context (last X messages

The trick is managing the context (number of tokens). It’s tempting to have a long conversation about tweaking some code, but a shorter context (last X messages), or ‘forking’ the conversation regularly can both improve performance and drastically lowers cost.

eurekin 3y ago
How do you go about that (besides manually editing the whole prompt)?
- blackoil 3y ago
  Get GPT3.5 (10th cost) to summarize previous conversation after every few prompts or once conversation history reaches certain size. Once your history is 5k tokens bring it down to 500.
  - eurekin 3y ago
    Oh! That's actually cool. Could use that mixtral on the next pc as well