3 ms·

You don't want your tooling to have any effect on your output...

• mewpmewp2 3y ago
  There are no absolutes, especially with LLMs. What if tooling altered output by 0.01 percentage, the general variance of output is 5 percent and to build alternative to that tooling would cost quite a bit of money and time?
• fnordpiglet 3y ago
  I would expect reinjecting into the context repeats of the prior context would be considerably more disruptive. The language is primarily a syntax for specifying manipulations on prior context. It’s hard to imagine a tooling that could more concisely and unambiguously create the needed structure to refer to things existing in the prior context without repetition. And I would note the instructions to manipulate the context are an important part of the context and output. This is just a highly concise and unambiguous way that primarily exists outside the latent semantic space, so I would expect attention would be high given the way it’s composed. I would have used more unique tokens to even more draw out the instruction language.