3 ms·
IDK. Ignoring the "transformers predict the next token" statement, which feels at best technically correct but missing the point, I imagine this comes down to t
by TeMPOraL 3y ago
IDK. Ignoring the "transformers predict the next token" statement, which feels at best technically correct but missing the point, I imagine this comes down to the network learning "low-frequency" patterns in the training data. That is, in both training and instruct fine-tuning, the model is likely to encounter text structured like:
DATA DATA
DATA DATA ...
-- boundary --
QUERY
or the arguably equivalent:
QUOTED TEXT
-- boundary --
REPLY / COMMENTARY
The inverse shape is also common:
INSTRUCTIONS
-- boundary --
DATA / TEXT ON WHICH TO WORK
For example, most exercise lists and test books are written like that.
The somewhat less frequent patterns are more random mix of:
WHAT
-- boundary --
ON WHAT
-- boundary --
WHAT ELSE
-- boundary --
ON WHAT ELSE
-- boundary --
(...)
CLOSING REMARKS
Most of my HN comments are structured like that, for example. Including this one.
Boundary here can take many forms. Extra newlines, --, ``` blocks ```, > - prefixed text, and lists (both OL and UL) are all common methods used to structure text, and are seen both in training data and in inference. We know LLM picks up on those structure markets at high-frequency level (e.g. using extra newlines or -- lines to separate distinct blocks seems effective). But I imagine it also picks up on the low-frequency patterns, which is why payload followed by, or bracketed with, instructions is something it "knows" how to process, whereas if you use less common structuring patterns, you're more likely to confuse the LLM.