3 ms·
My point is that truth and lie are ambiguous. There is nothing in the metaphorical map to distinguish between them. Because a truth and a lie are written the sa
by thomastjeffery 3y ago
My point is that truth and lie are ambiguous. There is nothing in the metaphorical map to distinguish between them. Because a truth and a lie are written the same, they are the same. The model puts them in the same place.
The only difference an LLM can have is what other semantic groups of tokens are associated with the [truth|lie] tokens. If the surrounding [context of a truth] is separate enough from the surrounding [context of a lie], then the connections themselves can be unambiguous enough that the LLM doesn't mix them up. In the average case, there is some semantic/narrative overlap, which allows the [truth|lie] itself to act as a semantic bridge between the [context of truth] and the [context of lie]. Because of that, simply adding more content to a model can create bridges of ambiguity that fundamentally dissolve the model's ability to present unambiguous continuations.