4 ms·
> As a last step, the final transformer layers produce numbers called logits. From these logits, the transformer conjures a list of possible words and assigns e
by mdp2021 2mo ago
> As a last step, the final transformer layers produce numbers called logits. From these logits, the transformer conjures a list of possible words and assigns each a probability. And here is where SynthID diverges. Your friendly transformer rolls the dice and selects the next word from the list for output. But not your transformer with SynthID, as it plays by different rules. With SynthID, the game is rigged. SynthID loads the dice, changing probabilities in subtle ways. And it does it not to win any game but to create a signature of the generated text hidden in plain sight.
This may be a possible feature of current LLMs, but it will lose feasibility when we will achieve deterministic output (as duly). Duly output is the best possible reply, not a "good enough" reply.