The reason, i suspect, that this method won't work, is because the training dataset has knowledge embedded in it, regardless of whether you think it only contai

The reason, i suspect, that this method won't work, is because the training dataset has knowledge embedded in it, regardless of whether you think it only contains in-context logic. The evidence is that there are these "glitched" tokens : https://www.youtube.com/watch?v=WO2X3oZEJOA https://www.youtube.com/watch?v=WO2X3oZEJOA if you watched to the end of that video, it explores a very plausible reason why such glitched tokens exist - it is because the training dataset has a bunch of "nonsense" (a large dump of the subreddit that just counts a number, as well as various other "data"). So my hypothesis is that if you attempted the in-context logic training dataset, it would produce large number of glitched tokens (different ones, but equally as "illogical" to the user).