3 ms·
There's a video of a previous version of this tool here which I found really helped me understand what it was demonstrating: https://twitter.com/thesephist/stat
by simonw 3y ago
There's a video of a previous version of this tool here which I found really helped me understand what it was demonstrating: https://twitter.com/thesephist/status/1617747154231259137 https://twitter.com/thesephist/status/1617747154231259137
It's really neat to see how this sentence:
> The first time I write this sentence, the model is quite confused about what token is about to come next, especially if I throw in weird words like pumpkin, clown, tweets, alpha, teddy bear.
Shows that the words pumpkin, clown etc are considered really unlikely. But when the sentence is repeated a moment later, all of the words become extremely predictable to the model.
Also worth noting: this demo runs entirely in the browser! It loads a 120MB ONNX version of GPT-2 using Transformers.js.