4 ms·
I doubt it. There are billions of parameters and weights all combining in non trivial ways to produce the output. I don’t think you can describe any of its work
by uup 4y ago
I doubt it. There are billions of parameters and weights all combining in non trivial ways to produce the output. I don’t think you can describe any of its workings with simple terms like “doing token substitution” even if that’s what it looks like to us when we look at the output.
- joe-collins 4y agoTry it. Reduce the tokens to single characters but keep the structure. Does it work? Maybe single characters looks too much like an abbreviation and that throws it off, because those are found in sufficiently different contexts. Exchange to two characters, three. Throw similar inputs at it and find the point where it reliably "gets it".