3 ms·
I know you were trying to prove some random guy on the internet wrong (and failed hilariously), but I linked a peer reviewed paper. That paper has even been cit
by Der_Einzige 4y ago
I know you were trying to prove some random guy on the internet wrong (and failed hilariously), but I linked a peer reviewed paper. That paper has even been cited by gwern in his article on GPT-3's creative writing capabilities: https://gwern.net/gpt-3 https://gwern.net/gpt-3. Sometimes people here DO know what they are talking about!
And even if you do somehow get lucky and get one prompt where it does this one time, it's never going to be reliable without significant evolution of the tokenizers.
Even if this is solved in the future in some technique involving fixing BPE/subword tokenizers, it's still sad that filter assisted decoding works today to fix it and no one is implementing it despite two tech demos being available showing that it works from the author.
- lorepieri 4y agoThe previous poster is gwern... Regardless, peer review does not mean that the arguments of the paper are correct (and viceversa), as gwern just shown.
- Der_Einzige 4y agoThe previous poster is not gwern. Gwern posted after. Gwern yet again agreed with me after failing 4 times in a row to get a paragraph without the letter e in it. (He finally got it on the 5th time...) Reading comprehension!