> GPT-3 was terrible at jokes. Many LLMs are because it's hard to model puns in a stochastic parrot GPT-3 was great at jokes. The Navy Seals were hilarious (ht

> GPT-3 was terrible at jokes. Many LLMs are because it's hard to model puns in a stochastic parrot GPT-3 was great at jokes. The Navy Seals were hilarious (https://gwern.net/gpt-3#navy-seals https://gwern.net/gpt-3#navy-seals). And the difficulty of modeling puns has nothing to do with 'stochastic parrots' and has everything to do with tokenization (https://gwern.net/gpt-3#bpes https://gwern.net/gpt-3#bpes), in the same way that all those hyberbolic takes about how image-generation models were 'fundamentally flawed' because they couldn't do good text in images turned out to be BS and solely a matter of tokenization - drop in a character-tokenized LLM instead, even an obsolete & dumb one, and it instantly works (https://arxiv.org/abs/2105.13626#google https://arxiv.org/abs/2105.13626#google).