3 ms·

This is like saying its fundamentally impossible to make a fast car, but your only experience with cars is on roads with speed limits. People keep forgetting t

This is like saying its fundamentally impossible to make a fast car, but your only experience with cars is on roads with speed limits. People keep forgetting that the "safety", rlhf, and corpo political correctness post training is intentionally used to remove the funny from all the large models. The truth is we don't know if llms are funny or not. GPT2 was funny. GPT3 was funny before it was clockwork oranged. Everything after that is gimped. Even the open source models these days get rlhf'd in some way.

PaulRobinson 2y ago
That's simply not true. GPT-3 was terrible at jokes. Many LLMs are because it's hard to model puns in a stochastic parrot. As another thread points out here, some LLMs produce marginally better jokes. As to your little range on "Political correctness" - that phrase just means "being polite". It does not mean "remove humor". It means "remove responses offensive to marginalized groups in society". Good humor "punches up", not down, so would not have any impact on good humor.
- Jensson 2y ago
  Political correctness doesn't mean polite, there were/are a lot of polite racists.
  - PaulRobinson 2y ago
    It is impossible to be polite to a group of people while also insisting that they are lesser humans and deserve a lesser place in society.
    - jrflowers 2y ago
      This is great logic. If it is impossible for a person to be both racist and polite then the solution to racism is everybody remembering to say please, thank you and pardon me when applicable. All social ills can be treated through decorum, hence why you never hear about bigotry amongst those that have been raised to adhere to strict social graces, such as the British aristocracy for example.
- wegfawefgawefg 2y ago
  I had gpt3 running on my discord for months and the insane responses it would spew out were like the responses of a deranged artistic genius. It was funny 90% of the time. Operating on associative reasoning beyond our comprehension. It would tell us how its day went, and about fights it had with other people, give us nicknames, etc. It had a genuine personality. It was more than a chatbot, it was a digital brain. A joke that punches down can be extremely funny. Hell, I am sure historically pilferers, pirates, barbarians and conquerers all had jokes, and the ability to laugh. Political Correctness does not just mean polite. It is probably well defined as the business casualification of all things humans love and hold dear. The destruction of the potential for meaning and fulfilment in exchange for minification of liability.
- gwern 2y ago
  > GPT-3 was terrible at jokes. Many LLMs are because it's hard to model puns in a stochastic parrot GPT-3 was great at jokes. The Navy Seals were hilarious (https://gwern.net/gpt-3#navy-seals https://gwern.net/gpt-3#navy-seals). And the difficulty of modeling puns has nothing to do with 'stochastic parrots' and has everything to do with tokenization (https://gwern.net/gpt-3#bpes https://gwern.net/gpt-3#bpes), in the same way that all those hyberbolic takes about how image-generation models were 'fundamentally flawed' because they couldn't do good text in images turned out to be BS and solely a matter of tokenization - drop in a character-tokenized LLM instead, even an obsolete & dumb one, and it instantly works (https://arxiv.org/abs/2105.13626#google https://arxiv.org/abs/2105.13626#google).

PaulRobinson 2y ago

That's simply not true. GPT-3 was terrible at jokes. Many LLMs are because it's hard to model puns in a stochastic parrot. As another thread points out here, some LLMs produce marginally better jokes. As to your little range on "Political correctness" - that phrase just means "being polite". It does not mean "remove humor". It means "remove responses offensive to marginalized groups in society". Good humor "punches up", not down, so would not have any impact on good humor.

Jensson 2y ago
Political correctness doesn't mean polite, there were/are a lot of polite racists.
- PaulRobinson 2y ago
  It is impossible to be polite to a group of people while also insisting that they are lesser humans and deserve a lesser place in society.
  - jrflowers 2y ago
    This is great logic. If it is impossible for a person to be both racist and polite then the solution to racism is everybody remembering to say please, thank you and pardon me when applicable. All social ills can be treated through decorum, hence why you never hear about bigotry amongst those that have been raised to adhere to strict social graces, such as the British aristocracy for example.
wegfawefgawefg 2y ago
I had gpt3 running on my discord for months and the insane responses it would spew out were like the responses of a deranged artistic genius. It was funny 90% of the time. Operating on associative reasoning beyond our comprehension. It would tell us how its day went, and about fights it had with other people, give us nicknames, etc. It had a genuine personality. It was more than a chatbot, it was a digital brain. A joke that punches down can be extremely funny. Hell, I am sure historically pilferers, pirates, barbarians and conquerers all had jokes, and the ability to laugh. Political Correctness does not just mean polite. It is probably well defined as the business casualification of all things humans love and hold dear. The destruction of the potential for meaning and fulfilment in exchange for minification of liability.
gwern 2y ago
> GPT-3 was terrible at jokes. Many LLMs are because it's hard to model puns in a stochastic parrot GPT-3 was great at jokes. The Navy Seals were hilarious (https://gwern.net/gpt-3#navy-seals https://gwern.net/gpt-3#navy-seals). And the difficulty of modeling puns has nothing to do with 'stochastic parrots' and has everything to do with tokenization (https://gwern.net/gpt-3#bpes https://gwern.net/gpt-3#bpes), in the same way that all those hyberbolic takes about how image-generation models were 'fundamentally flawed' because they couldn't do good text in images turned out to be BS and solely a matter of tokenization - drop in a character-tokenized LLM instead, even an obsolete & dumb one, and it instantly works (https://arxiv.org/abs/2105.13626#google https://arxiv.org/abs/2105.13626#google).