3 ms·
My prediction? In about 6 months, every test, task, or use of a LLM for anything that requires a modicum of creativity is going to find that it only has a fixed
by NBJack 3y ago
My prediction? In about 6 months, every test, task, or use of a LLM for anything that requires a modicum of creativity is going to find that it only has a fixed set of "ideas" before it starts regurgitating them. [0] I can easily imagine this in their hypothetical shoe pitch question, and many models going for more factual answers have been rapidly showing this bias by design.
[0] https://www.marktechpost.com/2023/06/16/this-paper-tests-chatgpts-sense-of-humor-over-90-of-chatgpt-generated-jokes-were-the-same-25-jokes/ https://www.marktechpost.com/2023/06/16/this-paper-tests-cha...
- simonw 3y agoI'm very unimpressed by that study. Look at how they generated the jokes - they fed it a prompt that was a slight variation on "please tell me a joke" and then wrote about how the jokes weren't varied enough. https://github.com/DLR-SC/JokeGPT-WASSA23/blob/main/01_joke_generation.py https://github.com/DLR-SC/JokeGPT-WASSA23/blob/main/01_joke_... That's a bad way to use an LLM for joke generation. Try "tell me a joke about a sea lion" - then replace sea lion with any other animal. Or "tell me ten jokes about a lawyer on the moon" - combine concepts like that and you get an infinite variety of jokes. Some of them might even be funny!