3 ms·
Imagine if we applied this train of logic to humans. "That artist saw a pelican at the beach once!" [cue the outrage] "He's not a real artist, he's a cheater a
by oceanplexian 3mo ago
Imagine if we applied this train of logic to humans.
"That artist saw a pelican at the beach once!" [cue the outrage] "He's not a real artist, he's a cheater and produces nothing original!"
- computably 3mo agoExcept, of course, LLMs are not humans, and they do not learn or "reason" in a way which even remotely resembles humans. Plus obviously humans can still overfit to a specific style of test.
- program_whiz 3mo agoThis is a sight-reading test. If a musician practices a piece for thousands of hours, it would no longer be an effective sight reading / creativity test. The purpose of the test was to see how models would compose something novel requiring the ability to compose orthogonal, normally unrelated, components into a coherent image.
- alexjplant 3mo agoWe do. People who, for example, memorize question banks to pass certification tests without knowing the underlying material are equally frowned upon for not having the problem solving skills that they purport to. I'll leave the contrasts between LLMs and people to the well-written sibling comments.
- xigoi 3mo agoMore like “This artist won the drawing competition because someone told her the theme in advance and the specifically practiced drawing pelicans for hundreds of hours.”