3 ms·
It absolutely does not mean nothing to say that poetry and songs exist in its training data. The criticism is not "it's just spitting out existing poems and son
by danaris 4y ago
It absolutely does not mean nothing to say that poetry and songs exist in its training data. The criticism is not "it's just spitting out existing poems and songs"; it's "all it knows how to do is mimic (to greater or lesser degrees of accuracy) things in its training data."
GPT-3 takes a massive corpus of text in, and then spits out something that it creates by munging that together in probabilistic ways, assisted (if I understand correctly) by various humans providing feedback on that.
It has no comprehension of what it is doing. It has no "mental model" of the world. It has no "consciousness" of any kind, at any level.
Because of this, it cannot genuinely imagine, synthesize, or be creative. Sure; it can put together sequences of words that had never been put together in exactly that configuration before, but that's not really the same thing as "creating something new."
It can be entertaining, sure. It can even inspire insights in us—but it can never be genuinely insightful itself, because that requires understanding.
- mavhc 4y agoBut are humans any different? We gather inputs, mix them together, produce outputs. The joke on humans is such a simple program as GPT3 can emulate humans so well. Is GPT3 already smarter than all non human animals?
- danaris 4y agoBefore any stories had ever been told, humans thought of them, and told them to each other. Before any paintings had ever been painted, humans imagined them from nothing, and painted what they imagined. Just because reading other stories increases our breadth of material to draw from doesn't mean we require them to create. GPT-3 has no ability to create something truly new, without reference to anything it's seen before. Period. It cannot imagine. It does not have consciousness. The hype around it has tricked you into thinking it is much more than it is—either that, or that we are much less.
- mavhc 4y agoHumans had eyes, and then recreated what they saw. Humans had memories, and retold what had happened. Rearranging previous things into new combinations is creation