3 ms·

> God of the Gaps, in other words. No, not really. Make a sufficiently-powerful predictive text engine, and you get something like GPT-3. Make a sufficiently-p

> God of the Gaps, in other words. No, not really. Make a sufficiently-powerful predictive text engine, and you get something like GPT-3. Make a sufficiently-powerful expert system, and you get a chess engine that can beat every grandmaster. These things were predicted in sci-fi novels with descriptions of their algorithms decades before we had computers powerful enough to execute them. For things like "intelligence" and "consciousness", the best we've got is on the order of Isaac Asimov's positronic brain: a total handwave, maybe with some technobabble or some oblique references to now-outdated models of psychology. The stuff humans can do that computers can't, are the things we don't have algorithms for, and – funnily enough – things that "just throw a bigger neural net at it" don't seem to be actually solving (however much they're getting better at fooling people into believing they're solving). "We might be able to solve this one day" doesn't mean we've already solved it.

CamperBob2 4y ago
The stuff humans can do that computers can't, are the things we don't have algorithms for What are some examples? GPT-3 is already composing halfway-credible poetry and songs. (Edit: and it means nothing to say that "poetry and songs exist in its training data." The songs and poems it just gave me didn't.) I just asked it to write some code to walk a k-d tree, and it did that, too, complete with a unit test. Don't focus on the current state of the art; pay attention to its time derivative. That's all that matters. When the tech gets 10x better, it will be almost indistinguishable from human thought. Another 10x will tie up the remaining loose ends.
- wizzwizz4 4y ago
  > GPT-3 is already composing halfway-credible poetry and songs. Poetry and songs abound in its training data. > I just asked it to write some code to walk a k-d tree, and it did that, too, complete with a unit test. Such things abound in its training data. GPT-3 can't produce anything novel. It's a search engine over natural generalisations of its training data, but it can't think for itself. It couldn't solve any of the unsolved problems of our day. Give it a maths problem with a known solution, and it might give you a half-way decent proof. Give it a problem without a known solution – even an easier one – and it'll flounder.
  - _dain_ 4y ago
    >Poetry and songs abound in its training data. your training data is all the poetry and songs you've seen in your life. so go on, write me a poem right now. a good one.
  - _Algernon_ 4y ago
    99.99% of what humans do, if not more, is not original work by your definition. Replacing those 99.99% is on its own already revolutionary.
- danaris 4y ago
  It absolutely does not mean nothing to say that poetry and songs exist in its training data. The criticism is not "it's just spitting out existing poems and songs"; it's "all it knows how to do is mimic (to greater or lesser degrees of accuracy) things in its training data." GPT-3 takes a massive corpus of text in, and then spits out something that it creates by munging that together in probabilistic ways, assisted (if I understand correctly) by various humans providing feedback on that. It has no comprehension of what it is doing. It has no "mental model" of the world. It has no "consciousness" of any kind, at any level. Because of this, it cannot genuinely imagine, synthesize, or be creative. Sure; it can put together sequences of words that had never been put together in exactly that configuration before, but that's not really the same thing as "creating something new." It can be entertaining, sure. It can even inspire insights in us—but it can never be genuinely insightful itself, because that requires understanding.
  - mavhc 4y ago
    But are humans any different? We gather inputs, mix them together, produce outputs. The joke on humans is such a simple program as GPT3 can emulate humans so well. Is GPT3 already smarter than all non human animals?
    - danaris 4y ago
      Before any stories had ever been told, humans thought of them, and told them to each other. Before any paintings had ever been painted, humans imagined them from nothing, and painted what they imagined. Just because reading other stories increases our breadth of material to draw from doesn't mean we require them to create. GPT-3 has no ability to create something truly new, without reference to anything it's seen before. Period. It cannot imagine. It does not have consciousness. The hype around it has tricked you into thinking it is much more than it is—either that, or that we are much less.
      - mavhc 4y ago
        Humans had eyes, and then recreated what they saw. Humans had memories, and retold what had happened. Rearranging previous things into new combinations is creation

CamperBob2 4y ago

The stuff humans can do that computers can't, are the things we don't have algorithms for What are some examples? GPT-3 is already composing halfway-credible poetry and songs. (Edit: and it means nothing to say that "poetry and songs exist in its training data." The songs and poems it just gave me didn't.) I just asked it to write some code to walk a k-d tree, and it did that, too, complete with a unit test. Don't focus on the current state of the art; pay attention to its time derivative. That's all that matters. When the tech gets 10x better, it will be almost indistinguishable from human thought. Another 10x will tie up the remaining loose ends.

wizzwizz4 4y ago
> GPT-3 is already composing halfway-credible poetry and songs. Poetry and songs abound in its training data. > I just asked it to write some code to walk a k-d tree, and it did that, too, complete with a unit test. Such things abound in its training data. GPT-3 can't produce anything novel. It's a search engine over natural generalisations of its training data, but it can't think for itself. It couldn't solve any of the unsolved problems of our day. Give it a maths problem with a known solution, and it might give you a half-way decent proof. Give it a problem without a known solution – even an easier one – and it'll flounder.
- _dain_ 4y ago
  >Poetry and songs abound in its training data. your training data is all the poetry and songs you've seen in your life. so go on, write me a poem right now. a good one.
- _Algernon_ 4y ago
  99.99% of what humans do, if not more, is not original work by your definition. Replacing those 99.99% is on its own already revolutionary.
danaris 4y ago
It absolutely does not mean nothing to say that poetry and songs exist in its training data. The criticism is not "it's just spitting out existing poems and songs"; it's "all it knows how to do is mimic (to greater or lesser degrees of accuracy) things in its training data." GPT-3 takes a massive corpus of text in, and then spits out something that it creates by munging that together in probabilistic ways, assisted (if I understand correctly) by various humans providing feedback on that. It has no comprehension of what it is doing. It has no "mental model" of the world. It has no "consciousness" of any kind, at any level. Because of this, it cannot genuinely imagine, synthesize, or be creative. Sure; it can put together sequences of words that had never been put together in exactly that configuration before, but that's not really the same thing as "creating something new." It can be entertaining, sure. It can even inspire insights in us—but it can never be genuinely insightful itself, because that requires understanding.
- mavhc 4y ago
  But are humans any different? We gather inputs, mix them together, produce outputs. The joke on humans is such a simple program as GPT3 can emulate humans so well. Is GPT3 already smarter than all non human animals?
  - danaris 4y ago
    Before any stories had ever been told, humans thought of them, and told them to each other. Before any paintings had ever been painted, humans imagined them from nothing, and painted what they imagined. Just because reading other stories increases our breadth of material to draw from doesn't mean we require them to create. GPT-3 has no ability to create something truly new, without reference to anything it's seen before. Period. It cannot imagine. It does not have consciousness. The hype around it has tricked you into thinking it is much more than it is—either that, or that we are much less.
    - mavhc 4y ago
      Humans had eyes, and then recreated what they saw. Humans had memories, and retold what had happened. Rearranging previous things into new combinations is creation