3 ms·

> This kind of facile moving the goalposts is imho a cheap shot and (not imho, fact) is a recurring phenomenon as we make incremental progress toward AI. I thi

> This kind of facile moving the goalposts is imho a cheap shot and (not imho, fact) is a recurring phenomenon as we make incremental progress toward AI. I think you missed my point. I think we're going in the wrong direction for AI entirely, and these "advances" are fundamentally misguided. OpenAI is explicitly about "intelligence," and so we should question if this is in fact that. It's clear that humans have fundamental intelligence much better than all of this stuff with 6 orders of magnitude less input (at least of the same data sort) on a problem. Perhaps it would be better to say, "I think the ML winter is just around the corner" as opposed to "the AI winter is just around the corner." That said, this really is math, and these algos still don't actually do anything resembling true intelligence.

canjobear 6y ago
> think we're going in the wrong direction for AI entirely This direction has produced results that eluded 30+ years of research. What is the evidence that this is the wrong direction?
a9h74j 6y ago
> 6 orders of magnitude less input IIRC the following is attributable to either Margaret Atwood or Iris Murdoch: "A writer should be able to look into a room [full of people] and understand [in breadth] what is going on."
natch 6y ago
But we’re not done yet. Give it time. We can go in plenty of directions. Just because you don’t think the current direction is right, that doesn’t rule out other directions happening. And who’s to say this stuff won’t end up being somehow useful? There’s a great talk on how trying to make progress toward an objective in evolutionary algorithms is not a good way to get there. https://www.youtube.com/watch?v=dXQPL9GooyI https://www.youtube.com/watch?v=dXQPL9GooyI Of course evolutionary algorithms are just one direction as well. But that doesn’t mean that nothing else is happening.
natch 6y ago
It’s actually about AI which is distinct from intelligence. >6 orders of magnitude less input That is utterly mistaken. We have the input of millions of generations of evolution which have shaped our brains and given us a lot of instinctive knowledge that we do not need to learn from environmental input that happens during our lifetime. Instead it was learned over the course of billions of years, during the lifetimes of other organisms that preceded us. Our brain structure was developed and tuned by all these inputs to have some built in pretrained models. That’s what instincts are. Billions of years in the making. Millions, at the very least, if you want to restrict it to recent primates, although doing so is nonsensical.
- abernard1 6y ago
  >>6 orders of magnitude less input >That is utterly mistaken. I did say of data of the "same sort". What's absolutely crazy is somehow we think of our DNA base pairs as somehow more important than the physical context that DNA ends up in (society, humans, talking, etc.) We have the ability to be intelligent and make thoughts with 1 millionth the amount of textual data as this OpenAI GPT-3 study. Maybe... just maybe... intelligence is far more related to things other than just having more data. I'll actually expand on this and throw this out there: intelligence is in a way antagonistic to more data. A more intelligent agent needs less knowledge to make a better decision. It's like a function that can do the same computation with fewer inputs. A less intelligent agent requires a lookup table of previously computed intelligent things instead of figuring it out on its own. I think all these "AI" studies are glorified lookup tables.
  - MauranKilom 6y ago
    Throw some novel text prompt/task at it and see what happens. If it was just "glorified lookup tables" then the result should be consistently garbage. Note in particular that "like a function that can do the same computation with fewer inputs" maps very well to GPT-3 - it can complete many interesting tasks by just having a few samples provided to it, instead of having to fine-tune it with more training.
    - 6y ago

canjobear 6y ago

> think we're going in the wrong direction for AI entirely This direction has produced results that eluded 30+ years of research. What is the evidence that this is the wrong direction?

a9h74j 6y ago

> 6 orders of magnitude less input IIRC the following is attributable to either Margaret Atwood or Iris Murdoch: "A writer should be able to look into a room [full of people] and understand [in breadth] what is going on."

natch 6y ago

But we’re not done yet. Give it time. We can go in plenty of directions. Just because you don’t think the current direction is right, that doesn’t rule out other directions happening. And who’s to say this stuff won’t end up being somehow useful? There’s a great talk on how trying to make progress toward an objective in evolutionary algorithms is not a good way to get there. https://www.youtube.com/watch?v=dXQPL9GooyI https://www.youtube.com/watch?v=dXQPL9GooyI Of course evolutionary algorithms are just one direction as well. But that doesn’t mean that nothing else is happening.

natch 6y ago

It’s actually about AI which is distinct from intelligence. >6 orders of magnitude less input That is utterly mistaken. We have the input of millions of generations of evolution which have shaped our brains and given us a lot of instinctive knowledge that we do not need to learn from environmental input that happens during our lifetime. Instead it was learned over the course of billions of years, during the lifetimes of other organisms that preceded us. Our brain structure was developed and tuned by all these inputs to have some built in pretrained models. That’s what instincts are. Billions of years in the making. Millions, at the very least, if you want to restrict it to recent primates, although doing so is nonsensical.

abernard1 6y ago
>>6 orders of magnitude less input >That is utterly mistaken. I did say of data of the "same sort". What's absolutely crazy is somehow we think of our DNA base pairs as somehow more important than the physical context that DNA ends up in (society, humans, talking, etc.) We have the ability to be intelligent and make thoughts with 1 millionth the amount of textual data as this OpenAI GPT-3 study. Maybe... just maybe... intelligence is far more related to things other than just having more data. I'll actually expand on this and throw this out there: intelligence is in a way antagonistic to more data. A more intelligent agent needs less knowledge to make a better decision. It's like a function that can do the same computation with fewer inputs. A less intelligent agent requires a lookup table of previously computed intelligent things instead of figuring it out on its own. I think all these "AI" studies are glorified lookup tables.
- MauranKilom 6y ago
  Throw some novel text prompt/task at it and see what happens. If it was just "glorified lookup tables" then the result should be consistently garbage. Note in particular that "like a function that can do the same computation with fewer inputs" maps very well to GPT-3 - it can complete many interesting tasks by just having a few samples provided to it, instead of having to fine-tune it with more training.
  - 6y ago