6 ms·

It's a weird thing to somehow think that an AI with only 4 days of training data should somehow outperform even a human child, which has a decade or more of imi

It's a weird thing to somehow think that an AI with only 4 days of training data should somehow outperform even a human child, which has a decade or more of imitation learning experience. A human doesn't come to minecraft "knowing nothing", like the machine learning programs in this challenge. They already know basically everything there is to know in order to perform task-oriented goal planning and only need to focus on task discovery. If anyone had pass the bar this challenge set, they'd have basically solved the biggest hurdle in general AI, and would probably have received a call from John Carmack.

YeGoblynQueenne 7y ago
That's kind of the point. The trend in machine learning right now is to try and solve problems "from scratch" by training "end-to-end", which means without any prior knowledge. For example, DeepMind's last three or so iterations of their Alpha-X architecture for game playing did not even have any prior knowledge of the rules of the games they tackled (chess, shoggi, go and Atari games) and were trained entirely through self-play and without anything like an opening book of moves etc, like the origional AlphaGo architecture (or DeepBlue and traditional chess engines). That is very big in AI right now, to try and remove the need for the "human in the loop" and learn complex tasks from 0 background knowledge. Obviously, like you point out, this is very difficult to do and it requires lots and lots of data and compute (no, more than that. Really lots). As the article reports, this eliminates any hopes of "democratising" machine learning and makes advancing the state of the art a game that only large corporations can play with any hope of winning.
- raverbashing 7y ago
  But there is a lot of "common knowledge" that's useful All the problems solved by fine-tuning image net for example, instead of training from zero Identifying objects in a 3D environment is useful across several domains and it shouldn't be limited to Minecraft
- imtringued 7y ago
  >The trend in machine learning right now is to try and solve problems "from scratch" by training "end-to-end", which means without any prior knowledge. That's not a very useful goal. I can trivially make a game that is impossible for such an AI to win. It would require you to enter a 1024 bit code [0] and if you are correct you win. For a speed runner the time it takes to beat the game is merely limited by their typing speed because the code is always the same but an AI with no prior knowledge can only brute force the solution. How does the speed runner win in the first place? They ask the game developer and then post the code on a wiki. [0] an euphemism for minecrafts crafting system
  - Jasper_ 7y ago
    What if you had 22 hours of footage showing speedrunners entering the 1024 bit code to train on?
    - chmod775 7y ago
      It would be possible to do that, but it's not really what was asked. The way you'd train a neural network with that footage is comparing the recording of the neural network playing the game to the recording of the speedrunner, and scoring it based on the difference in the footage. However it wouldn't really be learning "from scratch" and "end-to-end" anymore, since you're essentially teaching it to imitate and play like a human player. Also your network would likely be just blindly executing a series of learned actions, without any real understanding of the game. If you confronted it from some situation that's not part of some speedrun it memorized, it would probably be at a loss. Pretty useless.
      - pishpash 7y ago
        How about it watches how the human obtains the secret number (1024) in this game, then next time extrapolates the process to a different game with a different secret number? That is the kind of learning that humans do and AI's can conceivably do the same.
        7y ago
- semi-extrinsic 7y ago
  > The trend in machine learning right now is to try and solve problems "from scratch" by training "end-to-end", which means without any prior knowledge. In a strong sense, this implies ML is not "intelligent" in any conventional sense of the word. What little we have of understanding of learning and intelligence in the scope of humans strongly implies that drawing parallels with solutions to other tasks, building on previous knowledge, is the key to intelligence. Perhaps the purest expression of this is in pedagogy of math. Piaget and Brissiaud are some of the scholars who I think really grokked this. They brought forward the concept of compression as essential for learning. E.g. if you have learned but not yet compressed the operation of addition, you will have a very hard time learning multiplication. After addition has been compressed such that you don't have to spend any effort doing it, you are ready to learn multiplication. Maybe it would be worthwile to explore if there are AI techniques that can learn simple tasks first, then use that knowledge to solve harder tasks?
  - icebraining 7y ago
    But they do build on previous knowledge (that's the training part), it's just the such knowledge is built from scratch, rather than taught by others.
    - joe_the_user 7y ago
      A human can learn something through a fairly small amount of specific knowledge via written, verbal, etc instruction. All the data and the training process together are very clunky in comparison. IE, an inability to be taught by others is a serious limitation to say the least.
      - ben_w 7y ago
        Perhaps we can learn from small data, perhaps that’s an illusion. How much of our mind is hard-wired by our genes, our genes being a very slow-learning form of intelligence? Or, when we see “one” example of a new animal (for example), do we see a single picture? Or do we see it moving for about a second? Effectively multiple images? And, having previously learned a general model for physics, stereoscopic vision, and lighting, does that one second of motion gives us the size, shape, and a guess at the musculature of the animal? If you strip those advantages, how many training examples do we need? The examples of people blind from birth or from extreme youth who regain vision in adulthood[0] implies that we learn a lot in our youth about how to see, which we can share between recognition tasks we learn later. Keeping those basic skills — avoiding “catastrophic forgetting” — still seems to be a rare and advanced feature in AI, but it’s not impossible. There is still a lot we don’t know about how our own minds work. We might be closer than we realise, or much further. [0] https://en.m.wikipedia.org/wiki/Mike_May_(skier) https://en.m.wikipedia.org/wiki/Mike_May_(skier) amongst others. I never did find the name of the guy who learned to see from touching a monkey statue…
elif 7y ago
60 models trained on one dataset as well.
allovernow 7y ago
>It's a weird thing to somehow think that an AI with only 4 days of training data should somehow outperform even a human child, which has a decade or more of imitation learning experience. Well...not so much if you consider that machines are orders of magnitude faster than humans in any other task, including processing and recording data, a.k.a. learning.
- TheRealPomax 7y ago
  Not really, no. Procesing and recording data is not "learning". Machines are incredibly much faster at _classifying_ but that's been our problem for decades now: we figured out how to make insanely efficient and unreasonably good classifiers. However, we haven't been really able to turn that into efficient "learning", the best we've been able to do is what Deep Mind does, which requires very power machines working with literally decades worth of data.
codegladiator 7y ago
> with only 4 days of training data so 4 days to train their AI and not 4 days of training data.. on the data which was "60 million frames of recorded human player data"
- TheRealPomax 7y ago
  My bad: they got 2 weeks of footage. Other than a clerical error, of course, this changes nothing about the fact that that's nowhere near enough data unless you've already cracked general AI.
gcpwnd 7y ago
Now we just deny the prophecies of the upcoming singularity?
ars 7y ago
> with only 4 days of training data You misread it. It's around 300 hours of training data. It's 4 days to absorb those 300 hours. I bet a human child who has never even seen a computer could learn to do this in far less than 300 hours. I suspect even 1 hour would be enough.

YeGoblynQueenne 7y ago

That's kind of the point. The trend in machine learning right now is to try and solve problems "from scratch" by training "end-to-end", which means without any prior knowledge. For example, DeepMind's last three or so iterations of their Alpha-X architecture for game playing did not even have any prior knowledge of the rules of the games they tackled (chess, shoggi, go and Atari games) and were trained entirely through self-play and without anything like an opening book of moves etc, like the origional AlphaGo architecture (or DeepBlue and traditional chess engines). That is very big in AI right now, to try and remove the need for the "human in the loop" and learn complex tasks from 0 background knowledge. Obviously, like you point out, this is very difficult to do and it requires lots and lots of data and compute (no, more than that. Really lots). As the article reports, this eliminates any hopes of "democratising" machine learning and makes advancing the state of the art a game that only large corporations can play with any hope of winning.

raverbashing 7y ago
But there is a lot of "common knowledge" that's useful All the problems solved by fine-tuning image net for example, instead of training from zero Identifying objects in a 3D environment is useful across several domains and it shouldn't be limited to Minecraft
imtringued 7y ago
>The trend in machine learning right now is to try and solve problems "from scratch" by training "end-to-end", which means without any prior knowledge. That's not a very useful goal. I can trivially make a game that is impossible for such an AI to win. It would require you to enter a 1024 bit code [0] and if you are correct you win. For a speed runner the time it takes to beat the game is merely limited by their typing speed because the code is always the same but an AI with no prior knowledge can only brute force the solution. How does the speed runner win in the first place? They ask the game developer and then post the code on a wiki. [0] an euphemism for minecrafts crafting system
- Jasper_ 7y ago
  What if you had 22 hours of footage showing speedrunners entering the 1024 bit code to train on?
  - chmod775 7y ago
    It would be possible to do that, but it's not really what was asked. The way you'd train a neural network with that footage is comparing the recording of the neural network playing the game to the recording of the speedrunner, and scoring it based on the difference in the footage. However it wouldn't really be learning "from scratch" and "end-to-end" anymore, since you're essentially teaching it to imitate and play like a human player. Also your network would likely be just blindly executing a series of learned actions, without any real understanding of the game. If you confronted it from some situation that's not part of some speedrun it memorized, it would probably be at a loss. Pretty useless.
    - pishpash 7y ago
      How about it watches how the human obtains the secret number (1024) in this game, then next time extrapolates the process to a different game with a different secret number? That is the kind of learning that humans do and AI's can conceivably do the same.
      - 7y ago
semi-extrinsic 7y ago
> The trend in machine learning right now is to try and solve problems "from scratch" by training "end-to-end", which means without any prior knowledge. In a strong sense, this implies ML is not "intelligent" in any conventional sense of the word. What little we have of understanding of learning and intelligence in the scope of humans strongly implies that drawing parallels with solutions to other tasks, building on previous knowledge, is the key to intelligence. Perhaps the purest expression of this is in pedagogy of math. Piaget and Brissiaud are some of the scholars who I think really grokked this. They brought forward the concept of compression as essential for learning. E.g. if you have learned but not yet compressed the operation of addition, you will have a very hard time learning multiplication. After addition has been compressed such that you don't have to spend any effort doing it, you are ready to learn multiplication. Maybe it would be worthwile to explore if there are AI techniques that can learn simple tasks first, then use that knowledge to solve harder tasks?
- icebraining 7y ago
  But they do build on previous knowledge (that's the training part), it's just the such knowledge is built from scratch, rather than taught by others.
  - joe_the_user 7y ago
    A human can learn something through a fairly small amount of specific knowledge via written, verbal, etc instruction. All the data and the training process together are very clunky in comparison. IE, an inability to be taught by others is a serious limitation to say the least.
    - ben_w 7y ago
      Perhaps we can learn from small data, perhaps that’s an illusion. How much of our mind is hard-wired by our genes, our genes being a very slow-learning form of intelligence? Or, when we see “one” example of a new animal (for example), do we see a single picture? Or do we see it moving for about a second? Effectively multiple images? And, having previously learned a general model for physics, stereoscopic vision, and lighting, does that one second of motion gives us the size, shape, and a guess at the musculature of the animal? If you strip those advantages, how many training examples do we need? The examples of people blind from birth or from extreme youth who regain vision in adulthood[0] implies that we learn a lot in our youth about how to see, which we can share between recognition tasks we learn later. Keeping those basic skills — avoiding “catastrophic forgetting” — still seems to be a rare and advanced feature in AI, but it’s not impossible. There is still a lot we don’t know about how our own minds work. We might be closer than we realise, or much further. [0] https://en.m.wikipedia.org/wiki/Mike_May_(skier) https://en.m.wikipedia.org/wiki/Mike_May_(skier) amongst others. I never did find the name of the guy who learned to see from touching a monkey statue…

elif 7y ago

60 models trained on one dataset as well.

allovernow 7y ago

>It's a weird thing to somehow think that an AI with only 4 days of training data should somehow outperform even a human child, which has a decade or more of imitation learning experience. Well...not so much if you consider that machines are orders of magnitude faster than humans in any other task, including processing and recording data, a.k.a. learning.

TheRealPomax 7y ago
Not really, no. Procesing and recording data is not "learning". Machines are incredibly much faster at _classifying_ but that's been our problem for decades now: we figured out how to make insanely efficient and unreasonably good classifiers. However, we haven't been really able to turn that into efficient "learning", the best we've been able to do is what Deep Mind does, which requires very power machines working with literally decades worth of data.

codegladiator 7y ago

> with only 4 days of training data so 4 days to train their AI and not 4 days of training data.. on the data which was "60 million frames of recorded human player data"

TheRealPomax 7y ago
My bad: they got 2 weeks of footage. Other than a clerical error, of course, this changes nothing about the fact that that's nowhere near enough data unless you've already cracked general AI.

gcpwnd 7y ago

Now we just deny the prophecies of the upcoming singularity?

ars 7y ago

> with only 4 days of training data You misread it. It's around 300 hours of training data. It's 4 days to absorb those 300 hours. I bet a human child who has never even seen a computer could learn to do this in far less than 300 hours. I suspect even 1 hour would be enough.