4 ms·

This[1] paper does an excellent job gradually removing cultural priors from a simple videogame, and measuring the extent to which human and computer performance

This[1] paper does an excellent job gradually removing cultural priors from a simple videogame, and measuring the extent to which human and computer performance degrade. Their conclusion is that human ability to learn games fast depends on our ability to map them to real world concepts. Would it be permitted for entrants in your contest to pretrain on some other non-videogame task? [1] https://arxiv.org/pdf/1802.10217.pdf https://arxiv.org/pdf/1802.10217.pdf

chongli 7y ago
I didn’t state it very clearly but the whole point of my test is the ability to translate cultural priors into a new medium. Here’s another task I’d like to see: have a kid who has never seen a film before read a children’s book and then watch the film adaptation. Test the child on their ability to identify all of the characters in the film. This is the sort of task any kid could do with high accuracy but an AI would struggle with unless it were specifically trained on a corpus of books and their film adaptations.
- meowface 7y ago
  But like the parent said, you still have to make it fair. The child may have never seen a film before, but they've probably read other books, or have had books read to them, or have had stories told to them (real or fictional). And they've probably seen TV shows or videos of some sort. And they've probably had an upbringing with lots of adults and kids speaking to and around them about topics, and have had (rudimentary) experience telling stories themselves, like what they did today. All of this helps them form a model of the world that makes understanding a book or a film, and the relation between them, easier. So maybe training on a corpus of books and film adaptations is cheating in this scenario, but the AI should still be allowed to train on a massive corpus of all of the things a child may be exposed to. Maybe instead of showing it films, show it thousands of hours of footage from head-mounted cameras that people have worn while going through daily life, and children's TV shows, books, and games, and have it talk to a bunch of humans.
  - irio9kfkf 7y ago
    Here’s my take on why it’s not intelligence: The hardware has no capacity to independently observe itself or the conditions that give rise to its existence, and given its observations, improvise as needed. Everything we would label without question as possessing intelligence has an instinctual drive and mechanical ability to improve itself, to beat itself at its own games. To move out of the way of danger if it has the opportunity to see it coming. To improve the probability of its survival. There’s more to intelligence than building a machine of a predetermined spec to mechanically sort a predetermined set of data. These notions in comp sci seem to completely ignore physical reality. I can conceive of a planet called Krypton, can it exist? I can conceive things like tautology but can physical reality provide an example? I can conceive of a phone app giving me a sense it has intelligence but it doesn’t. It’s not self aware and cannot self replicate. It can’t save itself or beat the odds on its own. This physical reality we occupy invented what we’d define as intelligence. Our definition ought to very specifically include physical capabilities of independent not programmed self awareness & capacity for self care. AFAIK, that’s what our basis for the concept of intelligence is based on. Why should we accept an artificial intelligence that’s less fully capable of that as somehow on par or better than is given its obvious limitations?
    - meowface 7y ago
      I agree. But I think it's a long and gradual path to eventually reach that stage. This would just be one of the many stepping stones before true intelligence may be within reach.

chongli 7y ago

I didn’t state it very clearly but the whole point of my test is the ability to translate cultural priors into a new medium. Here’s another task I’d like to see: have a kid who has never seen a film before read a children’s book and then watch the film adaptation. Test the child on their ability to identify all of the characters in the film. This is the sort of task any kid could do with high accuracy but an AI would struggle with unless it were specifically trained on a corpus of books and their film adaptations.

meowface 7y ago
But like the parent said, you still have to make it fair. The child may have never seen a film before, but they've probably read other books, or have had books read to them, or have had stories told to them (real or fictional). And they've probably seen TV shows or videos of some sort. And they've probably had an upbringing with lots of adults and kids speaking to and around them about topics, and have had (rudimentary) experience telling stories themselves, like what they did today. All of this helps them form a model of the world that makes understanding a book or a film, and the relation between them, easier. So maybe training on a corpus of books and film adaptations is cheating in this scenario, but the AI should still be allowed to train on a massive corpus of all of the things a child may be exposed to. Maybe instead of showing it films, show it thousands of hours of footage from head-mounted cameras that people have worn while going through daily life, and children's TV shows, books, and games, and have it talk to a bunch of humans.
- irio9kfkf 7y ago
  Here’s my take on why it’s not intelligence: The hardware has no capacity to independently observe itself or the conditions that give rise to its existence, and given its observations, improvise as needed. Everything we would label without question as possessing intelligence has an instinctual drive and mechanical ability to improve itself, to beat itself at its own games. To move out of the way of danger if it has the opportunity to see it coming. To improve the probability of its survival. There’s more to intelligence than building a machine of a predetermined spec to mechanically sort a predetermined set of data. These notions in comp sci seem to completely ignore physical reality. I can conceive of a planet called Krypton, can it exist? I can conceive things like tautology but can physical reality provide an example? I can conceive of a phone app giving me a sense it has intelligence but it doesn’t. It’s not self aware and cannot self replicate. It can’t save itself or beat the odds on its own. This physical reality we occupy invented what we’d define as intelligence. Our definition ought to very specifically include physical capabilities of independent not programmed self awareness & capacity for self care. AFAIK, that’s what our basis for the concept of intelligence is based on. Why should we accept an artificial intelligence that’s less fully capable of that as somehow on par or better than is given its obvious limitations?
  - meowface 7y ago
    I agree. But I think it's a long and gradual path to eventually reach that stage. This would just be one of the many stepping stones before true intelligence may be within reach.