5 ms·

The results are obviously impressive, but even then there is a lot of work to do as far as learning efficiency goes: "The AlphaStar league was run for 14 days,

The results are obviously impressive, but even then there is a lot of work to do as far as learning efficiency goes: "The AlphaStar league was run for 14 days, using 16 TPUs for each agent. During training, each agent experienced up to 200 years of real-time StarCraft play. " MaNa probably played less than 2-3 years of Starcraft in his whole life (by that I mean 24hr x 365d x 3), and was learning with a much less focused/rigorous methodology.

derefr 8y ago
Another way to think about it is that a human brain is mostly doing transfer-learning, on top of a 99%-baked deep net that was wired up during foetal development from our DNA, where that DNA-persisted model has "seen" hundreds of millions of years of training data. Humans don't have to learn to process, recognize, and classify objects in visual sense-data, for example. We can do that from the moment we're born, because we already have hundreds of precisely-tuned "layers" laying around in our brains for doing just that. We just need to transfer-learn the relevant classes.
- dcl 8y ago
  This is a widely underappreciated fact when it comes to comes to comparing the 'training experience' of humans versus bots. And it extends far beyond processing 'sense data' - A human likely has some level of understanding of how the game works based on experience from other games it has played and from 'real life' - we know almost instinctively that 'high ground' is likely to give a combat advantage without having test it in game.
  - derefr 8y ago
    Not only that, humans (and many other eusocial species) have an instinctual intuitional understanding of many aspects of game theory. For example, humans, even from infancy, prefer games where it is possible to punish cheating (i.e. take revenge upon cheaters) to games where it is not. This isn't just "we're animals that have evolved to enact tit-for-tat strategies [by e.g. injustice triggering rage] because they lead to cooperation which leads to egalitarian utility"; this is actual analysis—instantaneous, intuitive analysis—of a system of rules, to notice, in advance of ever being slighted, whether you'll be likely to end up in an "unjust" social situation if you agree to the given ruleset. There is an "accelerated co-processor" of high-level abstract game-theoretic information—and layers to extract that information from sense-data—that ship as part-and-parcel of the human brain model. We never need to learn how to judge unfairness, any more than we need to learn how to see.
    - cjbprime 8y ago
      And perhaps worth noting that the great apes we evolved alongside have the same kind of outrage to unfair trades.
    - gdy 8y ago
      "humans, even from infancy, prefer games where it is possible to punish cheating...this is actual analysis—instantaneous, intuitive analysis—of a system of rules, to notice, in advance of ever being slighted" [Citation needed]
  - saberience 8y ago
    All of our knowledge of how to play games and so on has come from our current lifetime. We do not have a "genetic memory" that means we have learnings from cavemen or some other such nonsense. Our DNA contains instructions on how to grow a human, it's not a mega hard drive with millions of years of collective memory. If a 19 year old is good at Starcraft, he's good at Starcraft because he spent two or three years playing a shit load of Starcraft and we are much more efficient at learning higher level strategies than AI are. These AI agents nead to try damn near every possibility to adjust their weightings for various actions. Humans understand pretty much the first time when something goes wrong, oh better not do that OR similar things again. It's incredibly impressive that a given human can become GM level at Starcraft within a few years and to take an AI to that level takes 200 years of training, as well as an inhuman reaction time, perfect micro/clicking, etc. It shows how amazing our learning skills are.
    - taneq 8y ago
      We may not have "genetic memory" but a ton of human capabilities are baked in at the DNA level. Sure, we need to practice in order to specialise those abilities for particular tasks, but that's more of a calibration phase on a fantastically capable machine, rather than a construction phase. Totally agree with how impressive humans are, though. In fact, one of the most amazing things to me about robotics is finding out how close to global optimal some humans can actually get.
      - deleted 8y ago
        [deleted]
    - posterboy 8y ago
      The GP is underselling the fact that in the human years of being a pro player they think through many more games and may even dream of it. I certainly went to bed after a lengthy session with images of the game still in front of me. Although that might be more about micro, the macro skills are somewhat transferable from other "games". RTS simulate economy, amongst other things, after all. GP's claim, "99%-baked deep net that was wired up during foetal development from our DNA" is also unfounded, if not completely overblown. I am far from a student of biology, much less an expert, but intelligence is still seen as an emergent property. The real kicker might be that organizing thoughts might be a "game" of it self, that is learned in development and constantly exercised. Talk about self-play. I recently read a similar question about "inherent mathematical language", ie. capability, and the given opinion was that there is no consensus, except perhaps for basic addition, which I guess concerns vision, ie. seeing a set of things and knowing the count is +++++. That works only up to around +++++++ items at best, according to findings.
- dooglius 8y ago
  Is there evidence for this? My mental model has been that DNA encodes more along the lines of hyperparameters: amount of gray matter vs white matter, locations of brain regions and folds, etc, but the connections between neurons, and their weights, were all learned. There isn't that much information you can stuff into DNA, after all.
  - YinLuck- 8y ago
    Connections between neurons, the synapses, are encoded. So much so that they are given individual names. This is a fun one to read about to get an idea: https://en.wikipedia.org/wiki/Calyx_of_Held https://en.wikipedia.org/wiki/Calyx_of_Held
- asteli 8y ago
  Perhaps a nit, but still fascinating: the human visual cortex finishes developing after birth. A newborn can't really distinguish between objects. The ability to differentiate, focus on and track objects is developed over the course of several months.
  - derefr 8y ago
    True. Humans are pretty unique in that regard, though; pretty much no other animal is like that. It's easier to understand human neonatal development if you just considering all humans to be born premature. (It'd be really interesting to know whether that's literally true—whether keeping a human baby in the womb for an extra few months would actually result in the same stages of mental development being passed that occur in a regular baby of that age who has been sensing and interacting with the world.)
    - Ar-Curunir 8y ago
      I've read somewhere that we are basically born prematurely (as you said) because if we waited any longer then our enlarged head sizes would make delivery quite possibly fatal.
      - intuitionist 8y ago
        My brother was born a week or so after his due date; they induced labor for him for exactly this reason. Perhaps unsurprisingly, his head circumference was literally off the charts.
      - posterboy 8y ago
        Maybe off-topic, but that's one side of the coin, and I suppose the other is that being exposed to more sensory input accelerates development, or makes it even possible (on higher levels of cognition). If this wasn't the case, why wouldn't we just be bigger and carry longer? Is size viz megafauna really that suboptimal for any more significant reasons than being hunted human hunters? I would almost say that longer pre-natal development was suboptimal, because we'd either become bored, or supersmart, but anyhow superegoistic for lack of nurture. Calling it premature is ironic, if we reach nominal maturity only after 10 or more years as far as fertility is concerned--the equivalent in AI would be the procreation of a neural net, perhaps after exploiting a bug in the game, breaking out to rewrite a better version of itself, or colluding with itself in self play. Yes, this is going off-topic.
- saberience 8y ago
  That's not how any of this works. We do not have "millions of years" of information encoded into DNA. DNA doesn't store that much data. In fact, it's about 1.6 gigabytes only! And most of that information is basically a ruleset for growing proteins which become our body. All the stuff we've learned about games and so on have come from our current lifetime. I don't have caveman memory for how to fight a tiger.
  - derefr 8y ago
    I said "deep net" for a reason. A DNN model almost always turns out to be far, far smaller than the training data that was used to create it. For one example: any smartphone's face-recognition feature. Each such feature is a DNN which took millions of hours of face data to train... but the resultant model fits on an ASIC. Our DNA doesn't directly encode such a model, but it encodes a particular morphogenic chemical gradient, and set of proteins, that go together to make specialized neural "organs" (like your substantia nigra, or your basal ganglia, or your superchiasmatic nucleus, etc.) which manage to serve the same function to your brain that access to a pre-trained "black box" DNN model would serve an untrained NN in achieving transfer learning.
    - saberience 8y ago
      Our DNA is NOT a trained deep net, nor is it a deep net period. Our DNA is a string of proteins which encode other proteins which gives the series of tasks needed to create and operate all the structures of the brain and body. The "training" of our deep net happens during our lifetime. We are not born with a trained deep net so your analogy that somehow we are born with a highly capable deep-net encoded into 1.6GB of DNA makes no sense. Can you imagine how capable a human being would be if it was born into a world with no other humans or learning sources? Imagine a new born baby born into a world with some accessible food/water close by so it wouldn't die from lack of nutrition or wild animals, but crucially without any other humans. It would be utterly fucking useless, no language/reading means no way of assimiliating new knowledge. That baby would end up being a totally incapable human, regardless of the DNA or structure of the brain. As far as we currently understand, if infants aren't exposed to language and communication at a very young age, they are either incapable or severely stunted in terms of communication for the rest of their life. My point is, that we are very much dependent on the learning that we get from the point of birth ONWARDS. We get the amazing capacity to learn from the structure of our brain and body, but we'd be absolutely incapable idiots without other people to teach us, our books, language etc. We understand "games" and game theory from playing games with other kids, we're not born with "game theory" encoded into our DNA as one other commenter seemed to think, the same for language learning, and everything else. Anyway, the point of this whole debate was that it's incredibly impressive that humans can learn to play a game as complex as SC2 in a tiny fraction of the time it takes a cluster of GPUs using a huge amount of energy and resources. Not forgetting that we also have to use a physical body to control our actions in the game, which adds a whole other level of complexity since we have to understand how to manipulate a mouse/keyboard etc, whereas the AI is essentially acting directly with the game, like a human with a neural link. The other kicker, is that if you just changed one aspect, like picking a new map neither player had seen, the AI would be sent hurtling back to square one whereas the human would only be partially affected. These series of demos only make me more impressed that given the huge resources given to Google, they can just about beat a human and even then after 200 years of training time and various other artificial advantages.
- lrem 8y ago
  Human genome isn't even a gigabyte of data. That's less than a byte per neuron and a big chunk of that data actually has to go into "how to make a kidney cell" and "which way to route veins". So while some basics have to be hard-coded, it can't be remotely close to "99% transfer from ancestors".
- erikpukinskis 8y ago
  > Humans don't have to learn to process, recognize, and classify objects in visual sense-data Do you have a citation for this? It doesn’t jibe with my understanding of development. For example, animals born paralyzed are blind: https://io9.gizmodo.com/the-seriously-creepy-two-kitten-experiment-1442107174 https://io9.gizmodo.com/the-seriously-creepy-two-kitten-expe...

derefr 8y ago

Another way to think about it is that a human brain is mostly doing transfer-learning, on top of a 99%-baked deep net that was wired up during foetal development from our DNA, where that DNA-persisted model has "seen" hundreds of millions of years of training data. Humans don't have to learn to process, recognize, and classify objects in visual sense-data, for example. We can do that from the moment we're born, because we already have hundreds of precisely-tuned "layers" laying around in our brains for doing just that. We just need to transfer-learn the relevant classes.

dcl 8y ago
This is a widely underappreciated fact when it comes to comes to comparing the 'training experience' of humans versus bots. And it extends far beyond processing 'sense data' - A human likely has some level of understanding of how the game works based on experience from other games it has played and from 'real life' - we know almost instinctively that 'high ground' is likely to give a combat advantage without having test it in game.
- derefr 8y ago
  Not only that, humans (and many other eusocial species) have an instinctual intuitional understanding of many aspects of game theory. For example, humans, even from infancy, prefer games where it is possible to punish cheating (i.e. take revenge upon cheaters) to games where it is not. This isn't just "we're animals that have evolved to enact tit-for-tat strategies [by e.g. injustice triggering rage] because they lead to cooperation which leads to egalitarian utility"; this is actual analysis—instantaneous, intuitive analysis—of a system of rules, to notice, in advance of ever being slighted, whether you'll be likely to end up in an "unjust" social situation if you agree to the given ruleset. There is an "accelerated co-processor" of high-level abstract game-theoretic information—and layers to extract that information from sense-data—that ship as part-and-parcel of the human brain model. We never need to learn how to judge unfairness, any more than we need to learn how to see.
  - cjbprime 8y ago
    And perhaps worth noting that the great apes we evolved alongside have the same kind of outrage to unfair trades.
  - gdy 8y ago
    "humans, even from infancy, prefer games where it is possible to punish cheating...this is actual analysis—instantaneous, intuitive analysis—of a system of rules, to notice, in advance of ever being slighted" [Citation needed]
- saberience 8y ago
  All of our knowledge of how to play games and so on has come from our current lifetime. We do not have a "genetic memory" that means we have learnings from cavemen or some other such nonsense. Our DNA contains instructions on how to grow a human, it's not a mega hard drive with millions of years of collective memory. If a 19 year old is good at Starcraft, he's good at Starcraft because he spent two or three years playing a shit load of Starcraft and we are much more efficient at learning higher level strategies than AI are. These AI agents nead to try damn near every possibility to adjust their weightings for various actions. Humans understand pretty much the first time when something goes wrong, oh better not do that OR similar things again. It's incredibly impressive that a given human can become GM level at Starcraft within a few years and to take an AI to that level takes 200 years of training, as well as an inhuman reaction time, perfect micro/clicking, etc. It shows how amazing our learning skills are.
  - taneq 8y ago
    We may not have "genetic memory" but a ton of human capabilities are baked in at the DNA level. Sure, we need to practice in order to specialise those abilities for particular tasks, but that's more of a calibration phase on a fantastically capable machine, rather than a construction phase. Totally agree with how impressive humans are, though. In fact, one of the most amazing things to me about robotics is finding out how close to global optimal some humans can actually get.
    - deleted 8y ago
      [deleted]
  - posterboy 8y ago
    The GP is underselling the fact that in the human years of being a pro player they think through many more games and may even dream of it. I certainly went to bed after a lengthy session with images of the game still in front of me. Although that might be more about micro, the macro skills are somewhat transferable from other "games". RTS simulate economy, amongst other things, after all. GP's claim, "99%-baked deep net that was wired up during foetal development from our DNA" is also unfounded, if not completely overblown. I am far from a student of biology, much less an expert, but intelligence is still seen as an emergent property. The real kicker might be that organizing thoughts might be a "game" of it self, that is learned in development and constantly exercised. Talk about self-play. I recently read a similar question about "inherent mathematical language", ie. capability, and the given opinion was that there is no consensus, except perhaps for basic addition, which I guess concerns vision, ie. seeing a set of things and knowing the count is +++++. That works only up to around +++++++ items at best, according to findings.
dooglius 8y ago
Is there evidence for this? My mental model has been that DNA encodes more along the lines of hyperparameters: amount of gray matter vs white matter, locations of brain regions and folds, etc, but the connections between neurons, and their weights, were all learned. There isn't that much information you can stuff into DNA, after all.
- YinLuck- 8y ago
  Connections between neurons, the synapses, are encoded. So much so that they are given individual names. This is a fun one to read about to get an idea: https://en.wikipedia.org/wiki/Calyx_of_Held https://en.wikipedia.org/wiki/Calyx_of_Held
asteli 8y ago
Perhaps a nit, but still fascinating: the human visual cortex finishes developing after birth. A newborn can't really distinguish between objects. The ability to differentiate, focus on and track objects is developed over the course of several months.
- derefr 8y ago
  True. Humans are pretty unique in that regard, though; pretty much no other animal is like that. It's easier to understand human neonatal development if you just considering all humans to be born premature. (It'd be really interesting to know whether that's literally true—whether keeping a human baby in the womb for an extra few months would actually result in the same stages of mental development being passed that occur in a regular baby of that age who has been sensing and interacting with the world.)
  - Ar-Curunir 8y ago
    I've read somewhere that we are basically born prematurely (as you said) because if we waited any longer then our enlarged head sizes would make delivery quite possibly fatal.
    - intuitionist 8y ago
      My brother was born a week or so after his due date; they induced labor for him for exactly this reason. Perhaps unsurprisingly, his head circumference was literally off the charts.
    - posterboy 8y ago
      Maybe off-topic, but that's one side of the coin, and I suppose the other is that being exposed to more sensory input accelerates development, or makes it even possible (on higher levels of cognition). If this wasn't the case, why wouldn't we just be bigger and carry longer? Is size viz megafauna really that suboptimal for any more significant reasons than being hunted human hunters? I would almost say that longer pre-natal development was suboptimal, because we'd either become bored, or supersmart, but anyhow superegoistic for lack of nurture. Calling it premature is ironic, if we reach nominal maturity only after 10 or more years as far as fertility is concerned--the equivalent in AI would be the procreation of a neural net, perhaps after exploiting a bug in the game, breaking out to rewrite a better version of itself, or colluding with itself in self play. Yes, this is going off-topic.
saberience 8y ago
That's not how any of this works. We do not have "millions of years" of information encoded into DNA. DNA doesn't store that much data. In fact, it's about 1.6 gigabytes only! And most of that information is basically a ruleset for growing proteins which become our body. All the stuff we've learned about games and so on have come from our current lifetime. I don't have caveman memory for how to fight a tiger.
- derefr 8y ago
  I said "deep net" for a reason. A DNN model almost always turns out to be far, far smaller than the training data that was used to create it. For one example: any smartphone's face-recognition feature. Each such feature is a DNN which took millions of hours of face data to train... but the resultant model fits on an ASIC. Our DNA doesn't directly encode such a model, but it encodes a particular morphogenic chemical gradient, and set of proteins, that go together to make specialized neural "organs" (like your substantia nigra, or your basal ganglia, or your superchiasmatic nucleus, etc.) which manage to serve the same function to your brain that access to a pre-trained "black box" DNN model would serve an untrained NN in achieving transfer learning.
  - saberience 8y ago
    Our DNA is NOT a trained deep net, nor is it a deep net period. Our DNA is a string of proteins which encode other proteins which gives the series of tasks needed to create and operate all the structures of the brain and body. The "training" of our deep net happens during our lifetime. We are not born with a trained deep net so your analogy that somehow we are born with a highly capable deep-net encoded into 1.6GB of DNA makes no sense. Can you imagine how capable a human being would be if it was born into a world with no other humans or learning sources? Imagine a new born baby born into a world with some accessible food/water close by so it wouldn't die from lack of nutrition or wild animals, but crucially without any other humans. It would be utterly fucking useless, no language/reading means no way of assimiliating new knowledge. That baby would end up being a totally incapable human, regardless of the DNA or structure of the brain. As far as we currently understand, if infants aren't exposed to language and communication at a very young age, they are either incapable or severely stunted in terms of communication for the rest of their life. My point is, that we are very much dependent on the learning that we get from the point of birth ONWARDS. We get the amazing capacity to learn from the structure of our brain and body, but we'd be absolutely incapable idiots without other people to teach us, our books, language etc. We understand "games" and game theory from playing games with other kids, we're not born with "game theory" encoded into our DNA as one other commenter seemed to think, the same for language learning, and everything else. Anyway, the point of this whole debate was that it's incredibly impressive that humans can learn to play a game as complex as SC2 in a tiny fraction of the time it takes a cluster of GPUs using a huge amount of energy and resources. Not forgetting that we also have to use a physical body to control our actions in the game, which adds a whole other level of complexity since we have to understand how to manipulate a mouse/keyboard etc, whereas the AI is essentially acting directly with the game, like a human with a neural link. The other kicker, is that if you just changed one aspect, like picking a new map neither player had seen, the AI would be sent hurtling back to square one whereas the human would only be partially affected. These series of demos only make me more impressed that given the huge resources given to Google, they can just about beat a human and even then after 200 years of training time and various other artificial advantages.
lrem 8y ago
Human genome isn't even a gigabyte of data. That's less than a byte per neuron and a big chunk of that data actually has to go into "how to make a kidney cell" and "which way to route veins". So while some basics have to be hard-coded, it can't be remotely close to "99% transfer from ancestors".
erikpukinskis 8y ago
> Humans don't have to learn to process, recognize, and classify objects in visual sense-data Do you have a citation for this? It doesn’t jibe with my understanding of development. For example, animals born paralyzed are blind: https://io9.gizmodo.com/the-seriously-creepy-two-kitten-experiment-1442107174 https://io9.gizmodo.com/the-seriously-creepy-two-kitten-expe...