4 ms·

You may be right for the time being, but I don't think we're permanently entering a brave new world where machine learning startups are doomed to fail to the bi

You may be right for the time being, but I don't think we're permanently entering a brave new world where machine learning startups are doomed to fail to the big tech companies. It's well known that existing machine learning algorithms are extremely inefficient learners, which is why you need enormous datasets and computing resources to be competitive currently. Right now, for instance, a computer would need to see thousands of different images of a new animal to recognize it consistently, whereas a human only needs to see a single hand-drawn sketch. If someone can develop machine learning techniques that make computers generalize as flexibly as humans, that would be a huge game changer and would completely upend the advantages that the big tech companies have.

AndrewKemendo 9y ago
If someone can develop machine learning techniques that make computers generalize as flexibly as humans Funny you should mention that... Deepmind (acquired by Google in 2014) goal: "DeepMind Technologies' goal is to "solve intelligence" Demis has gone one to say elsewhere their goal is to create AGI Also, the data you need to get there is owned by a few companies. Also those companies hire effectively every great ML PhD on the planet. It's well known that existing machine learning algorithms are extremely inefficient learners Right, which is why the heads of research at AFGAM are the pre-eminent minds in the field (Lecun, Hinton etc...) and are actively working on this along with unsupervised learning. These companies are either funding university labs that are developing this stuff or doing it in house.
trhway 9y ago
>Right now, for instance, a computer would need to see thousands of different images of a new animal to recognize it consistently, whereas a human only needs to see a single hand-drawn sketch. valid question here would be - would a fresh human mind perform that well too? Or may be such performance is conditioned upon having at least several years of experience of living on planet Earth and thus having the models of animals and everything else around been built using billions of images that human eyes would have generated during those years? > It's well known that existing machine learning algorithms are extremely inefficient learners, which is why you need enormous datasets and computing resources to be competitive currently. when put into perspective, i.e. compared with the cardinality of the above mentioned set of images human eyes generate and computational performance of the brain - 40K TFlops - computers don't look that inefficient learners to me.
- antognini 9y ago
  > valid question here would be - would a fresh human mind perform that well too? I think it's questions like this that will point the way forward. Humans definitely don't come working at 100% as soon as we're born. Lots of interaction with the environment is necessary to become a functioning person. But at the same time, we still seem to have a real ability to generalize hard-wired in. A human in a small society might only speak with a few dozen individuals growing up, but they don't get confused when they meet a new person and hear a new voice. It seems to me that lots of data will be necessary to get things off the ground --- but I think the sort of "big data" needed for this will be qualitatively different than the big datasets used to train modern state of the art models. My hunch is that it will involve an agent's interactions with its environment (so it will look something like reinforcement learning).
  - pas 9y ago
    For a playful yet pretty interesting take on "psychogenesis" I recommend reading Diaspora by Greg Egan. Also, we are born with very good models (for faces, cognition of human languages + speaking and hearing them, seeing things on Earth, movement coordination, other cognitive fundamentals, like counting, categorization, and so on) plus all of that really well packaged as a starter kit, a do-it-yourself general intelligence, with amazing supervisory framework (parents, peers, society, reflective optimization about one's actions) and reinforcement (emotions, memory coding is waaay too drastically modulated by emotions). And we're still not sure what else is in there, how all this communicates. What's the operating system of consciousness? (We "know" that consciousness is just a "program", but is it the main thread, is it a scheduler, what's the right mental model for understanding the interaction of brain faculties, components of cognition and consciousness itself?)
  - webmaven 9y ago
    > A human in a small society might only speak with a few dozen individuals growing up, but they don't get confused when they meet a new person and hear a new voice. It does seems pretty well established that some humans overfit on the particular aspects of verbal communication that we generally call "accent" (not to mention actual lects), even when given access to a much larger and more diverse set of individuals and their utterances to generalise from.

AndrewKemendo 9y ago

If someone can develop machine learning techniques that make computers generalize as flexibly as humans Funny you should mention that... Deepmind (acquired by Google in 2014) goal: "DeepMind Technologies' goal is to "solve intelligence" Demis has gone one to say elsewhere their goal is to create AGI Also, the data you need to get there is owned by a few companies. Also those companies hire effectively every great ML PhD on the planet. It's well known that existing machine learning algorithms are extremely inefficient learners Right, which is why the heads of research at AFGAM are the pre-eminent minds in the field (Lecun, Hinton etc...) and are actively working on this along with unsupervised learning. These companies are either funding university labs that are developing this stuff or doing it in house.

trhway 9y ago

>Right now, for instance, a computer would need to see thousands of different images of a new animal to recognize it consistently, whereas a human only needs to see a single hand-drawn sketch. valid question here would be - would a fresh human mind perform that well too? Or may be such performance is conditioned upon having at least several years of experience of living on planet Earth and thus having the models of animals and everything else around been built using billions of images that human eyes would have generated during those years? > It's well known that existing machine learning algorithms are extremely inefficient learners, which is why you need enormous datasets and computing resources to be competitive currently. when put into perspective, i.e. compared with the cardinality of the above mentioned set of images human eyes generate and computational performance of the brain - 40K TFlops - computers don't look that inefficient learners to me.

antognini 9y ago
> valid question here would be - would a fresh human mind perform that well too? I think it's questions like this that will point the way forward. Humans definitely don't come working at 100% as soon as we're born. Lots of interaction with the environment is necessary to become a functioning person. But at the same time, we still seem to have a real ability to generalize hard-wired in. A human in a small society might only speak with a few dozen individuals growing up, but they don't get confused when they meet a new person and hear a new voice. It seems to me that lots of data will be necessary to get things off the ground --- but I think the sort of "big data" needed for this will be qualitatively different than the big datasets used to train modern state of the art models. My hunch is that it will involve an agent's interactions with its environment (so it will look something like reinforcement learning).
- pas 9y ago
  For a playful yet pretty interesting take on "psychogenesis" I recommend reading Diaspora by Greg Egan. Also, we are born with very good models (for faces, cognition of human languages + speaking and hearing them, seeing things on Earth, movement coordination, other cognitive fundamentals, like counting, categorization, and so on) plus all of that really well packaged as a starter kit, a do-it-yourself general intelligence, with amazing supervisory framework (parents, peers, society, reflective optimization about one's actions) and reinforcement (emotions, memory coding is waaay too drastically modulated by emotions). And we're still not sure what else is in there, how all this communicates. What's the operating system of consciousness? (We "know" that consciousness is just a "program", but is it the main thread, is it a scheduler, what's the right mental model for understanding the interaction of brain faculties, components of cognition and consciousness itself?)
- webmaven 9y ago
  > A human in a small society might only speak with a few dozen individuals growing up, but they don't get confused when they meet a new person and hear a new voice. It does seems pretty well established that some humans overfit on the particular aspects of verbal communication that we generally call "accent" (not to mention actual lects), even when given access to a much larger and more diverse set of individuals and their utterances to generalise from.