3 ms·

"They improve ROI on the margins, but automating basically broken inferential rules at scale -- it is the automation here, and speed that is useful. Not the act

"They improve ROI on the margins, but automating basically broken inferential rules at scale -- it is the automation here, and speed that is useful. Not the actual quality of inference." This is trivially incorrect, and again shows a lack of curiosity. A couple obvious counterexamples are AlphaGo (and descendants) and AlphaFold, both of which have quality of inference far out stripping previous efforts. In AlphaFold's case the results are pushing forward a whole field of science. Plenty of systems at this point in history outperform human experts, and not just on speed metrics. '"The pen is mightier than the sword!" is something I can agree, or disagree with; I have a feeling about it. And in inferential contexts which require me to feel, I feel.' What are these feelings? Are they necessary for learning? Will airplanes never fly until they learn to flap their wings like birds? 'we the dumb apes who have the concepts are fooled into thinking that some machine assembling of these words had something to say.' Still arguing with straw men here... 'The relevant structure to be learnt, the representation, has nothing to do with sequences of characters.' Of course it does. For BERT, the output is a conditional probability of words based on surrounding context.

goodtraveler 5y ago
OP makes great points that are not being addressed here. If you believe that animal intelligence is reducible to abstract symbol shuffling then it should be possible to extend the existing approaches to actual robotic automatons but it is clear that these models are unable to deal with the dynamic structure of the world and there is no reasonable avenue to actually make that happen. So either intelligence is reducible to abstract symbol shuffling or there is a semantic fib happening here that substitutes "abstract symbol shuffling" for "intelligence". The fact that we imbue these abstract symbols with meaning is where the intelligence comes from, i.e. people attribute intelligent behavior to these systems because we have learned to attribute intelligent behavior to the production of abstract symbols. It's clear to me that current mathematics is insufficient for creating intelligence (let alone general intelligence). If you know of anyone that makes a coherent case for the mathematical basis of intelligence then I would like to see those references.
- sdenton4 5y ago
  The OP makes overly broad arguments about AI that get details the details wrong and have obvious counterexamples in current ML techniques and results. On the ML side, there's been steady progress with an accumulation of very visible wins, accompanied by nearly-invisible adoption of techniques in day-to-day usage. (eg, keyboard text completion and text to speech systems.) Much of this progress has been in specific subdomains, as combining domains requires massively more effort and investment. However, it's starting to happen; CLIP methods create joint embedding spaces for multiple modalities, and have led to the zero-shot learning we see exhibited in DALL-E. This is stuff that simply wasn't possible five years ago, building on sub-domain tools which have greatly increased in their quality such as BERT. These joint embedding spaces will get better, encompass more modalities, and fuel further results. For example, as joint text-video embeddings become more powerful, we'll have embeddings which jointly encode text and physics. As a result, I would be extremely hesitant to write off robotics applications. There's no big-headlines breakthrough on ML robotics /right now/, but we also didn't have a clear road to zero-shot image generation from text inputs a couple years ago. Notice that many of the existing 'big' results were obviously impossible until they very much were not. Finally, I don't believe that we have a good enough definition of 'intelligence' to say with any certainty whether current mathematics is sufficient or not. Don't underestimate the potential for simple components to produce complex behaviors, though. But also, don't expect Pinnochio; intelligence may be broader than the human implementation.
  - goodtraveler 5y ago
    OpenAI shuttered their robotics division because they did not see any viable path to commercial applications so they pivoted to generating pixel art. Similarly, DeepMind has not been making any claims of achieving AGI because they're smart enough to realize that statistical modeling of physical systems is a very small subset of what counts as intelligence. I'm not dismissive of the progress in the field. What I find confusing is why so many people are convinced that what we are seeing with these abstract symbol shuffling systems is intelligence. All it does is confuse the average person about what these tools are capable of because at the moment they are only capable of amplifying biases in existing data sets. No statistical model can escape this trap and at the moment we essentially have automated bias amplifiers that are being sold as some kind of revolution in designing intelligent systems.

goodtraveler 5y ago

OP makes great points that are not being addressed here. If you believe that animal intelligence is reducible to abstract symbol shuffling then it should be possible to extend the existing approaches to actual robotic automatons but it is clear that these models are unable to deal with the dynamic structure of the world and there is no reasonable avenue to actually make that happen. So either intelligence is reducible to abstract symbol shuffling or there is a semantic fib happening here that substitutes "abstract symbol shuffling" for "intelligence". The fact that we imbue these abstract symbols with meaning is where the intelligence comes from, i.e. people attribute intelligent behavior to these systems because we have learned to attribute intelligent behavior to the production of abstract symbols. It's clear to me that current mathematics is insufficient for creating intelligence (let alone general intelligence). If you know of anyone that makes a coherent case for the mathematical basis of intelligence then I would like to see those references.

sdenton4 5y ago
The OP makes overly broad arguments about AI that get details the details wrong and have obvious counterexamples in current ML techniques and results. On the ML side, there's been steady progress with an accumulation of very visible wins, accompanied by nearly-invisible adoption of techniques in day-to-day usage. (eg, keyboard text completion and text to speech systems.) Much of this progress has been in specific subdomains, as combining domains requires massively more effort and investment. However, it's starting to happen; CLIP methods create joint embedding spaces for multiple modalities, and have led to the zero-shot learning we see exhibited in DALL-E. This is stuff that simply wasn't possible five years ago, building on sub-domain tools which have greatly increased in their quality such as BERT. These joint embedding spaces will get better, encompass more modalities, and fuel further results. For example, as joint text-video embeddings become more powerful, we'll have embeddings which jointly encode text and physics. As a result, I would be extremely hesitant to write off robotics applications. There's no big-headlines breakthrough on ML robotics /right now/, but we also didn't have a clear road to zero-shot image generation from text inputs a couple years ago. Notice that many of the existing 'big' results were obviously impossible until they very much were not. Finally, I don't believe that we have a good enough definition of 'intelligence' to say with any certainty whether current mathematics is sufficient or not. Don't underestimate the potential for simple components to produce complex behaviors, though. But also, don't expect Pinnochio; intelligence may be broader than the human implementation.
- goodtraveler 5y ago
  OpenAI shuttered their robotics division because they did not see any viable path to commercial applications so they pivoted to generating pixel art. Similarly, DeepMind has not been making any claims of achieving AGI because they're smart enough to realize that statistical modeling of physical systems is a very small subset of what counts as intelligence. I'm not dismissive of the progress in the field. What I find confusing is why so many people are convinced that what we are seeing with these abstract symbol shuffling systems is intelligence. All it does is confuse the average person about what these tools are capable of because at the moment they are only capable of amplifying biases in existing data sets. No statistical model can escape this trap and at the moment we essentially have automated bias amplifiers that are being sold as some kind of revolution in designing intelligent systems.