5 ms·

Call it what you will, but from what I can see progress exploded when theoretical justification was thrown out the window. Maybe intelligence just is messy at i

Call it what you will, but from what I can see progress exploded when theoretical justification was thrown out the window. Maybe intelligence just is messy at its core? Science is all about induction, but what if there are no simple rules to break this down? The only successful intelligence we know of came about through evolution, not science. The way I see it, researchers are now doing what evolution did, but in a more guided way and much more quickly. Whether you call that science or engineering or tinkering, it doesn't really matter.

The_rationalist 6y ago
You're totally omitting deduction and abduction. Those will require at least symbolic logic on graph.
otabdeveloper4 6y ago
> progress exploded Did it? Deep Learning provided us with some mindblowing performance art (FaceApp, DeepFake, etc.), but as far as I know any actual business value that claims to be derived from 'AI' is fake news.
- mellosouls 6y ago
  Business value is hardly the only or even main determinant of progress.
  - otabdeveloper4 6y ago
    Of course, just pointing out that not everyone's point of view agrees with the glib pronunciation that 'progress exploded'. In my little world attitudes have soured on 'AI'; 'AI' is posed to become the next 'blockchain'.
    - plutonorm 6y ago
      This is the natural course of the hype cycle. Like kids with a new toy, infatuation and then boredom. But the underlying technology in this case is absolutely world changing. I think we are only a few orders of magnitude away from systems that are genuinely intelligent. GPT-5 or 6 will have your jaw on the floor and GPT-10 will probably be smarter than all of us.
      - mellosouls 6y ago
        I think we are only a few orders of magnitude away from systems that are genuinely intelligent. A few orders of magnitude is potentially very significant. It's worth remembering that despite the hype and fake-clever systems playing chess and constructing plausible sentences, we still have nothing remotely approaching AGI at the moment, and no reason to think it is on the horizon. But I agree that the AI tools we have have made significant progress in the line of powerful un-intelligence, and will have significant impact in automation.
        plutonorm 6y ago
        "It's worth remembering that despite the hype and fake-clever systems playing chess and constructing plausible sentences". I completely disagree. I'm a connectionist and subscribe to the "Meaning is use" mantra from wittgenstein, when it comes to language. Language is the medium of thoughts, if GPT-x understands language, it is a general intelligence. If you read into the the output of GPT-3 you can see glimmers of genuine intelligence. An intelligent machine is not something you design, it is a machine that designs itself. All that is needed is to step out of the way of the machine. All you need is gradient descent and extraordinary processing power, there's no magic sauce left to find. Being clever and creating complex learning strategies just buys you more power at the expense of generality. Transformers and convolutional layers and the other tricks we have are probably now enough to get us within shooting distance of AGI. If you look at the progress of machine learning the best predictor of performance is processing power. I see no reason why it will not continue to scale. In the minds of many there is something very un-satisfying, unsettling even, in the idea progress without understanding. But the very definition of intelligence requires that you give up the responsibility of understanding to the machine. Now there is probably an overarching theory here that we don't understand - why is it that search within the space of algorithms described by a neural network is an effective strategy when solving the problems we tend to encounter in the real world? And that probably isn't a pure theory question, it's a question about physics - why do the emergent physics of the world permit easy solutions within the space of possible functions described by neural networks? Why are these solutions discoverable? No one has a handle on these questions yet, I suspect it's a very very deep question and unlikely to be solved within our lifetime. So everyone should "stop worrying and learn to love the neural network"
    - visarga 6y ago
      AI is like the internet pre 2000's or electricity a century ago - barely been applied to the world. It's not time to get disappointed yet.
      - mellosouls 6y ago
        It's also like AI last century also in the lack of progress towards AGI, so plenty of time to already have been disappointed - certainly nothing to get excited about.
    - mellosouls 6y ago
      I agree with both those claims completely. Absurd hype, discussed many times here on HN.
- lukasLansky 6y ago
  Voice recognition, translation, OCR, image classification, etc. is done through neural networks mostly nowadays. If https://en.m.wikipedia.org/wiki/Google_Translate https://en.m.wikipedia.org/wiki/Google_Translate does not seem like having business value to you, then I don't understand what you mean by "business value".
  - otabdeveloper4 6y ago
    > If https://en.m.wikipedia.org/wiki/Google_Translate https://en.m.wikipedia.org/wiki/Google_Translate does not seem like having business value to you, then I don't understand what you mean by "business value". There's no way Google Translate will ever be monetized. So yes, it's effectively an expensive art project for Google, not something that brings real business value. Hotword detection ('ok google') is a practical application of AI, but that's really slim pickings considering how hyped 'AI' was. (Also, the costs of data collection for this feature are still way too high.)
    - lukasLansky 6y ago
      I think we are really torturing language here. I don't want to get into specifics of what constitutes business value, how to value brand and customer satisfaction and so on. I can add more examples like https://deepmind.com/blog/article/deepmind-ai-reduces-google-data-centre-cooling-bill-40 https://deepmind.com/blog/article/deepmind-ai-reduces-google... that shows how neural networks helps in the pure bottom-line sense you care about. Let me close my argument: neural network techniques developed in the last ten years are super useful here and now, they are used by billions of people every day and they do make their lives easier.
      - otabdeveloper4 6y ago
        > I think we are really torturing language here. Not really. My point is simple: as of 2020, any proposition for a business to 'invest in AI' is a money-losing proposition. (Just like 'blockchain'.)
    - tchaffee 6y ago
- nl 6y ago
  I'm at my third company making money from neural networks, so I don't know where you get this idea. 2 have been around text and one on video.
jsinai 6y ago
Actually most of the scientific progress in these tasks had already been made and all that was missing was the compute power. Just like the know-how and basic principles of how electric cars and have been around for more than a 100 years and what’s most been missing are cheap and powerful enough batteries. So it depends on how you measure progress? Scientifically and advancement of knowledge, or success in applying such knowledge to business problems and use cases. That the deep learning community insists on publishing most of these breakthroughs as research is the basis of the critique being made in the OP.
- nl 6y ago
  This isn't correct. No amount of compute power will let a RNN/LSTM outperform a Transformer. No amount of compute power will let an AlexNet or Inception style network train as well as a ResNet.
  - anukriti 6y ago
    > I think the fact that we're seeing so much progress in deep learning without any theoretical basis I don't understand what you mean by this? Any neural network is first build in theory and then tried out practically by coding and testing. I think practitioner and scientist play equal role.
- martindbp 6y ago
  I think it was Nassim Taleb[1] that made a case for that engineering and tinkering drives science rather than the other way around, and that most of the knowledge gained comes from the practitioners rather than the scientists, who swoop in to claim credit after the fact (he uses the jet-engine as an example of this). That's not to say fundamental science is unimportant, just that it's getting too much of the credit for all the stuff that's been built. There's a tremendous amount of know-how required to successfully build an airplane, a jet-engine, a processor, or to successfully train a large neural network. And while theoretical physics no doubt can explain it all, it's not sufficient to build something that works. I think the fact that we're seeing so much progress in deep learning without any theoretical basis is another example of this. [1] Antifragile, Chapter: History Written By the Losers

The_rationalist 6y ago

You're totally omitting deduction and abduction. Those will require at least symbolic logic on graph.

otabdeveloper4 6y ago

> progress exploded Did it? Deep Learning provided us with some mindblowing performance art (FaceApp, DeepFake, etc.), but as far as I know any actual business value that claims to be derived from 'AI' is fake news.

mellosouls 6y ago
Business value is hardly the only or even main determinant of progress.
- otabdeveloper4 6y ago
  Of course, just pointing out that not everyone's point of view agrees with the glib pronunciation that 'progress exploded'. In my little world attitudes have soured on 'AI'; 'AI' is posed to become the next 'blockchain'.
  - plutonorm 6y ago
    This is the natural course of the hype cycle. Like kids with a new toy, infatuation and then boredom. But the underlying technology in this case is absolutely world changing. I think we are only a few orders of magnitude away from systems that are genuinely intelligent. GPT-5 or 6 will have your jaw on the floor and GPT-10 will probably be smarter than all of us.
    - mellosouls 6y ago
      I think we are only a few orders of magnitude away from systems that are genuinely intelligent. A few orders of magnitude is potentially very significant. It's worth remembering that despite the hype and fake-clever systems playing chess and constructing plausible sentences, we still have nothing remotely approaching AGI at the moment, and no reason to think it is on the horizon. But I agree that the AI tools we have have made significant progress in the line of powerful un-intelligence, and will have significant impact in automation.
      - plutonorm 6y ago
        "It's worth remembering that despite the hype and fake-clever systems playing chess and constructing plausible sentences". I completely disagree. I'm a connectionist and subscribe to the "Meaning is use" mantra from wittgenstein, when it comes to language. Language is the medium of thoughts, if GPT-x understands language, it is a general intelligence. If you read into the the output of GPT-3 you can see glimmers of genuine intelligence. An intelligent machine is not something you design, it is a machine that designs itself. All that is needed is to step out of the way of the machine. All you need is gradient descent and extraordinary processing power, there's no magic sauce left to find. Being clever and creating complex learning strategies just buys you more power at the expense of generality. Transformers and convolutional layers and the other tricks we have are probably now enough to get us within shooting distance of AGI. If you look at the progress of machine learning the best predictor of performance is processing power. I see no reason why it will not continue to scale. In the minds of many there is something very un-satisfying, unsettling even, in the idea progress without understanding. But the very definition of intelligence requires that you give up the responsibility of understanding to the machine. Now there is probably an overarching theory here that we don't understand - why is it that search within the space of algorithms described by a neural network is an effective strategy when solving the problems we tend to encounter in the real world? And that probably isn't a pure theory question, it's a question about physics - why do the emergent physics of the world permit easy solutions within the space of possible functions described by neural networks? Why are these solutions discoverable? No one has a handle on these questions yet, I suspect it's a very very deep question and unlikely to be solved within our lifetime. So everyone should "stop worrying and learn to love the neural network"
  - visarga 6y ago
    AI is like the internet pre 2000's or electricity a century ago - barely been applied to the world. It's not time to get disappointed yet.
    - mellosouls 6y ago
      It's also like AI last century also in the lack of progress towards AGI, so plenty of time to already have been disappointed - certainly nothing to get excited about.
  - mellosouls 6y ago
    I agree with both those claims completely. Absurd hype, discussed many times here on HN.
lukasLansky 6y ago
Voice recognition, translation, OCR, image classification, etc. is done through neural networks mostly nowadays. If https://en.m.wikipedia.org/wiki/Google_Translate https://en.m.wikipedia.org/wiki/Google_Translate does not seem like having business value to you, then I don't understand what you mean by "business value".
- otabdeveloper4 6y ago
  > If https://en.m.wikipedia.org/wiki/Google_Translate https://en.m.wikipedia.org/wiki/Google_Translate does not seem like having business value to you, then I don't understand what you mean by "business value". There's no way Google Translate will ever be monetized. So yes, it's effectively an expensive art project for Google, not something that brings real business value. Hotword detection ('ok google') is a practical application of AI, but that's really slim pickings considering how hyped 'AI' was. (Also, the costs of data collection for this feature are still way too high.)
  - lukasLansky 6y ago
    I think we are really torturing language here. I don't want to get into specifics of what constitutes business value, how to value brand and customer satisfaction and so on. I can add more examples like https://deepmind.com/blog/article/deepmind-ai-reduces-google-data-centre-cooling-bill-40 https://deepmind.com/blog/article/deepmind-ai-reduces-google... that shows how neural networks helps in the pure bottom-line sense you care about. Let me close my argument: neural network techniques developed in the last ten years are super useful here and now, they are used by billions of people every day and they do make their lives easier.
    - otabdeveloper4 6y ago
      > I think we are really torturing language here. Not really. My point is simple: as of 2020, any proposition for a business to 'invest in AI' is a money-losing proposition. (Just like 'blockchain'.)
  - tchaffee 6y ago
nl 6y ago
I'm at my third company making money from neural networks, so I don't know where you get this idea. 2 have been around text and one on video.

jsinai 6y ago

Actually most of the scientific progress in these tasks had already been made and all that was missing was the compute power. Just like the know-how and basic principles of how electric cars and have been around for more than a 100 years and what’s most been missing are cheap and powerful enough batteries. So it depends on how you measure progress? Scientifically and advancement of knowledge, or success in applying such knowledge to business problems and use cases. That the deep learning community insists on publishing most of these breakthroughs as research is the basis of the critique being made in the OP.

nl 6y ago
This isn't correct. No amount of compute power will let a RNN/LSTM outperform a Transformer. No amount of compute power will let an AlexNet or Inception style network train as well as a ResNet.
- anukriti 6y ago
  > I think the fact that we're seeing so much progress in deep learning without any theoretical basis I don't understand what you mean by this? Any neural network is first build in theory and then tried out practically by coding and testing. I think practitioner and scientist play equal role.
martindbp 6y ago
I think it was Nassim Taleb[1] that made a case for that engineering and tinkering drives science rather than the other way around, and that most of the knowledge gained comes from the practitioners rather than the scientists, who swoop in to claim credit after the fact (he uses the jet-engine as an example of this). That's not to say fundamental science is unimportant, just that it's getting too much of the credit for all the stuff that's been built. There's a tremendous amount of know-how required to successfully build an airplane, a jet-engine, a processor, or to successfully train a large neural network. And while theoretical physics no doubt can explain it all, it's not sufficient to build something that works. I think the fact that we're seeing so much progress in deep learning without any theoretical basis is another example of this. [1] Antifragile, Chapter: History Written By the Losers