4 ms·

At what point have our neural networks crossed over to demonstrating algorithmic behavior and we no longer consider them fancy interpolating functions? Is there

At what point have our neural networks crossed over to demonstrating algorithmic behavior and we no longer consider them fancy interpolating functions? Is there a way to quantify this?

whatshisface 4y ago
A model with no training data would know nothing, so in a sense they're always going to be something like a fancy form of interpolation/extrapolation.
- Xcelerate 4y ago
  There’s a difference between interpolation and induction. You’re not going to interpolate a hash function.
  - whatshisface 4y ago
    There's no way a neural network could ever learn a hash function directly (unless it had every possible input and output in its table), and if there was an indirect way to train it, you'd discover that it was interpolating between (say) possible hash functions by working in a larger space, for example if it was trained to generate and test C programs that computed hashes.
IIAOPSW 4y ago
Never. Instead, we begin understanding ourselves as fancy interpolating functions.
- akiselev 4y ago
  See Moravec's paradox [1], which has formulated decades ago. The more we build these models the more it's starting to seem like the paradox is basically true: that the hard part of intelligence isn't higher reasoning but the basic perception and sensorimotor control that animals have been evolving for hundreds of millions of years before homo sapiens came on the scene. [1] https://en.wikipedia.org/wiki/Moravec%27s_paradox https://en.wikipedia.org/wiki/Moravec%27s_paradox
- otabdeveloper4 4y ago
  People in the 17th century viewed themselves as fancy clockwork machines, because of course mechanical clocks were the peak of high-tech hype at the time. (Really what we should be thinking about is information complexity, not facile analogies.)
  - sacado2 4y ago
    And in theological arguments back then, god was called "the great watchmaker". Now the question "are we living in a computer simulation?" keeps popping here and there. I wonder what the analogy will be in 400 years.
    - quickthrower2 4y ago
      In 400 years: "back in the day the meatbags thought they were special"

At what point have our neural networks crossed over to demonstrating algorithmic behavior and we no longer consider them fancy interpolating functions? Is there a way to quantify this?

whatshisface 4y ago

A model with no training data would know nothing, so in a sense they're always going to be something like a fancy form of interpolation/extrapolation.

Xcelerate 4y ago
There’s a difference between interpolation and induction. You’re not going to interpolate a hash function.
- whatshisface 4y ago
  There's no way a neural network could ever learn a hash function directly (unless it had every possible input and output in its table), and if there was an indirect way to train it, you'd discover that it was interpolating between (say) possible hash functions by working in a larger space, for example if it was trained to generate and test C programs that computed hashes.

IIAOPSW 4y ago

Never. Instead, we begin understanding ourselves as fancy interpolating functions.

akiselev 4y ago
See Moravec's paradox [1], which has formulated decades ago. The more we build these models the more it's starting to seem like the paradox is basically true: that the hard part of intelligence isn't higher reasoning but the basic perception and sensorimotor control that animals have been evolving for hundreds of millions of years before homo sapiens came on the scene. [1] https://en.wikipedia.org/wiki/Moravec%27s_paradox https://en.wikipedia.org/wiki/Moravec%27s_paradox
otabdeveloper4 4y ago
People in the 17th century viewed themselves as fancy clockwork machines, because of course mechanical clocks were the peak of high-tech hype at the time. (Really what we should be thinking about is information complexity, not facile analogies.)
- sacado2 4y ago
  And in theological arguments back then, god was called "the great watchmaker". Now the question "are we living in a computer simulation?" keeps popping here and there. I wonder what the analogy will be in 400 years.
  - quickthrower2 4y ago
    In 400 years: "back in the day the meatbags thought they were special"