5 ms·

A better title would have been: how to make the universe do (a whole lot of) math for us [1]. What so-called neural networks do should not be confused with thi

by lisper 4y ago

A better title would have been: how to make the universe do (a whole lot of) math for us [1]. What so-called neural networks do should not be confused with thinking, at least not yet. And the fact that we can get the universe to do math for us should not be surprising: we can model the universe with math, so of course that mapping works in the other direction as well. And this is not news. There were analog computers long before there were digital ones. --- [1] ... using surprisingly small amounts of hardware relative to what a digital computer would require for certain kinds of computations that turn out to be kind of interesting and useful in specific domains. But that's not nearly as catchy as the original.

misja111 4y ago
> we can model the universe with math, so of course that mapping works in the other direction as well. This is not so obvious as you make it appear. For instance, we can model the weather for the next couple of days using math. But letting the weather of the next couple of days calculate math for us doesn't work very well. The reason is that we can't set the inputs for the weather. This problem comes up in various forms and shapes in other 'nature computers' as well. Quantum computers are another example where the model works brilliantly but setting the pre- and side conditions in the real world is a major headache.
- lisper 4y ago
  I didn’t mean to imply that implementing it should be easy. Only that it should be unsurprising that it is possible.
- goldenkey 4y ago
  You can use the weather or a bucket of water or well, any sufficiently complex chaotic system, as a reservoir computer though: https://en.wikipedia.org/wiki/Reservoir_computing https://en.wikipedia.org/wiki/Reservoir_computing
  - yetihehe 4y ago
    Reservoir computing still needs to provide input. How do you input into weather? And if you find that you indeed can provide inputs to weather, you should ask if you should provide inputs into weather. Using weather as a computer might simply be unethical and could get you killed (after some angry farmers come knocking on your lab door because you ruined their harvest).
    - IIAOPSW 4y ago
      > How do you input into weather? Butterfly conservatory.
zmgsabst 4y ago
> What so-called neural networks do should not be confused with thinking, at least not yet. I disagree: I think neural networks are learning an internal language in which they reason about decisions, based on the data they’ve seen. I think tensor DAGs correspond to an implicit model for some language, and we just lack the tools to extract that. We can translate reasoning in a type theory into a tensor DAG, so I’m not sure why people object to that mapping working the other direction as well.
- V__ 4y ago
  This internal language, if I'm not mistaken, is exactly what the encoder and decoder parts of the neural networks do. > in which they reason about decisions I'm in awe of what the latest neural networks can produce, but I'm wary to call it “reasoning” or “deciding”. NNs are just very complex math equations and calling this intelligence is, in my opinion, muddying the waters of how far away we are from actual AI.
  - zmgsabst 4y ago
    > NNs are just very complex math equations So is the equation modeling your brain. > This internal language, if I'm not mistaken, is exactly what the encoder and decoder parts of the neural networks do. The entire ANN is also a model for a language, with the “higher” parts defining what terms are legal and the “lower” defining how terms are constructed. Roughly. > I'm in awe of what the latest neural networks can produce, but I'm wary to call it “reasoning” or “deciding”. What do you believe you do, besides develop an internal language in response to data in which you then make decisions? The process of ANN evaluation is the same as fitting terms in a type theory and producing an outcome based on that term. We call that “reasoning” in most cases. I don’t care if submarines “swim”; I care they propel themselves through the water. > calling this intelligence is, in my opinion, muddying the waters of how far away we are from actual AI Goldfish show mild intelligence because they can learn mazes; ants farm; bees communicate the location of food via dance; etc. I think you’re the one muddying the waters by placing some special status on human like intelligence without recognizing the spectrum of natural intelligence and that neural networks legitimately fit on that spectrum.
hans1729 4y ago
assertion: thinking is synonymous with computation (composed operations on symbolic systems). computation is boolean algebra. -> therefore, doing math is to think. I'm not trying to be pedantic, I just don't think using intuitive associations with words helps clarifying things. If your definition for thought diverges here, please try to specify how exactly: what is thought, then? Semi-autonomous "pondering"? Because the closer I look at it, that, too, becomes boolean algebra, calling eval() on some semantic construct, which boils down to symbolic logic. What you may mean is that "neural" networks are performing statistics instead of algebra, but that's not what the article is about, is it?
- sweetdreamerit 4y ago
  > I don't think using intuitive associations with words helps clarify things Sincere question: do you think that "think using intuitive associations with words" can be safely translated to "compute using intuitive associations with words"? I don't think so. Therefore, even if thinking is also computing, reducing thinking to boolean algebra is a form of reductionism that ignores a number of emergent properties of (human) thinking.
  - hans1729 4y ago
    Fair question/point. Yes, I do think so. The intuitive model associated with some variable/word as a concept relates to other structures/models/systems that it interfaces with. Just because the operator that accesses these models with rather vague keys (words) has no clear picture of what exactly is being computed on the surface, doesn't mean that the totality of the process is not computation. It just means that the emergent properties are not mapped into the semantic space which the operator (our attention mechanisms) operates on. From my understanding, the totality I just referred to is a graph-space, it doesn't escape mathematics. Then again, I can't know or claim to do so.
    - polypodiopsi 4y ago
      Hybris
- 4y ago
troyvit 4y ago
We can model the universe with math because math is what we have to model the universe with. The fact that it can talk back to us in math is amazing because to me it means that math is not a dead end cosmically, which means we might be able to use it to communicate with other intelligences after all.

by lisper 4y ago

misja111 4y ago

> we can model the universe with math, so of course that mapping works in the other direction as well. This is not so obvious as you make it appear. For instance, we can model the weather for the next couple of days using math. But letting the weather of the next couple of days calculate math for us doesn't work very well. The reason is that we can't set the inputs for the weather. This problem comes up in various forms and shapes in other 'nature computers' as well. Quantum computers are another example where the model works brilliantly but setting the pre- and side conditions in the real world is a major headache.

lisper 4y ago
I didn’t mean to imply that implementing it should be easy. Only that it should be unsurprising that it is possible.
goldenkey 4y ago
You can use the weather or a bucket of water or well, any sufficiently complex chaotic system, as a reservoir computer though: https://en.wikipedia.org/wiki/Reservoir_computing https://en.wikipedia.org/wiki/Reservoir_computing
- yetihehe 4y ago
  Reservoir computing still needs to provide input. How do you input into weather? And if you find that you indeed can provide inputs to weather, you should ask if you should provide inputs into weather. Using weather as a computer might simply be unethical and could get you killed (after some angry farmers come knocking on your lab door because you ruined their harvest).
  - IIAOPSW 4y ago
    > How do you input into weather? Butterfly conservatory.

zmgsabst 4y ago

> What so-called neural networks do should not be confused with thinking, at least not yet. I disagree: I think neural networks are learning an internal language in which they reason about decisions, based on the data they’ve seen. I think tensor DAGs correspond to an implicit model for some language, and we just lack the tools to extract that. We can translate reasoning in a type theory into a tensor DAG, so I’m not sure why people object to that mapping working the other direction as well.

V__ 4y ago
This internal language, if I'm not mistaken, is exactly what the encoder and decoder parts of the neural networks do. > in which they reason about decisions I'm in awe of what the latest neural networks can produce, but I'm wary to call it “reasoning” or “deciding”. NNs are just very complex math equations and calling this intelligence is, in my opinion, muddying the waters of how far away we are from actual AI.
- zmgsabst 4y ago
  > NNs are just very complex math equations So is the equation modeling your brain. > This internal language, if I'm not mistaken, is exactly what the encoder and decoder parts of the neural networks do. The entire ANN is also a model for a language, with the “higher” parts defining what terms are legal and the “lower” defining how terms are constructed. Roughly. > I'm in awe of what the latest neural networks can produce, but I'm wary to call it “reasoning” or “deciding”. What do you believe you do, besides develop an internal language in response to data in which you then make decisions? The process of ANN evaluation is the same as fitting terms in a type theory and producing an outcome based on that term. We call that “reasoning” in most cases. I don’t care if submarines “swim”; I care they propel themselves through the water. > calling this intelligence is, in my opinion, muddying the waters of how far away we are from actual AI Goldfish show mild intelligence because they can learn mazes; ants farm; bees communicate the location of food via dance; etc. I think you’re the one muddying the waters by placing some special status on human like intelligence without recognizing the spectrum of natural intelligence and that neural networks legitimately fit on that spectrum.

hans1729 4y ago

assertion: thinking is synonymous with computation (composed operations on symbolic systems). computation is boolean algebra. -> therefore, doing math is to think. I'm not trying to be pedantic, I just don't think using intuitive associations with words helps clarifying things. If your definition for thought diverges here, please try to specify how exactly: what is thought, then? Semi-autonomous "pondering"? Because the closer I look at it, that, too, becomes boolean algebra, calling eval() on some semantic construct, which boils down to symbolic logic. What you may mean is that "neural" networks are performing statistics instead of algebra, but that's not what the article is about, is it?

sweetdreamerit 4y ago
> I don't think using intuitive associations with words helps clarify things Sincere question: do you think that "think using intuitive associations with words" can be safely translated to "compute using intuitive associations with words"? I don't think so. Therefore, even if thinking is also computing, reducing thinking to boolean algebra is a form of reductionism that ignores a number of emergent properties of (human) thinking.
- hans1729 4y ago
  Fair question/point. Yes, I do think so. The intuitive model associated with some variable/word as a concept relates to other structures/models/systems that it interfaces with. Just because the operator that accesses these models with rather vague keys (words) has no clear picture of what exactly is being computed on the surface, doesn't mean that the totality of the process is not computation. It just means that the emergent properties are not mapped into the semantic space which the operator (our attention mechanisms) operates on. From my understanding, the totality I just referred to is a graph-space, it doesn't escape mathematics. Then again, I can't know or claim to do so.
  - polypodiopsi 4y ago
    Hybris
4y ago

troyvit 4y ago

We can model the universe with math because math is what we have to model the universe with. The fact that it can talk back to us in math is amazing because to me it means that math is not a dead end cosmically, which means we might be able to use it to communicate with other intelligences after all.