3 ms·

"Vectors seem inadequate for capturing the structure of language, logic, and other cognitive domains, yet neural networks achieve impressive performance in thes

"Vectors seem inadequate for capturing the structure of language, logic, and other cognitive domains, yet neural networks achieve impressive performance in these areas". Missing the forest for the trees? Aren't neural networks modeled after biological systems? Our brains are obviously able to contain symbolic structure despite not having a "symbol processing unit".

andytratt 1mo ago
yep
antonvs 1mo ago
I hate that whole intro - the first four sentences - so much. It’s nothing but unsupported assumptions. Basically, a strawman that they can do battle with in the paper. Not an auspicious start.
- andytratt 1mo ago
  got to top 2 HN tho lol
  - andytratt 1mo ago
    -1 karma jesus guess i should never make meta commentary lol
- suddenlybananas 1mo ago
  These aren't really strawmen, they're more or less than mainstream opinion in the cognitive sciences from the 80s to maybe 2015-2020 or so.
  - jephs 1mo ago
    Paul Smolensky is a cognitive science titan from that era. He worked with Hinton, Rumelhart, and McClelland on parallel distributed processing, and literally wrote the book on tensor product representations in cognition, with Geraldine Legendre: https://mitpress.mit.edu/9780262516198/the-harmonic-mind-volume-1/ https://mitpress.mit.edu/9780262516198/the-harmonic-mind-vol... He's the axis of this particular group of researchers, being the most senior at the place where they all met, Johns Hopkins. So this is less a straw man and more a quick reminder to his peers: "Right, so, remember this particular thread we've spent the last 40 years hashing out, here we've got another contribution to that particular conversation."
    - antonvs 1mo ago
      If it was supposed to be sarcasm, that does help.
  - antonvs 1mo ago
    I’m aware, but it was a bad mainstream opinion which was contradicted by every bit of evidence ever since early LISP efforts in the late 1960s (still waiting for the release of SHRDLU 2.0!) 65 years is a long time to be beating your head against an obvious dead end. The article intro presents the claims as self-evident, when they’re not at all. But, someone rise pointed out this may have been a dig at those attitudes, which makes more sense.
suddenlybananas 1mo ago
People really overstate the relationship between ANNs and the brain, they have very different mechanisms and only have a similarity if you squint at 100000 feet. ANNs don't have neurotransmitters or even action potentials.
- wetpaws 1mo ago
  [dead]
- imtringued 1mo ago
  Correct. Worst of all, even if you do build a spiking neural network, the update rule is kind of a mystery. To have a good update rule, a biological neuron needs to be kind of like a tiny computer in its own right. You might be able to model synapses as weights between neurons, but the neuron carries further internal states within in itself and how the "update rule" uses those internal states is not known at all.
- addag 1mo ago
  While I agree with this statement, one could argue that what's important is not much the support but the emergent properties - in the same way that a wave is still a wave whether it is in the water or in the air. Thus putting things similar to neurons in a network and making them able to learn could create behaviors similar to the brain. The fact that that biology used chemistry + electrical signal and computers use ReLU-like activation could be merely choosing the most efficient way to enable training.
- azakai 1mo ago
  But neurotransmitters and action potentials don't help in modeling symbolic structure. That is, yes, ANNs are not brains. There are countless differences. But are there differences at the computational level? ANNs are meant to model brain computation, not brain biology. (There is still a lot to debate there, I'm not saying "ANNs are perfect computational models for the brain")
  - suddenlybananas 1mo ago

andytratt 1mo ago

yep

antonvs 1mo ago

I hate that whole intro - the first four sentences - so much. It’s nothing but unsupported assumptions. Basically, a strawman that they can do battle with in the paper. Not an auspicious start.

andytratt 1mo ago
got to top 2 HN tho lol
- andytratt 1mo ago
  -1 karma jesus guess i should never make meta commentary lol
suddenlybananas 1mo ago
These aren't really strawmen, they're more or less than mainstream opinion in the cognitive sciences from the 80s to maybe 2015-2020 or so.
- jephs 1mo ago
  Paul Smolensky is a cognitive science titan from that era. He worked with Hinton, Rumelhart, and McClelland on parallel distributed processing, and literally wrote the book on tensor product representations in cognition, with Geraldine Legendre: https://mitpress.mit.edu/9780262516198/the-harmonic-mind-volume-1/ https://mitpress.mit.edu/9780262516198/the-harmonic-mind-vol... He's the axis of this particular group of researchers, being the most senior at the place where they all met, Johns Hopkins. So this is less a straw man and more a quick reminder to his peers: "Right, so, remember this particular thread we've spent the last 40 years hashing out, here we've got another contribution to that particular conversation."
  - antonvs 1mo ago
    If it was supposed to be sarcasm, that does help.
- antonvs 1mo ago
  I’m aware, but it was a bad mainstream opinion which was contradicted by every bit of evidence ever since early LISP efforts in the late 1960s (still waiting for the release of SHRDLU 2.0!) 65 years is a long time to be beating your head against an obvious dead end. The article intro presents the claims as self-evident, when they’re not at all. But, someone rise pointed out this may have been a dig at those attitudes, which makes more sense.

suddenlybananas 1mo ago

People really overstate the relationship between ANNs and the brain, they have very different mechanisms and only have a similarity if you squint at 100000 feet. ANNs don't have neurotransmitters or even action potentials.

wetpaws 1mo ago
[dead]
imtringued 1mo ago
Correct. Worst of all, even if you do build a spiking neural network, the update rule is kind of a mystery. To have a good update rule, a biological neuron needs to be kind of like a tiny computer in its own right. You might be able to model synapses as weights between neurons, but the neuron carries further internal states within in itself and how the "update rule" uses those internal states is not known at all.
addag 1mo ago
While I agree with this statement, one could argue that what's important is not much the support but the emergent properties - in the same way that a wave is still a wave whether it is in the water or in the air. Thus putting things similar to neurons in a network and making them able to learn could create behaviors similar to the brain. The fact that that biology used chemistry + electrical signal and computers use ReLU-like activation could be merely choosing the most efficient way to enable training.
azakai 1mo ago
But neurotransmitters and action potentials don't help in modeling symbolic structure. That is, yes, ANNs are not brains. There are countless differences. But are there differences at the computational level? ANNs are meant to model brain computation, not brain biology. (There is still a lot to debate there, I'm not saying "ANNs are perfect computational models for the brain")
- suddenlybananas 1mo ago