4 ms·

Is that the best way to describe the architecture if the n-1 layers are fed with completely different input data?

by MRD85 8y ago

mollerhoj 8y ago
No I guess not. My mistake: If the input is completely separate, then it makes sense to call it 1000 different NNs. If the input is not completely separate, then the architure I described with a high percentage of dropout in the first layer seems like a reasonable description? This doesn't address my quarrel with the theory though; Why would that be either superior performance wise or more biologically realistic?
- jmmcd 8y ago
  I agree that a "voting" mechanism sounds all wrong, not only for the reason you said. In the scenario described, where we have some neurons coming ultimately from the hand and sense of touch, and others coming ultimately from the eye and sense of vision, and they disagree about what they're perceiving, they're not voting! First of all, it might be better described as a "negotiation", and in fairness I think their theory does envisage this (but they do use the term "vote", albeit in scare quotes). Second, what this theory misses (as does a lot of recent work which takes neural networks as a jumping-off point) is the transition from sensory/sub-cognitive processing up to conscious/cognitive processing. If my hand feels a cat and my eye sees a coffee cup, then I'll consciously notice the contradiction and gather more evidence.
  - your-nanny 8y ago
    what may be envisioned is a process like that of information integration in two-choice tasks about which we have several neural models.
- deleted 8y ago
  [deleted]
- mannykannot 8y ago
  To address your last question, I think they are proposing (but have not yet demonstrated) that the human neocortex works this way, and I imagine the motivation for being more biologically realistic (beyond its usefulness in learning more about the biology) is that biology is still outperforming AI in these tasks.

Is that the best way to describe the architecture if the n-1 layers are fed with completely different input data?

by MRD85 8y ago

Is that the best way to describe the architecture if the n-1 layers are fed with completely different input data?

No I guess not. My mistake: If the input is completely separate, then it makes sense to call it 1000 different NNs. If the input is not completely separate, then the architure I described with a high percentage of dropout in the first layer seems like a reasonable description? This doesn't address my quarrel with the theory though; Why would that be either superior performance wise or more biologically realistic?

jmmcd 8y ago
I agree that a "voting" mechanism sounds all wrong, not only for the reason you said. In the scenario described, where we have some neurons coming ultimately from the hand and sense of touch, and others coming ultimately from the eye and sense of vision, and they disagree about what they're perceiving, they're not voting! First of all, it might be better described as a "negotiation", and in fairness I think their theory does envisage this (but they do use the term "vote", albeit in scare quotes). Second, what this theory misses (as does a lot of recent work which takes neural networks as a jumping-off point) is the transition from sensory/sub-cognitive processing up to conscious/cognitive processing. If my hand feels a cat and my eye sees a coffee cup, then I'll consciously notice the contradiction and gather more evidence.
- your-nanny 8y ago
  what may be envisioned is a process like that of information integration in two-choice tasks about which we have several neural models.
deleted 8y ago
[deleted]
mannykannot 8y ago
To address your last question, I think they are proposing (but have not yet demonstrated) that the human neocortex works this way, and I imagine the motivation for being more biologically realistic (beyond its usefulness in learning more about the biology) is that biology is still outperforming AI in these tasks.