4 ms·

> The knowledge (heuristics) are encoded in the structure, not in some kind of information. When you're talking about neural computation, distinguishing betwee

by _dps 10y ago

> The knowledge (heuristics) are encoded in the structure, not in some kind of information. When you're talking about neural computation, distinguishing between an evolved network structure/architecture and "information" (or at least biases toward certain kind of information) is not very useful. The nature of these systems as computing substrates is that they encode information in network structure and rules for updating that structure.

lngnmn 10y ago
> they encode information in network structure and rules for updating that structure I think use of the term 'information' is not accurate. There are no bits and the whole information theory, it seems, is synthetic and has nothing to do with how the brain actually works. It is not a digital machine but electro-mechanical, if you wish.
- _dps 10y ago
  Respectfully, this is begging the question :) You're correct that neural networks don't encode information in the way current digital machines do (i.e. with explicit records/files/etc encoded in bytes). I would disagree that this is the only useful view of information. And certainly, from a Shannon information perspective, the network structure contains information (in the Shannon sense) about the environment or problem for which it was optimized.
  - lngnmn 10y ago
    > the network structure contains information (in the Shannon sense) about the environment ... the way a map represents a territory instead of how a guide book does ;) Or the way a 3D structure made out of the same 20 joined amino-acids could represent either an enzyme (the code) or a protein (the data) and that there is no distinction between code and data (which is the great insight realized in the original Lisp. It is not a coincidence that Lisp was the language or the classic AI. Sorry for the reminiscence - just love this part).

4 ms·

> The knowledge (heuristics) are encoded in the structure, not in some kind of information. When you're talking about neural computation, distinguishing betwee

by _dps 10y ago

lngnmn 10y ago
> they encode information in network structure and rules for updating that structure I think use of the term 'information' is not accurate. There are no bits and the whole information theory, it seems, is synthetic and has nothing to do with how the brain actually works. It is not a digital machine but electro-mechanical, if you wish.
- _dps 10y ago
  Respectfully, this is begging the question :) You're correct that neural networks don't encode information in the way current digital machines do (i.e. with explicit records/files/etc encoded in bytes). I would disagree that this is the only useful view of information. And certainly, from a Shannon information perspective, the network structure contains information (in the Shannon sense) about the environment or problem for which it was optimized.
  - lngnmn 10y ago
    > the network structure contains information (in the Shannon sense) about the environment ... the way a map represents a territory instead of how a guide book does ;) Or the way a 3D structure made out of the same 20 joined amino-acids could represent either an enzyme (the code) or a protein (the data) and that there is no distinction between code and data (which is the great insight realized in the original Lisp. It is not a coincidence that Lisp was the language or the classic AI. Sorry for the reminiscence - just love this part).