4 ms·

On one level, yes you’re right. Computing weights and propagating values through an ANN is well defined and very algorithmic. On the level where the learning i

by bwoj 2y ago

On one level, yes you’re right. Computing weights and propagating values through an ANN is well defined and very algorithmic. On the level where the learning is done and knowledge is represented in these networks there is no evidence anyone really understands how it works. I suspect maybe at that level you can think of it as an algorithm with unreliable outputs. I don’t know what that idea gains over thinking it’s not algorithmic and just a heuristic approximation.

dTal 2y ago
"Heuristic" and "algorithmic" are not antipodes. A heuristic is a category of algorithm, specifically one that returns an approximate or probabilistic result. An example of a widely recognized algorithm that is also a heuristic is the Miller-Rabin primality test. https://xlinux.nist.gov/dads/HTML/heuristic.html https://xlinux.nist.gov/dads/HTML/heuristic.html https://xlinux.nist.gov/dads/HTML/millerRabin.html https://xlinux.nist.gov/dads/HTML/millerRabin.html https://en.wikipedia.org/wiki/Miller%E2%80%93Rabin_primality_test https://en.wikipedia.org/wiki/Miller%E2%80%93Rabin_primality...
seanhunter 2y ago
“Algorithm” just means something which follows a series of steps (like a recipe). It absolutely does not require understanding and doesn’t require determinism or reliable outputs. I am sympathetic to the distinction that (I think) you’re trying to make but ANNs and inference are most certainly algorithms.
xpe 2y ago
> On the level where the learning is done and knowledge is represented in these networks there is no evidence anyone really understands how it works. It is hard to assess the comment above. Depending on what you mean, it is incorrect, inaccurate, and/or poorly framed. The word “really” is a weasel word. It suggests there is some sort of threshold of understanding, but the threshold is not explained and is probably arbitrary. The problem with these kinds of statements is that they are very hard to pin down. They use a rhetorical technique that allows a person to move the goal posts repeatedly. This line of discussion is well covered by critics of the word “emergence”.

by bwoj 2y ago

dTal 2y ago

"Heuristic" and "algorithmic" are not antipodes. A heuristic is a category of algorithm, specifically one that returns an approximate or probabilistic result. An example of a widely recognized algorithm that is also a heuristic is the Miller-Rabin primality test. https://xlinux.nist.gov/dads/HTML/heuristic.html https://xlinux.nist.gov/dads/HTML/heuristic.html https://xlinux.nist.gov/dads/HTML/millerRabin.html https://xlinux.nist.gov/dads/HTML/millerRabin.html https://en.wikipedia.org/wiki/Miller%E2%80%93Rabin_primality_test https://en.wikipedia.org/wiki/Miller%E2%80%93Rabin_primality...

seanhunter 2y ago

“Algorithm” just means something which follows a series of steps (like a recipe). It absolutely does not require understanding and doesn’t require determinism or reliable outputs. I am sympathetic to the distinction that (I think) you’re trying to make but ANNs and inference are most certainly algorithms.

xpe 2y ago

> On the level where the learning is done and knowledge is represented in these networks there is no evidence anyone really understands how it works. It is hard to assess the comment above. Depending on what you mean, it is incorrect, inaccurate, and/or poorly framed. The word “really” is a weasel word. It suggests there is some sort of threshold of understanding, but the threshold is not explained and is probably arbitrary. The problem with these kinds of statements is that they are very hard to pin down. They use a rhetorical technique that allows a person to move the goal posts repeatedly. This line of discussion is well covered by critics of the word “emergence”.