3 ms·
Turing completeness requires the presence of at least three operations: addition, negation, branching. A usual Neural Network (NN) implements only two operatio
by garganzol 3y ago
Turing completeness requires the presence of at least three operations: addition, negation, branching.
A usual Neural Network (NN) implements only two operations though: addition and negation. The branching operation can be achieved by:
1) recursion as in Recursive Neural Network (RNN), -or/and-
2) some kind of a conditional intermediary loop that occurs between NN runs.
- drpixie 3y agoYou might be a little pessimistic about needing 3 operators. Turing's original machines have only decision making and substitution. With that you can emulate anything you want, and go on to higher levels and more and more complexity. Apparently even Conway's Game of Life is Turing complete ... it's horribly obtuse but can be used to build something, which is used to build something, and so on, until eventually you have an automata that could compute anything computable (though rather slowly!)
- garganzol 3y agoI'm neither pessimistic nor optimistic. The formal criteria of Turing completeness are dictated by the math. If a system can implement those or equivalent ops in a disguised form - then it is Turing complete.
- Xen9 3y agoEducative comment!
- jmalicki 3y agoSoftmax in attention is, well, a smoothed max function, so it gets you branching kinda... You're choosing which memory cell to use.