3 ms·
Reasonable point - in the case of parity, one bit of recurrent state avoids the combinatorial issues mentioned in OP (as OP more or less says in writing the fin
by mturmon 3y ago
Reasonable point - in the case of parity, one bit of recurrent state avoids the combinatorial issues mentioned in OP (as OP more or less says in writing the finite state automaton).
It's interesting to think about parity in relation to the image classification tasks that sparked the ANN renaissance. In image classification, the class does not strongly depend on a single feature - you don't change from cat to dog by altering one pixel, and then back to cat by altering a different pixel.
But parity does depend strongly on each bit in the input.
I think it's important to keep this special aspect of the parity problem in mind when thinking about good architectures for various applications.
- yarg 3y agoAs I noted elsewhere, the structure's not specific to parity/XOR but works on n-ary merge operations in general. It will impact the breadth and depth required for the module; the {2, 2, 1} module structure required for XOR is not generally useful - but it's a substitutable component. A bigger problem is keeping your processor fed with the decaying layer size (you'll want to target a fixed wave size if you're running on a GPU, mostly empty waves are a bad thing), but you can get around that reasonably easily (at least for base-2).