It becomes a lot easier to think about these circuits if you realize that many of the connections are actually spurious. That is, if you remove them and systema

It becomes a lot easier to think about these circuits if you realize that many of the connections are actually spurious. That is, if you remove them and systematically test the function over a range you'll realize that many of these weights have no information bearing value. For instance, if you have a w_i_j=0.01 this may be totally insignificant for any sigmoidal function and for all intents and purposes should be w_i_j=0 in which case this complex web of neural connections is trimmed, resulting in the underlying circuitry. If you're interested I wrote a paper in Nature Systems Biology that shows this for artificial gene networks (just another branch of ANNs): Survival of the Sparsest: Robust Gene Networks are Parsimonious