Networks of perceptrons are universal in the standard sense used when talking about circuits --- they can compute any finite Boolean function. I agree that the

Networks of perceptrons are universal in the standard sense used when talking about circuits --- they can compute any finite Boolean function. I agree that the relationship between circuit complexity and Turing machines is somewhat subtle, for the reasons you mention. The relationship is greatly clarified by the notion of uniform circuit complexity, which makes it possible to prove an equivalence between a (carefully defined notion of) circuit complexity and Turing machine complexity. Unfortunately, I don't know of a good online treatment of uniform circuit complexity. I learnt it through a 1993 paper by Andy Yao, but that's definitely not a good introductory reference! In any case, in my book I'm using the term universal in the same way as people usually use it for circuits, i.e., it means the same thing as when people say that the NAND gate is universal for computation. Hope that clarifies things.