> and the slow step in training is the outer product of two matrices.... I don't believe there is an algorithmic way out of that one. Well, there are several,

> and the slow step in training is the outer product of two matrices.... I don't believe there is an algorithmic way out of that one. Well, there are several, but the obvious ones tend to require strange or unrealistic assumptions about the hardware. The most obvious such assumption, IMO, being that the hardware is arranged in 3D space in a manner roughly analogous to a human brain, which tends to be at odds with the common practice of mostly-planar photolithography, and with the preference to be able to change the network topology experimentally without building new hardware.