> feed forward networks do not have immediate state. Not in any relevant sense. Feed-forward networks do have state, but all the useful parts all obtained thro

> feed forward networks do not have immediate state. Not in any relevant sense. Feed-forward networks do have state, but all the useful parts all obtained through explicit training (ye olde backprop, ye older hebbian). The typical scenario is "train model (write mode), deploy model (read-only mode)," which as you point out, has no "thought" since at runtime, no changes or introspections are happening. > So therefore they can never have thought. The key idea here would be: generative models. Most current AI fads are driven by discriminative models (image recognition, speech recognition, etc) which provide very narrow "faster than human" output, but, as you point out, have no thought or will or motives of their own. But, once you have a sufficiently connected network, you can start to ask it open-ended questions ("draw a cat for me") in the form of sampling from the network (gibbs sampling, MCMC, ...) and it fills in the blanks. The extra oomph of providing actual agency and intent and desire to the model is an exercise left to the reader. > (which should be a slower, longer term process). Sleep is a requirement of all things with neural network based brains as far as we know.