> From what I see and understand, even if you have multiple layers of perceptrons and back propagation. It is still really "deeply lacking" because you are not

> From what I see and understand, even if you have multiple layers of perceptrons and back propagation. It is still really "deeply lacking" because you are not able to train networks with all context there is. I don't know what you're trying to say here. NNs can have plenty of context. Let's take just the Transformer architecture for concreteness, ignoring all the others. Something like GPT-2 uses a fixed window, but it can be expanded arbitrarily widely if you're willing to pay the compute price; the scaling can be improved within that window by not computing attention over the full window, per Sparse Transformers; recurrency can be added to make it more RNN-like, as does TransformerXL; compressed versions of the memory can be stored and fed back in, allowing lookback arbitrarily far, per Compressive Transformers; multi-modal input, like images + embeddings + text, are of course completely possible and can represent arbitrary metadata and context; there are many different ways to add in external memory stores (DeepMind regularly experiments with these, like the DNC); and so on. What exactly do you think is impossible in principle in the connectionist paradigm to make NNs do (and why do you think your brain is exempt from this)?