I intended these questions as broad examples though some are specific to a given AI system or architecture. You're quite right in that it depends on the task an

I intended these questions as broad examples though some are specific to a given AI system or architecture. You're quite right in that it depends on the task and the model. I was thinking specifically about neural machine translation (NMT) with the encoder-decoder architecture. The encoder-decoder architectures converts a sentence in language X into a fixed size vector representation then aims to "expand" that vector into the equivalent sentence in language Y. As the vector size is fixed, short sentences can be adequately represented but longer sentences end up being lossily compressed. This realization illustrated a fundamental limitation in the encoder-decoder architecture and motivated the use of attention mechanisms. There's a great figure (Figure 1) in Nvidia's Introduction to Neural Machine Translation[1] that shows the dramatic drop in accuracy with respect to the sentence's length. [1]: https://devblogs.nvidia.com/parallelforall/introduction-neural-machine-translation-gpus-part-3/ https://devblogs.nvidia.com/parallelforall/introduction-neur...