There is deep discussion of that topic in the paper. Attention, audio processing, and lexical layers are brought to the forefront of their discussion. To quot

There is deep discussion of that topic in the paper. Attention, audio processing, and lexical layers are brought to the forefront of their discussion. To quote the authors: ”We propose that deep neural networks encode a variety of features…”