Love it. Especially the echo state network trick. I wonder how much of BERT/ELMO performance is simply due to them having a such a high dimensionality. Not tha

Love it. Especially the echo state network trick. I wonder how much of BERT/ELMO performance is simply due to them having a such a high dimensionality. Not that there is anything wrong with that, just makes a tad less practical for some applications.