4 ms·

I really wish that CS would switch to discussing related literature before (or with) contributions. The authors only cite themselves until the very end of the p

by zwaps 3y ago

I really wish that CS would switch to discussing related literature before (or with) contributions. The authors only cite themselves until the very end of the paper, and the whole time you are thinking: "Hasn't this been done before? Transformer-XL (recurrent Memory) is a years old paper!" And you need to read to the very and to find out that the contribution is actually: Keep your architecture (but still need to train it on specific tasks)... It's like a blog-post recipe in research paper form

ad404b8a372f2b9 3y ago
I feel like it used to be much more common to have "Related Works" as part 2 or way earlier in papers in earlier years.
macleginn 3y ago
They cite the paper where the architecture was introduced. If you go to that paper, you'll see that it mostly consists of a very detailed and careful comparison with Transformer-XL. In the new paper, they plug their memory system into vanilla BERT. This makes the resulting model essentially nothing like Transformer-XL, which was a strictly decoder-only generative language model.