3 ms·
Done 1. It is a drawdropper! Especially if you have done the rest of the series and seen results of older architectures. And I was like “where is the rest of it
by quickthrower2 3y ago
Done 1. It is a drawdropper! Especially if you have done the rest of the series and seen results of older
architectures. And I was like “where is the rest of it, you ain’t finished!” … and then … ah I see why they named the paper attention is all you need.
But even the crappy (small 500k param
IIRC) Transformer model trained on a free colab in a couple of minites was relatively impressive. Looking at only 8 chars back and train on a HN thread it got the structure / layout of the page pretty good, interspersed with drunken looking HN comments.