5 ms·
All that fuzz and in the last figure showing their touted great invention being beaten by vanilla Transformers, when their invention takes more parameters, more
by toxik 3y ago
All that fuzz and in the last figure showing their touted great invention being beaten by vanilla Transformers, when their invention takes more parameters, more FLOPs, and is not parallelizable. Did I miss something or is this absolutely useless?
- sterlind 3y agoI found the discussion of meta-optimization in existing transformers to be pretty fascinating, but it looks like they're not the first to discover it.
- toxik 3y agoWell it’s fascinating /if true/, but it’s not really proven and that’s why the experiments are so disappointing to me. They put forward this grand theory and then are entirely unable to capitalize on it or make some kind of useful prediction from it.