6 ms·
I found the discussion of meta-optimization in existing transformers to be pretty fascinating, but it looks like they're not the first to discover it.
by sterlind 3y ago
I found the discussion of meta-optimization in existing transformers to be pretty fascinating, but it looks like they're not the first to discover it.
- toxik 3y agoWell it’s fascinating /if true/, but it’s not really proven and that’s why the experiments are so disappointing to me. They put forward this grand theory and then are entirely unable to capitalize on it or make some kind of useful prediction from it.