4 ms·
The linked paper tested nanoGPT with this new transformer: https://www.techrxiv.org/users/685780/articles/1375955-topological-transformer-a-redesign-for-domain
by naasking 9mo ago
The linked paper tested nanoGPT with this new transformer:
https://www.techrxiv.org/users/685780/articles/1375955-topological-transformer-a-redesign-for-domain-memory-and-cheaper-kernel-operations https://www.techrxiv.org/users/685780/articles/1375955-topol...
- tuned 9mo agothanks for linking. Yes the paper compares the new architecture (that is also a fork of my implementation of nanoGPT) with Karpathy's nanoGPT. There are also links to the code and bench used.
- Herring 9mo agoNote I didn't say Karpathy's nanoGPT, I said use the speedrun. Transformers are universal function approximators. When well-tuned, they often start to approximate other innovations. Not always, thank god, but often enough that you have to be careful.
- tuned 9mo agook, thanks. I am taking it slow then