3 ms·
more like IOCCC+codegolf style implementation of GPT2 http://www.ioccc.org http://www.ioccc.org
by slt2021 2y ago
more like IOCCC+codegolf style implementation of GPT2
http://www.ioccc.org http://www.ioccc.org
- hnfong 2y agoThere is actually an IOCCC entry in 2019 that does almost exactly this except it’s LSTM instead of transformers: https://www.ioccc.org/2019/mills/prog.c https://www.ioccc.org/2019/mills/prog.c https://www.ioccc.org/2019/mills/hint.html https://www.ioccc.org/2019/mills/hint.html
- benob 2y agoThis one is actually impressive: implements Adam training of RNNs, LSTMs, GRUs.