4 ms·
The ~150 line implementation[0] mentioned in the post looks great. Anyone know of a similarly small implementation for transformers? [0]: https://github.com/B
by Buttons840 3y ago
The ~150 line implementation[0] mentioned in the post looks great.
Anyone know of a similarly small implementation for transformers?
[0]: https://github.com/BlinkDL/ChatRWKV/blob/main/RWKV_in_150_lines.py https://github.com/BlinkDL/ChatRWKV/blob/main/RWKV_in_150_li...
- ronsor 3y agokarpathy's nanoGPT is a good one