3 ms·
I wrote a C++ translation of it: https://github.com/verma7/microgpt/blob/main/microgpt.cc https://github.com/verma7/microgpt/blob/main/microgpt.cc 2x the numbe
by verma7 7mo ago
I wrote a C++ translation of it: https://github.com/verma7/microgpt/blob/main/microgpt.cc https://github.com/verma7/microgpt/blob/main/microgpt.cc
2x the number of lines of code (~400L), 10x the speed
The hard part was figuring out how to represent the Value class in C++ (ended up using shared_ptrs).
- WithinReason 7mo agoI made an explicit reverse pass (no autodiff), it was 8x faster in Python
- love2read 7mo agoCan you share a link?
- WithinReason 7mo agohttps://www.ideone.com/VAz4Nn https://www.ideone.com/VAz4Nn Doesn't run inside IDEone due to the external download link, but you can copy&paste the code over
- hu3 7mo agoI made an explicit double-reverse pass (no code!), it was 80x faster in my head!
- spopejoy 7mo ago"I've got an ipod -- In My Mind" https://theonion.com/i-have-an-ipod-in-my-mind-1819584018/ https://theonion.com/i-have-an-ipod-in-my-mind-1819584018/
- WithinReason 7mo agocode here, it's just not interesting to look at: https://news.ycombinator.com/item?id=47220542 https://news.ycombinator.com/item?id=47220542
- bear3r 7mo agotradeoff worth naming: you avoid the autodiff graph overhead (hence the speedup), but any architecture change means rewriting every gradient by hand. fine for a pedagogical project, but that's exactly why autodiff exists.
- freakynit 7mo ago24x speedup (over 10x already) and similar loss profile (for c++ version, optimized by claude): https://gist.github.com/freakynit/3982eab8413a89941bd0018e63345efb https://gist.github.com/freakynit/3982eab8413a89941bd0018e63......
- verma7 7mo agoThis is amazing! Thanks for optimizing the code using Claude!