5 ms·
C++ GPT-2 inference engine
- version_five 3y agoContext: https://twitter.com/a1k0n/status/1674644631156555780 https://twitter.com/a1k0n/status/1674644631156555780 "I reimplemented GPT-2 from scratch in C++ as an exercise to really understand the nuts and bolts of LLMs. GPT2-117M isn't a super great model, but it's extremely satisfying to get it to generate basically the same thing as other reference implementations." "I" refers to the guy that wrote this, I, version_five have nothing to do with it, I just thought it looked cool.
- metiscus 3y agoIndeed it does, thanks for posting this.
- eugenhotaj 3y agoThis is pretty cool. I had the same idea but in zig: https://github.com/EugenHotaj/zig_gpt2 https://github.com/EugenHotaj/zig_gpt2 Not fully finished yet, haven't gotten around to implementing bpe encoding/decoding and only some ops use BLAS.
- eclectic29 3y agoI found this extremely helpful even though it's not in C++: https://jaykmody.com/blog/gpt-from-scratch/ https://jaykmody.com/blog/gpt-from-scratch/.
- JPLeRouzic 3y agoYes it's very nice from the author to provide this very readable text. I saved this text to read it later.
- kken 3y agoFrom the same twitter thread, a dense implementation in less than 100 lines of plan C: https://github.com/davidar/eigenGPT/tree/c https://github.com/davidar/eigenGPT/tree/c
- rhelz 3y agoWell, I downloaded and compiled it (cool! Thanks!) but no matter what prompt I give it, it just prints out gibberish....where do I go now to learn how to properly use it?