4 ms·
I don't want to be dismissive, it's a fun project, but this has been done a lot already - maybe not with llama3 but the architecture is basically the same as ll
by andy99 2y ago
I don't want to be dismissive, it's a fun project, but this has been done a lot already - maybe not with llama3 but the architecture is basically the same as llama2. Look at the big list of from scratch implementations on Karpathys llama2.c page.
Is there something particularly different about this one?
Edit - guess not?
- fifilura 2y agoI think they learned a lot doing this? And they tried hard explaining each step!
- rvz 2y agoWell given the fast pace of AI, it should not be a surprise that this is similar to llama2 and that we’re seeing the n + 1 toy implementations and likely has bugs or leaks in the background. You might as well look at llama.cpp for a serious and production grade implementation to learn from. Otherwise, nothing to see here. > Is there something particularly different about this one? Other than the immature lowercase, anime BS, etc, then… No.
- tildef 2y agoThere's literally an image of Anya pointing at Karpathy on this GitHub page.
- _giorgio_ 2y agoWhat are your favourite implementations of a GPT? I like a lot the video series by Karpathy. Anyway, I'll take a look at this too, not sure if it has inference and training. Having just inference would be a disappointment.