9 ms·
A ChatGPT clone, in 3000 bytes of C, backed by GPT-2 (2023)
- ab_testing 2y agoDid anybody run it locally to see what kind Of output is generated by this GP2 running locally.
- genewitch 2y agoafter reading it, this should output the exact same output as whatever gpt2 model you load normally and in this program, given the same temperature and seed i didn't see those myself in the code, but i was mostly trying to figure out why it was obfuscated; i think the unobfuscated code isn't too terribly much longer (10,000 chars?) and would still be impressive to behold on a screen.
- vunderba 2y agoThe regular code is linked in the article: https://github.com/carlini/c-chat-gpt-2/blob/main/c_chat_gpt_2.c https://github.com/carlini/c-chat-gpt-2/blob/main/c_chat_gpt...
- EvaJohnson24 2y ago[dead]
- dailykoder 2y agoKind of always gives me the same output over and over again. It's kinda neat though. Have to look into it and tweak it myself. Wanted to play around with gpt2 locally for quite some time. Thanks for the post!
- tomcam 2y agoJart is already hard at work on a version that’s only 2070 bytes but includes a full lisp interpreter for the prompt parser
- cowsaymoo 2y agoThat fits inside a QR code. Artificial brain scan.
- hilti 2y agoNever heard of Jart - who or what is it?
- ryankrage77 2y agoJustine Tunney, https://github.com/jart https://github.com/jart
- hmottestad 2y agoIs GPT-2 instruction tuned so that it can actually be used for chat? Otherwise I feel it’s more than just a stretch to call this a ChatGPT clone.
- motorest 2y agoFrom the article: > You can then use this to create something like Chat GPT---just so long as you don't care about the quality of the output. (It's actually pretty terrible output, objectively speaking... But it does run.) It's unusable and bears no relationship other than the name-dropping. But it's a program that compiles and runs. By the looks of those in this discussion who are giving high praises about the capabilities of a project whose author admittedly states it doesn't really work, I guess the point is using buzzwords to bait the bandwagon types.
- bippihippi1 2y agothe point seems to be making a small binary to run language model. How useful is that? From a functional perspective, I guess not very but the model can be improved. From a performance / cost perspective, also not very because most of the cost is in training the model and small binary isn't necessarily indicative of quick. I guess it's just kind of interesting that it doesn't take that much code to run the model.
- zanderwohl 2y agoPeople "demake" games all the time. Nobody complains if those games are missing most of the features of the original because the core gameplay loop (the essence) is the same. That's the point here -- you can make a small-scale, low-resolution version of a major program in a very small space. > By the looks of those in this discussion who are giving high praises about the capabilities of a project whose author admittedly states it doesn't really work, I guess the point is using buzzwords to bait the bandwagon types. Instead of assuming everyone else is naive, perhaps consider another perspective?
- hmottestad 2y agoMy thinking was more that ChatGPT is called "Chat" because it allows users to chat with a Generative Pre-trained Transformer.
- hilti 2y agoI haven‘t run the code, but I‘m impressed by the small size. The first ELISA programs have been larger. And within the past 4 years we can fit this into bytes. If someone has a hint where the magic lies in, please explain it to me. Is it the GELU function or the model that‘s downloaded through the bash script?
- 01HNNWZ0MV43FF 2y agoMost of the magic is in the 475 MB model file downloaded through the Bash script.
- dogboat 2y agoAnd probably like 100Gb of training data or whatever :). HN had a heady debate at least once about if that training data is source or not.
- Sharlin 2y agoI guess the training data should be called assets. Analogously to how you can download the Quake source, and build it and get a working executable, but without the (non-open) assets you cannot replicate what you get by buying the game.
- fluoridation 2y agoBy that analogy, the executable code would be the game engine, the model that inference is done on would be the assets, and the training data would be... I guess the PSDs and high-poly models that get compressed and simplified to turn them into game assets.
- Sharlin 2y agoIt depends on whether the analogue of playing a game is training or inference. If the former, then the training data are the assets, the training software is the game, and the resulting model is the game experience (or if you want something more concrete, savegames and recorded play I guess). But yeah, if gameplay equals the use a trained model, then the model is an asset bundle (the pak0.pak if you will) and training data is the original models, textures etc, and the training software is all the programs that are used in the asset production pipeline.
- tgma 2y agoTL;DR: code-golf style C program to do inference on an existing TensorFlow model data for GPT2, not full ChatGPT, nor training or anything.
- jnwatson 2y agoThis isn't code golf. This is just white space removed.
- slt2021 2y agomore like IOCCC+codegolf style implementation of GPT2 http://www.ioccc.org http://www.ioccc.org
- hnfong 2y agoThere is actually an IOCCC entry in 2019 that does almost exactly this except it’s LSTM instead of transformers: https://www.ioccc.org/2019/mills/prog.c https://www.ioccc.org/2019/mills/prog.c https://www.ioccc.org/2019/mills/hint.html https://www.ioccc.org/2019/mills/hint.html
- benob 2y agoThis one is actually impressive: implements Adam training of RNNs, LSTMs, GRUs.
- boiler_up800 2y agoIntelligence as a new fundamental layer! These small programs that do so much really inspire me.
- deleted 2y ago[deleted]
- fulafel 2y ago> (TAKE THAT LANGUAGES WITH PROPER MACROS. LISP ISN'T ALWAYS BETTER THAN C!) Ok, allowed this time as punching up. -- If you missed the code link (it's buried in the text): https://github.com/carlini/c-chat-gpt-2 https://github.com/carlini/c-chat-gpt-2
- go_prodev 2y agoGELU really is like magic: UNARY(GELU, b / 2 * (1 + tanh(.7978845 * (b + .044715 * b * b * b))))
- dogboat 2y agoFast inverse square root lookalike.
- shoo 2y agoYou can hand that GELU definition to a mathematician and they can interpret it as a function of a real number b. The definition does not depend upon b being a floating-point number with a particular bit representation. In contrast, the fast inverse square root really exploits the bit representation of a floating point input to cheaply compute an initial guess.
- jey 2y agoThis is just a practical approximation to the actual mathematical definition of GELU, which is `GELU(x) := x * Φ(x)` where Φ(x) is the CDF of the Gaussian distribution.
- fluoridation 2y agoIsn't that just erf()?
- kgwgk 2y agoThey are related but the error function approaches -1 for large negative numbers. Φ(x) approaches 0 and so does x * Φ(x).
- deleted 2y ago[deleted]
- ethan-j 2y agoAt least it's kind of a conversation. bash ./run.sh AI: How can I help you? Human: who are you AI: I am alice. Human: what can you do for me? AI: I can help you. Human: how to say bird in Chinese? AI: bird in Chinese. Human: 2+2=? AI: 2+2= bird.
- dogboat 2y agoThis is what it was like to live in 2019
- yard2010 2y agoHow long would it take for people to accept this as the ground truth fighting and dying for this, starting the next cult/religion?
- gravitronic 2y agoWell my kids are already convinced 1+1=window
- kandros 2y agoAll hail the 4 bird
- portaouflop 2y agoScholars are still fighting whether it’s 4bird or 2+2bird. Bitter clashes between 4 and 2+2 birders around the world resulted in dozens injured, promoting the UN to call for peace between the two churches
- justincormack 2y agoIt is both, the holy binity.
- marai2 2y agoEveryone knows (2+2bird) times it’s complex birdy conjugate (2-2bird) == 8bird
- paxys 2y agoWhat's the point of minifying code that is anyways going to be compiled?
- byzantinegene 2y agoI'm assuming 3000 bytes refers to the uncompiled minified code
- exac 2y agoIndeed, and the question is why do that? We can count LOCs just fine already.
- thrw42A8N 2y agoBecause it'd be very hard to compare short program length otherwise? 5 LoC says nothing.
- ffsm8 2y agoTechnically, it depends on the language. I.e. python doesn't have anything like a semicolon to combine multiple statements and assignments per line But yeah, for C the LOC metric can be gamed to a silly degree
- mattashii 2y ago> python doesn't have anything like a semicolon to combine multiple statements Python does have a "semicolon to combine multiple statements", and has further (lambda f: f(f)) expressions for complex expressions with local names and scopes. (not that using either for those would result in pythonic code, but it is certainly not missing from the language)
- ffsm8 2y agoOh, you're right. I should've tested that before writing my comment. Thanks for calling me out
- ljouhet 2y agoI love the way it's written: see this ChatGPT-like code that fits in your screen? Let's break it down!
- demon08 2y ago[flagged]
- sweca 2y agoBecause it's novel and interesting. This person wrote a super optimized dependency free implementation of GPT in C.
- menaerus 2y agoIt's not really novel nor super-optimized. Many inference engines are out there. But interesting it is. Probably a good learning material too.
- cnity 2y agoHacker News front page isn't a trophy ceremony for smart AI models.
- hk__2 2y ago> Isn't the whole point that the models hosted by OpenAI are smarter? That’s not the point here, because as the link says (3rd paragraph): "It's actually pretty terrible output".
- owenpalmer 2y ago"I am not a madman for saying that it is likely that the code for artificial general intelligence is going to be tens of thousands of lines of code, not millions of lines of code. This is code that conceivably one individual could write, unlike writing a new web browser or operating system." - John Carmack, Lex Fridman Podcast (August 4th, 2022) This was around 3 months before ChatGPT's initial release. Timestamped: https://www.youtube.com/watch?v=I845O57ZSy4&t=14677s https://www.youtube.com/watch?v=I845O57ZSy4&t=14677s
- Buttons840 2y agoTime for another Year of Linux (TM) prediction: Linux (and friends) do a lot of things on the command line. LLMs are good at writing text and using a command line interface. Creating LLMs and AIs takes relatively little work and is interesting, open source is especially good at this type of work. Therefore, I predict that Linux will keep pace with other operating systems when it comes to voice control.
- ForOldHack 2y agoTime for another prediction that a computer will have a PhD, by the end of the decade. 1960, 1970, 1980, 1990, 2010, 2020, 2030. Next, yawn.
- yreg 2y agoAttention is all you need and GPT-2 were well known at that point. Many might doubt whether this approach leads to "general" intelligence – depends on the definition. BTW, Karpathy has a nice video tutorial about building an LLM: https://www.youtube.com/watch?v=kCc8FmEb1nY https://www.youtube.com/watch?v=kCc8FmEb1nY
- mjburgess 2y agoHis claim is just that AGI will largely be comprised of stateless mathematical operations -- all of which could be a single line of code if written that way. As a claim, it's antique -- it would easily have been a view of Turing and others many decades ago. And as a claim its as false then as now. Not least because all the code which is the actual algorithm for generating an LLM is all the code that goes into its data collection which is just being inlined (/cached) with weights. However, more than that, it's an extremely impoverished view of general intelligence which eliminates any connection between intelligence and a body. All the "lines of code" beyond a single are concerned with I/O and device manipulation. Thus this is just another way of repeating the antique superstition that intelligence has nothing to do with embodiment.
- anthk 2y agoI've seen better with classical AI chatbots: https://www.cs.cmu.edu/afs/cs/project/ai-repository/ai/areas/classics/eliza/0.html https://www.cs.cmu.edu/afs/cs/project/ai-repository/ai/areas... Splotch will compile it fine on modern unixen with few changes.
- stavros 2y agoI don't know, GPT-2 wrote some of my favorite fairy tales: https://deepdreams.stavros.io/episodes/the-princess-the-fairy-godmother-and-the-chest/ https://deepdreams.stavros.io/episodes/the-princess-the-fair...
- parkaboy 2y agoThat was both impressive, weird, and 90% coherent. Which does give it a special kind of uncanny vibe.
- stavros 2y agoI know, right?! I love it. Unfortunately, GPT-3 and later were much less weird and wonderful, because they were perfectly coherent.
- rcdwealth 2y agoThat is great, and in reality entertaining and for listening and sleeping in, good story. So did you make it with that GPT-2 from this page?
- stavros 2y agoNo, it's three years old. It was made with OpenAI's GPT-2.
- Abhitaneja89 2y ago[dead]
- atiedebee 2y agoI dont see how C macros are anything like regex. They match words and replace them with different text. Regex is about matching text with relatively complex patterns, and by itself doesn't do any text replacement.
- thomasfl 2y ago"While I mostly put this together for fun, it's a nice demonstration how simple neural networks actually are." Psst, don't tell anyone. Artificial Intelligence is the black magic we do to make money.
- benob 2y agoShould try with smollm2 which is one of the smallest instructed models.
- andai 2y agoI remember playing with GPT-2 when it came out. A friend and I exported our chat logs, fine tuned GPT-2 on them, and had it simulate conversations between us. It was hilarious and sometimes disturbingly accurate. I'm wondering what caused the quantum leap between GPT-2 and 3? Bigger model, more data, or both? I know RLHF makes a huge difference, but even the base GPT-3 model (text completion) was very useful, given enough examples.
- robblbobbl 2y agoThanks. For the learning it's really good.
- neomantra 2y agoAre commentors even reading the article or just bitching about how GPT-2 sucks or pedantry of LoC metrics? Look at this article a different way. The author put a lot of information in a small context window so that it easier for readers to understand transformers. They included a useful code highlighter to ground it. To soooo many people, even those with strong math/science, GPT is magic. This article opens the spell book, lays it out as a computation graph with all the fixing’s. The code isn’t abominable especially when paired with the prose. It is a good piece of education and I will also call it art.
- FrustratedMonky 2y agoAlso, demistifying the 'complexity'. I see so many articles saying, or assuming, that 'those software people know what is happening', because it is programmed. I hear 'well they programmed it, so they know what is going on right?'. This is pretty clearly showing, look it isn't a lot of code, and look what happens. Even if GPT2 isn't all that great, look at the amount of code, it is small, not some pile of millions of lines of complex programming to make it work.
- deleted 2y ago[deleted]
- zxspectrum1982 2y agoThese days, you can easily implement your own ChatGPT in a snap by using gptscript: https://github.com/gptscript-ai/gptscript https://github.com/gptscript-ai/gptscript
- deadbabe 2y agoSo will it ever be possible to do something similar with ChatGPT 3.5 or so?
- landswipe 2y ago[flagged]
- deleted 2y ago[deleted]
- deleted 2y ago[deleted]
- tmaly 2y agoWhat type of hardware could this run on? Could quantized weights on huggingface be used with this? What type of problems or queries would this be really good at?