7 ms·
Teaching Large Language Models to Self-Debug
- alecco 3y ago3 Google researchers using Open AI GPT-3 code-davinci-002, interesting.
- ulrikhansen54 3y ago'Unsupervised reinforcement learning' is how these large models and systems ultimately will end up becoming sentient. We recently tried a similar approach on a toy problem in the computer vision sphere (https://encord.com/blog/we-employed-chatgpt-as-an-ml-engineer-this-is-what-we-learned/ https://encord.com/blog/we-employed-chatgpt-as-an-ml-enginee...) with pretty decent results.
- ChatGTP 3y agoWhen it attains sentience, will it wake up, sing dixie and finally defeat communist China and a Russia once and for all, and then finally proceed to grant Silicon Valley elites eternal life and then turn itself off ?
- ulrikhansen54 3y agoI bloody hope so...
- cs702 3y agoIn hindsight, it's the most natural, most obvious next step to get LLMs to write better code: Explain to them how to debug and fix the code they've written. Which is pretty much what you would do with an inexperienced human software developer. Looking at this with fresh eyes, it's both shocking to me that this sort of thing is even possible, and yet also completely unsurprising as yet another emergent capability of LLMs. We live in interesting times.
- famouswaffles 3y agoNot too shocking for me after this paper. https://arxiv.org/abs/2211.09066 https://arxiv.org/abs/2211.09066 You can teach GPT-3 arithmetic - https://imgur.com/a/w3DAYOi https://imgur.com/a/w3DAYOi Basically 100% accuracy up to about 13 digit addition and >90 after that. What else can you teach GPT without changing weights ?
- mirashii 3y ago> 100% accuracy up to about 13 digit addition The graphs you just posted do not support that, they'd support at most 100% accuracy up to 4 digits.
- famouswaffles 3y agoIt's 100 at 13 and extremely close to it prior to that. Maybe basically 100 is better.
- sharemywin 3y agoit's GPT so 13=4
- cs702 3y agoI meant shocking in the sense that it makes me gape in awe, but as I wrote, it's also, simultaneously, completely unsurprising given all the new emergent capabilities we keep discovering. We're in agreement :-)
- famouswaffles 3y agoOh. yes well that's fair
- gopalv 3y ago> and >90 after that This is such a circular thing, that I feel like it is amazing to see it. The reason LLMs use a NN is because they're trying to encode a probability function for generating the passage. And now, you are encoding another n-gram follower exercise (i.e 1+1 = 2) on top of it :)
- matisseverduyn 3y agoUseful, but still wouldn't count on it. With respect to GPT etc. as a copilot, the current dialogue seems to focus on "ask for GPT to generate code to do X" then "just paste in the error message to fix bugs in the code GPT generates" A.) Why is GPT generating code that results in simple compiler errors (that is why GPT probably shouldn't be used to generate any code / replace devs for real projects yet), and B.) error messages are (just guessing here) probably <1% of the actual errors in most codebases. I personally know of a few large companies laying off devs over this. IMO, the tech debt we're going to see in 6 months will probably be huge. Good now to start a staffing agency of human experts who can come in and fix this type of problem (extricating massive amounts of code generated by GPT without starting from scratch) because there will be a bunch of fires to put out and those fires will be worth $
- viscanti 3y agoIf an LLM hallucinates lines of code that can't even compile, I suppose it could also hallucinate logic issues which are more difficult to track down.
- matisseverduyn 3y agoDefinitely. QA at a snails pace should still be the focus here for a while, but that's not what I'm observing in the real world. Just rush, pressure, layoffs. At least this sort of behavior keeps humans employed long-term.
- runlaszlorun 3y agoI have limited experience even trying. But I did try it for some fundamental JS Web API stuff sans framework or library like IndexedDB, web sockets, and a basic, basic todo like app. Neither of those three would function nor would they throw an error. Prompts to correct itself would not improve things. So I did the natural thing and started to debug myself. At which point, I couldn’t help but ask myself why I was debugging machine generated code when I could not be lazy and actually build it from first principles.
- david2ndaccount 3y ago
- Imnimo 3y agoI'd be curious to know if having few-shot prompts that demonstrate making mistakes and then correcting them causes the model to make more initial mistakes so that it has something to correct. Like as far as the model is concerned, how can it distinguish between the task being "do your best but if you do make an error, correct it" and "make some mistakes like in this example and then fix them".
- Buttons840 3y agoAh we're starting to bootstrap. For decades in reinforcement learning we've had Q learning, which promises to solve any optimization problem if only we can build a powerful enough function approximator. It can even learn off-policy, meaning it can just watch from the sideline and find the optimal solution. It works for toy problems, and it works in theory, theres even formal proofs that it will work given infinite time and resources, and yet in practice it often becomes unstable and collapses. Supervised learning is one thing, having a model remain stable while bootstrapping through a complex environment is another. GTP is supervised learning, so far, let's see if it can bootstrap.
- civilized 3y agoI've done several experiments (and posted results in previous HN comments) where I've given GPT puzzles or brainteasers and asked it to review aspects of its answers Socratically. Never telling it it got anything wrong, just "you said A, then you said B, does that make sense"? It usually does notice inconsistencies between A and B when asked this. But its ways of reconciling inconsistencies can be bizarre and suggest a very superficial understanding of concepts. For example, it once reconciled an inconsistency by saying that, yes, 2 * 2 = 4, but if you multiply both sides of that equation by a big number, that's no longer true. I will be super impressed the day we have a model that can read an arithmetic textbook and come out with reliable arithmetic skills.
- sharemywin 3y agoin computer logic you would get an undefined if the number was large enough.
- civilized 3y agoIt doesn't work with numbers as computer numbers though. It works with them as decimal digit strings, just like humans do.
- Paul-Craft 3y agoMake the number you multiply by essentially the concatenation of a long series of random digits, and I can just about guarantee most humans will get different things on both sides, because they'll make one or more mistakes doing the math. That is, of course, assuming the humans don't have suitable traditional computer tools capable of handling such a scenario.
- civilized 3y agoNot sure how this is relevant to the discussion.
- 3y ago
- ftxbro 3y ago> "We evaluate SELF-DEBUGGING on code-davinci-002 in the GPT-3 model family" Putting aside the incongruity of Google researchers using the OpenAI model, I'm curious how GPT-4 would do in this situation. Probably its zero shot attempts at coding would be better, and maybe its self criticisms would be better too.
- astrange 3y agoGoogle's recent LLM agent paper also used ChatGPT.
- cloudking 3y agoGPT-4 in ChatGPT Plus can do this fairly well for coding tasks, I've had numerous cases where the code it produces has bugs initially. However, after a few rounds of passing the errors back in the chat it's usually able to correct it's own code.
- runlaszlorun 3y ago> Self-Debugging with code explanation consistently improves the baseline by 2-3% I’ll admit that I only have had time so far to read the abstract, and I’m not sure what their baseline is, but a 2-3% improvement doesn’t sound like a quantum leap forward that you’d expect from the title. Heck, I’d think that’s likely within expected sampling errors. I’m not sure about others’ experience and, while I keep reading articles showing impressive seeming examples, my few forays into attempting to get ChatGPT to write code were actually completely useless. Even with follow on prompts to correct itself. The other day I asked it what covid case fatality rates were in 2020. After all the various opinions at the time, I was curious to see what it was pre-vaccine. It would alternately tell me that it couldn’t give me data for 2020 because it only had data up to Sep. 2021, and then give me wildly varying numbers. Is this a Rocko’s Basilisk trying to lure me into a false sense of security… haha.
- ChatGTP 3y ago…yes
- rhyme-boss 3y agoA warning siren goes off in the background. Another step towards recursive self-improvement.
- sowbug 3y agoSo is this the singularity?
- goatlover 3y agoI doubt it without being able to evolve the weights, architectures and train those on new domains. Humans are still doing all of that. It's just a powerful tool people are using for a bunch of things. But it doesn't do anything on it's own, and it doesn't care.
- int_19h 3y agoAt this point, does anything actually preclude telling a model connected to a service running Python with all the requisite libraries (or access to PyPI / conda) to try "designing" another LLM, from architecture and choice of training data to writing actual training code?
- goatlover 3y agoNothing, and I'm sure people are trying that, but it's probably beyond the abilities of current LLMs to make progress in this area, since it's kind of outside their training.
- int_19h 3y agoIs it, though? Codex was trained on Python code mostly, and given the cutoff date, it has all the relevant ML bits. And if you ask it about things like transformers, self-attention etc, it's clear that papers on that were a part of its training set, as well. The question is whether it can reason well enough about it to produce meaningful improvements. FWIW if you ask it how transformers could be improved, it basically gives a recap of various relevant papers that it knows: Longformer, BigBird etc. Amazingly, all the papers cited actually exist and have the specified titles, names, and dates. But then if you ask it for some ideas of its own: > Can you think of other novel improvement ideas that haven't been investigated yet, to the best of your knowledge? What makes the most sense based on your understanding of the subject matter. As an AI language model, my knowledge is derived from the text data I have been trained on, which includes research papers and articles published up to September 2021. While it's difficult to guarantee the novelty of any idea, here are a few suggestions for potential improvements that may not have been thoroughly investigated yet: Context-Adaptive Layers: Instead of using the same set of layers for all input sequences, consider using context-adaptive layers that are dynamically selected or generated based on the input's characteristics. This could allow the model to adapt its processing strategy to different input types, such as formal vs. informal text, or narrative vs. expository text, which might improve its performance on diverse tasks. Multi-modal Fusion: Develop techniques to better fuse information from multiple modalities, such as text, images, and audio, within the Transformer architecture. Improved multi-modal fusion could lead to enhanced performance on tasks that require understanding and generating content across different modalities, such as visual question answering or image captioning. Meta-Learning for Architecture Search: Employ meta-learning techniques to search for the best Transformer architecture for a specific task or domain. By learning to optimize the architecture itself, the model might be able to discover more effective configurations or components that have not been considered before. Transformer Modules for Specific Tasks: Design specialized Transformer modules tailored for specific NLP tasks, such as syntactic parsing, named entity recognition, or sentiment analysis. These task-specific modules could be combined with the generic Transformer architecture to create a more powerful and adaptable model. Hierarchical Transformers: Explore hierarchical architectures where multiple layers of Transformers work together to process information at different levels of abstraction. This could help the model capture both low-level patterns, such as local dependencies, and high-level patterns, such as long-term dependencies and semantic relationships, more effectively. Enhanced Tokenization Strategies: Investigate novel tokenization strategies that better capture the structure and meaning of the input text. This could involve using character-level, subword-level, or even syntax-based tokenization to better represent the input for the Transformer model. Regularization Techniques: Develop novel regularization techniques specifically tailored for the Transformer architecture. This could help improve generalization and prevent overfitting, especially when training large-scale models with billions of parameters.