7 ms·

How to generate tested software packages using LLMs, a sandbox and a while loop

133 points by pierremenard 3y ago

puffyengineer 3y ago
[dead]
DylanSp 3y ago
This definitely seems like a potentially powerful approach. The said, maybe I'm missing something obvious, but is the LLM generating both the tests and the implementation? If that's the case, then it seems like there could be issues caused by the generated tests not matching what's specified in the initial prompt. Manually writing highly focused unit tests doesn't seem like the best way to work with this, but being able to manually write some sort of high-level, machine-checkable specs might be useful.
- mikeravkine 3y ago
  This is exactly what happened when I tried this approach personally: on a non trivial codebase, the tests were just as likely to be wrong as the code itself and during "debugging" the wrong would converge until the test passed but the results were useless. I still have my prompt from before and a Modal account, I'm going to give this one a shot to see if I missed something or if it really only works in the mickey mouse case.
d4rkp4ttern 3y ago
Modal-labs seems like an interesting cloud compute solution that I hadn’t heard of before. Could anyone summarize what are its value props, e.g is it like GCP but better because …
Joker_vD 3y ago
Ah, so we're turning "a million monkeys at a million keyboards could produce the complete works of Shakespeare" into actual, real thing? What a truly disruptive synergy.
- miahwilde 3y ago
  No - we are the monkeys. This is something different.
mike_hearn 3y ago
Very cool. I've played around with something similar a few months ago (without the sandbox). The tricky part is to find a diff format that the AI can use intuitively to patch code in place, as otherwise you can blow out the token limits quite fast. I wonder to what extent AI is going to kill comprehensibility of the infrastructure. Over the weekend I did a bit of hacking with a friend. He wants to move out of AWS to Hetzner to save money, but wasn't experienced with Linux sysadmin so I was teaching him some stuff, and simultaneously pitching a little prototype I made of a server config tool. It's sort of like a higher level Ansible: you specify what packages and containers you want, the URLs they should be served on, and it goes ahead and configures unattended upgrades, Docker and Traefik to deliver that outcome. The idea was to extend it to configuring backups with BorgBackup and other common sysadmin-y tasks. He was enthused! The next day though, things changed. He'd signed up for GPT-4 and it'd just blasted out a pile of Ansible configs for him. He doesn't know Ansible but it didn't matter, he just iterated with the AI a few times and now the problem is solved. This makes me wonder if there's much point anymore in improving the usability of systems software (programming languages, databases, operating systems, clouds, anything driven by textual configuration). A basic assumption that underlies making better tools is that user's time is valuable, that intuitive and simple systems therefore have value. But the AI's time isn't valuable. It can read some poorly written docs, then the source code of the system, then people getting frustrated on StackOverflow, synthesize all that together and spit out whatever pile of inscrutable config files are needed and it can do so within seconds. Given this experience, if we extrapolate it forwards, then maybe within a couple of decades many organizations will be running on infrastructure that nobody understands at all. You get cases today where maybe individual programs or workflows are only understood by one guy, but those are extremes and it's understood that it's a bad situation. Maybe in future it'll be normal and to do anything companies will have to ask an AI to do it for them. We already see skills atrophy around things like UNIX sysadmin because nowadays everyone learns AWS instead, which is one reason they can charge so much money for it, so I think as people retire knowledge of how to run Linux systems will steadily disappear. But conceivably people will stop learning the cloud too, and then to do anything with servers it'll be AI or the highway. You can also apply this to programming languages. Why engage in a high effort project like inventing a Java or Rust if an AI can spit out correct C++ for you, given enough tries at writing tests? Does the motivation to produce anything beyond incremental improvements to existing tools disappear now? I keep flipping back and forth on the answer to this. On one hand, it seems pretty pointless to further develop this prototype now. Maybe Ansible or AI written shell scripts is the end of the road for making Linux easier to use. On the other hand, AI is remarkably human like. It can also make mistakes, and also benefits from well written docs, simple interfaces and good error messages. So maybe usability still has a purpose, albeit maybe we now need to master a new field of "usability studies for AI".
politician 3y ago
Immediately after seeing ChatGPT, I felt that evolving programs in an OODA loop was going to be a big deal, so it's great to see someone iterating on the concept. I hope they try it with QuickCheck / property-based testing.
seeknotfind 3y ago
LLMs when given code mixing languages will typically execute as if it worked anyway. I have a theory this is because correct examples are the majority on the web. Since LLMs don't have the data with the feedback all the humans have seen constructing these examples, there is no way it could understand. We need to feed LLMs these types of input/error loops explicitly to improve performance or reduce iterations in this area. Otherwise it'll be stuck making many small mistakes forever. Again, this is my theory.
hellodanylo 3y ago
This should give a second life to Test-Driven Development. One of the under-appreciated wisdoms of TDD is that there is a complexity asymmetry in many problems between finding a solution and (fully or partially) verifying it. Examples of asymmetric problems: inverting matrices, sorting an array, computing a function's gradient, compressing a byte stream, etc. Human writes the easier part -- the test suite, the language model writes the harder part -- the solution. This can be a net gain in productivity.
- fodkodrasz 3y ago
  Finally something will kill the last somewhat satisfying part of the job finally, for the greater profit of the corporate overlords!
- BaseballPhysics 3y ago
  The problem is without a formal definition of the program semantics, you run the risk of overfitting or uncovered behaviors that, for a human developer who understands the intent of the program, would be implicit. And given how hard formal verification is, I don't know that you'll ever get away with not having to manually check these programs, at which point I question just how much productivity you've gained. It's kinda like self-driving cars: when they work, they work great. But when they fail, they fail in ways a human never would, and therefore a human struggles to anticipate or trust their behaviour. That said, I'm waiting to see the rise of programming languages designed with LLMs in mind, where a human could use contract oriented programming or similar (think: Ada) combined with TDD methods to more formally specify the problem that an LLM is being asked to solve.
  - hellodanylo 3y ago
    > I question just how much productivity you've gained. Me too. It's an empirical question to be answered by those who will dare to try. > It's kinda like self-driving cars Strong disagree. Yes, neural nets are blackboxes, but the generated code can be idiomatic, modular, easy to inspect with a debugger, etc. > more formally specify the problem that an LLM is being asked to solve. That would be a great direction to explore.
amelius 3y ago
No need to fix things, just make it search for leaks or null pointer references and explain them to the user, and you have a million dollar product.
luis_cho 3y ago
This takes programming by coincidence to a whole new level.
- AtlasBarfed 3y ago
  How can this possibly generate maintainable code? And if the tests change, does it have to fully rewrite the code via AI? If you had code that was reasonably working before, but you change the tests and the AI does the "full rewrite" for you, how do you have confidence that the code works? Go ahead, say "it passed the test suite". One of the things that happens in software dev is the devs grokking the overall desire/aims of the software and its users, and often anticipating things that aren't even in the requirements. The test suites are approximations of validating the requirements / derived intent. But what faith do you have in the AI black box, or the black boxes it generates?
choeger 3y ago
Program synthesis by a language model is certainly cool, but I don't think it's really novel. After all, the mutator could as well just be a random AST transformer. So what's the real deal here? Does the mutator end up with a usable program much quicker? Or does it cheat by looking at the test cases? As an example, if I specify a md5 hash function on strings, what will that tool produce?

133 points by pierremenard 3y ago

[dead]

This definitely seems like a potentially powerful approach. The said, maybe I'm missing something obvious, but is the LLM generating both the tests and the implementation? If that's the case, then it seems like there could be issues caused by the generated tests not matching what's specified in the initial prompt. Manually writing highly focused unit tests doesn't seem like the best way to work with this, but being able to manually write some sort of high-level, machine-checkable specs might be useful.

mikeravkine 3y ago
This is exactly what happened when I tried this approach personally: on a non trivial codebase, the tests were just as likely to be wrong as the code itself and during "debugging" the wrong would converge until the test passed but the results were useless. I still have my prompt from before and a Modal account, I'm going to give this one a shot to see if I missed something or if it really only works in the mickey mouse case.

d4rkp4ttern 3y ago

Modal-labs seems like an interesting cloud compute solution that I hadn’t heard of before. Could anyone summarize what are its value props, e.g is it like GCP but better because …

Joker_vD 3y ago

Ah, so we're turning "a million monkeys at a million keyboards could produce the complete works of Shakespeare" into actual, real thing? What a truly disruptive synergy.

miahwilde 3y ago
No - we are the monkeys. This is something different.

mike_hearn 3y ago

Very cool. I've played around with something similar a few months ago (without the sandbox). The tricky part is to find a diff format that the AI can use intuitively to patch code in place, as otherwise you can blow out the token limits quite fast. I wonder to what extent AI is going to kill comprehensibility of the infrastructure. Over the weekend I did a bit of hacking with a friend. He wants to move out of AWS to Hetzner to save money, but wasn't experienced with Linux sysadmin so I was teaching him some stuff, and simultaneously pitching a little prototype I made of a server config tool. It's sort of like a higher level Ansible: you specify what packages and containers you want, the URLs they should be served on, and it goes ahead and configures unattended upgrades, Docker and Traefik to deliver that outcome. The idea was to extend it to configuring backups with BorgBackup and other common sysadmin-y tasks. He was enthused! The next day though, things changed. He'd signed up for GPT-4 and it'd just blasted out a pile of Ansible configs for him. He doesn't know Ansible but it didn't matter, he just iterated with the AI a few times and now the problem is solved. This makes me wonder if there's much point anymore in improving the usability of systems software (programming languages, databases, operating systems, clouds, anything driven by textual configuration). A basic assumption that underlies making better tools is that user's time is valuable, that intuitive and simple systems therefore have value. But the AI's time isn't valuable. It can read some poorly written docs, then the source code of the system, then people getting frustrated on StackOverflow, synthesize all that together and spit out whatever pile of inscrutable config files are needed and it can do so within seconds. Given this experience, if we extrapolate it forwards, then maybe within a couple of decades many organizations will be running on infrastructure that nobody understands at all. You get cases today where maybe individual programs or workflows are only understood by one guy, but those are extremes and it's understood that it's a bad situation. Maybe in future it'll be normal and to do anything companies will have to ask an AI to do it for them. We already see skills atrophy around things like UNIX sysadmin because nowadays everyone learns AWS instead, which is one reason they can charge so much money for it, so I think as people retire knowledge of how to run Linux systems will steadily disappear. But conceivably people will stop learning the cloud too, and then to do anything with servers it'll be AI or the highway. You can also apply this to programming languages. Why engage in a high effort project like inventing a Java or Rust if an AI can spit out correct C++ for you, given enough tries at writing tests? Does the motivation to produce anything beyond incremental improvements to existing tools disappear now? I keep flipping back and forth on the answer to this. On one hand, it seems pretty pointless to further develop this prototype now. Maybe Ansible or AI written shell scripts is the end of the road for making Linux easier to use. On the other hand, AI is remarkably human like. It can also make mistakes, and also benefits from well written docs, simple interfaces and good error messages. So maybe usability still has a purpose, albeit maybe we now need to master a new field of "usability studies for AI".

politician 3y ago

Immediately after seeing ChatGPT, I felt that evolving programs in an OODA loop was going to be a big deal, so it's great to see someone iterating on the concept. I hope they try it with QuickCheck / property-based testing.

seeknotfind 3y ago

LLMs when given code mixing languages will typically execute as if it worked anyway. I have a theory this is because correct examples are the majority on the web. Since LLMs don't have the data with the feedback all the humans have seen constructing these examples, there is no way it could understand. We need to feed LLMs these types of input/error loops explicitly to improve performance or reduce iterations in this area. Otherwise it'll be stuck making many small mistakes forever. Again, this is my theory.

hellodanylo 3y ago

This should give a second life to Test-Driven Development. One of the under-appreciated wisdoms of TDD is that there is a complexity asymmetry in many problems between finding a solution and (fully or partially) verifying it. Examples of asymmetric problems: inverting matrices, sorting an array, computing a function's gradient, compressing a byte stream, etc. Human writes the easier part -- the test suite, the language model writes the harder part -- the solution. This can be a net gain in productivity.

fodkodrasz 3y ago
Finally something will kill the last somewhat satisfying part of the job finally, for the greater profit of the corporate overlords!
BaseballPhysics 3y ago
The problem is without a formal definition of the program semantics, you run the risk of overfitting or uncovered behaviors that, for a human developer who understands the intent of the program, would be implicit. And given how hard formal verification is, I don't know that you'll ever get away with not having to manually check these programs, at which point I question just how much productivity you've gained. It's kinda like self-driving cars: when they work, they work great. But when they fail, they fail in ways a human never would, and therefore a human struggles to anticipate or trust their behaviour. That said, I'm waiting to see the rise of programming languages designed with LLMs in mind, where a human could use contract oriented programming or similar (think: Ada) combined with TDD methods to more formally specify the problem that an LLM is being asked to solve.
- hellodanylo 3y ago
  > I question just how much productivity you've gained. Me too. It's an empirical question to be answered by those who will dare to try. > It's kinda like self-driving cars Strong disagree. Yes, neural nets are blackboxes, but the generated code can be idiomatic, modular, easy to inspect with a debugger, etc. > more formally specify the problem that an LLM is being asked to solve. That would be a great direction to explore.

amelius 3y ago

No need to fix things, just make it search for leaks or null pointer references and explain them to the user, and you have a million dollar product.

luis_cho 3y ago

This takes programming by coincidence to a whole new level.

AtlasBarfed 3y ago
How can this possibly generate maintainable code? And if the tests change, does it have to fully rewrite the code via AI? If you had code that was reasonably working before, but you change the tests and the AI does the "full rewrite" for you, how do you have confidence that the code works? Go ahead, say "it passed the test suite". One of the things that happens in software dev is the devs grokking the overall desire/aims of the software and its users, and often anticipating things that aren't even in the requirements. The test suites are approximations of validating the requirements / derived intent. But what faith do you have in the AI black box, or the black boxes it generates?

choeger 3y ago

Program synthesis by a language model is certainly cool, but I don't think it's really novel. After all, the mutator could as well just be a random AST transformer. So what's the real deal here? Does the mutator end up with a usable program much quicker? Or does it cheat by looking at the test cases? As an example, if I specify a md5 hash function on strings, what will that tool produce?