4 ms·

> Because of this, languages like Python are much better to use for GPT-4 coding as compared to more line-noise languages like Bash or Perl, for instance. By t

> Because of this, languages like Python are much better to use for GPT-4 coding as compared to more line-noise languages like Bash or Perl, for instance. By that argument, one should always make it use a language that's as hard as possible to write a compiling program. So Rust or Haskell or something? I guess at some point it's more important to have a lot of the language in the training data, too...

smt88 3y ago
I think this is exactly the right conclusion. The main complaint people have about strict, thorough type systems is that they have boilerplate. Obviously boilerplate doesn't matter if a machine writes the code. The type system also becomes helpful documentation of the intended behavior of the code that the LLM spits out.
- kridsdale3 3y ago
  Assembly has a lot of boilerplate, and every other language is an abstraction that gets a language-machine to write it for us. So we'll just move to a new standard where we write LLM prompts describing function behavior and it will output the Rust or whatever that we end up storing in our SCM.
  - staunton 3y ago
    There's a fundamental difference though. The LLM is itself inscrutable, while all of these programs used to be written and understood by humans. The language used for programming used to be specified and have unique (hopefully) coherent syntax and abstraction boundaries. Now it's "anything goes" and nobody seems to know how this stuff ends up getting used... Someone might accidentally find it works well and then we might all end up writing fairytales in iambic pentameter describing the use cases of software we want...
gwern 3y ago
Yes, you would think so. Haskell would also be good for encouraging stateless/FP programming which makes unit-testing or property testing much easier. I can make GPT-4 write test-suites for functions which are straightforward data structure transformations, like rewriting strings, but I struggle to create tests for any of the imperative stuff. There presumably would be some way to test all of the imperative buffer editing Elisp code, but I have no idea what. However, in my use so far, I have not noticed any striking differences in error rates between Haskell and the others.

smt88 3y ago

I think this is exactly the right conclusion. The main complaint people have about strict, thorough type systems is that they have boilerplate. Obviously boilerplate doesn't matter if a machine writes the code. The type system also becomes helpful documentation of the intended behavior of the code that the LLM spits out.

kridsdale3 3y ago
Assembly has a lot of boilerplate, and every other language is an abstraction that gets a language-machine to write it for us. So we'll just move to a new standard where we write LLM prompts describing function behavior and it will output the Rust or whatever that we end up storing in our SCM.
- staunton 3y ago
  There's a fundamental difference though. The LLM is itself inscrutable, while all of these programs used to be written and understood by humans. The language used for programming used to be specified and have unique (hopefully) coherent syntax and abstraction boundaries. Now it's "anything goes" and nobody seems to know how this stuff ends up getting used... Someone might accidentally find it works well and then we might all end up writing fairytales in iambic pentameter describing the use cases of software we want...

gwern 3y ago

Yes, you would think so. Haskell would also be good for encouraging stateless/FP programming which makes unit-testing or property testing much easier. I can make GPT-4 write test-suites for functions which are straightforward data structure transformations, like rewriting strings, but I struggle to create tests for any of the imperative stuff. There presumably would be some way to test all of the imperative buffer editing Elisp code, but I have no idea what. However, in my use so far, I have not noticed any striking differences in error rates between Haskell and the others.