3 ms·

I can confirm that it is a suitable use case for GPTs. I do GPT-assisted programming language design and experimentation. In some cases, GPT-4 can even generate

I can confirm that it is a suitable use case for GPTs. I do GPT-assisted programming language design and experimentation. In some cases, GPT-4 can even generate a basic interpreter that allows me to test my new language. Here is an example of GPT's output for Python with braces that was generated after just spending 10 seconds for the prompt: def preprocess_braces(code: str) -> str: lines = code.split('\n') processed_lines = [] indent_level = 0 indent_str = ' ' # 4 spaces for indentation for line in lines: stripped_line = line.strip() # Check for opening brace if stripped_line.endswith('{'): processed_lines.append(indent_str * indent_level + stripped_line[:-1].strip() + ':') indent_level += 1 # Check for closing brace elif stripped_line == '}': indent_level -= 1 else: processed_lines.append(indent_str * indent_level + stripped_line) return '\n'.join(processed_lines) # Example usage: code_with_braces = """ def example_function() { if True { print("Hello, world!") } for i in range(5) { print(i) } } """ processed_code = preprocess_braces(code_with_braces) exec(processed_code) # This will execute the transformed Python code print("Processed Code:\n", processed_code)

im3w1l 2y ago
This code is incorrect though, as it doesn't account for braces in strings and comments. There are other issues too, but that is the big one.
- AdieuToLogic 2y ago
  > This code is incorrect though, as it doesn't account for ... Isn't this the fundamental problem with code generated by a statistical text generation algorithm? In other words, "code" is short for encoding a solution. And to have a solution to encode is to understand the problem to solve. Without understanding, statistical code generation is little more than a popularity contest.
  - roywiggins 2y ago
    The model doesn't "understand" anything, but if you, the programmer, understands it well enough, that can be enough to direct it to find a solution. You can do a lot with ChatGPT or Claude if wrong code is easy to spot (which will obviously depend on what you're working on). If you can easily spot mistakes these things can often come up with a fix once you point it out. I've had some real success converting small-scale production C++ code into Python using Claude. Stuff that isn't really deep or complicated, but it's still faster and less annoying using an LLM to assist. I am sure there are large domains where it's crap, but for relatively simple stuff (CRUD logic, simple file parsing) it does remarkably well.
    - AdieuToLogic 2y ago
      > The model doesn't "understand" anything, but if you, the programmer, understands it well enough, that can be enough to direct it to find a solution. This is exactly my point. Whether a programmer uses past experience exclusively to author source code or a statistical code generator (LLM) and then their learned ability is orthogonal to my original premise. Without understanding, statistical code generation is little more than a popularity contest.
  - im3w1l 2y ago
    I disagree with everything you said. Code is not short for encoding a solution. Code is any kind of program whether correct or not. You can have a solution without understanding it. You can for instance know the rough shape a solution should have and try to guess at the details. And get it correct some of the time. And current models do have some form of understanding, although it is sometimes incomplete. They are clearly able to solve many problems after all.
- skissane 2y ago
  Just this morning I asked GPT-4o to write some code for me. And the code was correct, except for one stupid mistake GPT-4o made which caused a compilation error. I just fixed that myself, but I think it is likely if I gave GPT-4o the error message it could have fixed it too. And I was thinking if I set up a chain-of-thought agent with function-calling, GPT-4o probably could have discovered and fixed the compilation error itself without my involvement. Provide it with unit tests it may even get the code to pass the tests (even if it takes a few iterations)-which could address issues like the braces in strings and comments issue you mention, assuming the unit tests cover them. And if they don’t-if you notice an issue via code inspection or exploratory testing, GPT-4o (in my experience) often does a decent job of “here is the code and here is a description of the bug, modify the code to fix it”. Of course, sometimes chain-of-thought agents get stuck and fail to progress, but something that quickly gives you the right answer 80% of the time can be a big productivity boost. In my case earlier today, it helped that it was a relatively simple function and I gave it a rather detailed natural language spec of what I wanted it to do. I totally could have written it all myself, but writing a natural language spec and getting GPT-4o to translate it to code is (depending on my mood) less mental effort than just writing the code directly.

im3w1l 2y ago

This code is incorrect though, as it doesn't account for braces in strings and comments. There are other issues too, but that is the big one.

AdieuToLogic 2y ago
> This code is incorrect though, as it doesn't account for ... Isn't this the fundamental problem with code generated by a statistical text generation algorithm? In other words, "code" is short for encoding a solution. And to have a solution to encode is to understand the problem to solve. Without understanding, statistical code generation is little more than a popularity contest.
- roywiggins 2y ago
  The model doesn't "understand" anything, but if you, the programmer, understands it well enough, that can be enough to direct it to find a solution. You can do a lot with ChatGPT or Claude if wrong code is easy to spot (which will obviously depend on what you're working on). If you can easily spot mistakes these things can often come up with a fix once you point it out. I've had some real success converting small-scale production C++ code into Python using Claude. Stuff that isn't really deep or complicated, but it's still faster and less annoying using an LLM to assist. I am sure there are large domains where it's crap, but for relatively simple stuff (CRUD logic, simple file parsing) it does remarkably well.
  - AdieuToLogic 2y ago
    > The model doesn't "understand" anything, but if you, the programmer, understands it well enough, that can be enough to direct it to find a solution. This is exactly my point. Whether a programmer uses past experience exclusively to author source code or a statistical code generator (LLM) and then their learned ability is orthogonal to my original premise. Without understanding, statistical code generation is little more than a popularity contest.
- im3w1l 2y ago
  I disagree with everything you said. Code is not short for encoding a solution. Code is any kind of program whether correct or not. You can have a solution without understanding it. You can for instance know the rough shape a solution should have and try to guess at the details. And get it correct some of the time. And current models do have some form of understanding, although it is sometimes incomplete. They are clearly able to solve many problems after all.
skissane 2y ago
Just this morning I asked GPT-4o to write some code for me. And the code was correct, except for one stupid mistake GPT-4o made which caused a compilation error. I just fixed that myself, but I think it is likely if I gave GPT-4o the error message it could have fixed it too. And I was thinking if I set up a chain-of-thought agent with function-calling, GPT-4o probably could have discovered and fixed the compilation error itself without my involvement. Provide it with unit tests it may even get the code to pass the tests (even if it takes a few iterations)-which could address issues like the braces in strings and comments issue you mention, assuming the unit tests cover them. And if they don’t-if you notice an issue via code inspection or exploratory testing, GPT-4o (in my experience) often does a decent job of “here is the code and here is a description of the bug, modify the code to fix it”. Of course, sometimes chain-of-thought agents get stuck and fail to progress, but something that quickly gives you the right answer 80% of the time can be a big productivity boost. In my case earlier today, it helped that it was a relatively simple function and I gave it a rather detailed natural language spec of what I wanted it to do. I totally could have written it all myself, but writing a natural language spec and getting GPT-4o to translate it to code is (depending on my mood) less mental effort than just writing the code directly.