3 ms·

Death by semantic error. An AI tried to tell me the {} opens a new block scope in PHP and any variables in it are scoped to that block. I nearly lol’d and the

Death by semantic error. An AI tried to tell me the {} opens a new block scope in PHP and any variables in it are scoped to that block. I nearly lol’d and the code it gave me was so wrong it was cringey. Hopefully it is better at Python.

jacksensi 3y ago
Only python is supported for code execution.
anonzzzies 3y ago
Google and many people will tell you the same thing. So you must have a lot of lol moments in a day. Gpt4 is quite good (better than most humans I ever met) at php, but bad at facts; don’t ask it facts, ask it to write code. That’s what you would ask a human (outside interviews). Disclaimer; I am formally trained with proofs and proof assistants and I hate the current timeline where we ask ai to drivel up code, but I cannot say it’s bad at doing it; it just is not necessarily sound or even working code sure, but that’s the same as with most human first tries. Then you iterate and make better. My days of ‘I have proven it correct, now I just have to type it in’ are long gone, at least for things that pay for my bread.
- asabla 3y ago
  > My days of ‘I have proven it correct, now I just have to type it in’ are long gone, at least for things that pay for my bread. Kind of having the same feeling as well the more I use ChatGPT and similar systems/models. I wonder if this will result in a general decline in code quality. Or if it's going to help people become better IT professionals overall. I mean, in order to iterate and make things better after first try/implementation, also requires understanding what you typed and it can be improved. Exciting times nonetheless
- nicklecompte 3y ago
  When saying LLMs are "good at writing code" there's a distinction between "good at taking high-level English-language solutions to programming problems and translating them into low-level implementations via a specific language / design patterns / etc" versus "good at finding solutions to programming problems." GPT-4 is indeed quite good at the former - and the former is what most enterprise programming work actually is. A lot of the hard part of professional software development is understanding the problem well enough to describe a solution in English: once you do that, writing the C# or Java is typically somewhat rote. Likewise LLMs are genuinely useful when you know exactly what you want to do with a 3rd-party library, but have to trawl through a bunch of API documentation to figure out the magic words. All that said, LLMs still really suck at the latter problem: https://www.aisnakeoil.com/p/gpt-4-and-professional-benchmarks https://www.aisnakeoil.com/p/gpt-4-and-professional-benchmar... OpenAI's benchmarks fall off quite badly with actual programming benchmarks, compared to simpler tests of code generation. If it involves managing state, creating novel data structures, counting to numbers higher than 3, etc, LLMs just aren't smart enough.
  - anonzzzies 3y ago
    You are right, hence my disclaimer. For paid work, it works fine and that is indeed enterprise kind of mindnumbing plumbing. For other work, I do not use it (but will try every few months if anything changed of course).
- withinboredom 3y ago
  > So you must have a lot of lol moments in a day. I do! I use code generation for non-work-related things, then spend time refactoring it and cleaning up the mess. AI is pretty decent but lacks the ability to write "maintainable" software, which is easy to extend, and replace. It works as a fantastic starting point in that it fills in all the boilerplate code.

jacksensi 3y ago

Only python is supported for code execution.

anonzzzies 3y ago

Google and many people will tell you the same thing. So you must have a lot of lol moments in a day. Gpt4 is quite good (better than most humans I ever met) at php, but bad at facts; don’t ask it facts, ask it to write code. That’s what you would ask a human (outside interviews). Disclaimer; I am formally trained with proofs and proof assistants and I hate the current timeline where we ask ai to drivel up code, but I cannot say it’s bad at doing it; it just is not necessarily sound or even working code sure, but that’s the same as with most human first tries. Then you iterate and make better. My days of ‘I have proven it correct, now I just have to type it in’ are long gone, at least for things that pay for my bread.

asabla 3y ago
> My days of ‘I have proven it correct, now I just have to type it in’ are long gone, at least for things that pay for my bread. Kind of having the same feeling as well the more I use ChatGPT and similar systems/models. I wonder if this will result in a general decline in code quality. Or if it's going to help people become better IT professionals overall. I mean, in order to iterate and make things better after first try/implementation, also requires understanding what you typed and it can be improved. Exciting times nonetheless
nicklecompte 3y ago
When saying LLMs are "good at writing code" there's a distinction between "good at taking high-level English-language solutions to programming problems and translating them into low-level implementations via a specific language / design patterns / etc" versus "good at finding solutions to programming problems." GPT-4 is indeed quite good at the former - and the former is what most enterprise programming work actually is. A lot of the hard part of professional software development is understanding the problem well enough to describe a solution in English: once you do that, writing the C# or Java is typically somewhat rote. Likewise LLMs are genuinely useful when you know exactly what you want to do with a 3rd-party library, but have to trawl through a bunch of API documentation to figure out the magic words. All that said, LLMs still really suck at the latter problem: https://www.aisnakeoil.com/p/gpt-4-and-professional-benchmarks https://www.aisnakeoil.com/p/gpt-4-and-professional-benchmar... OpenAI's benchmarks fall off quite badly with actual programming benchmarks, compared to simpler tests of code generation. If it involves managing state, creating novel data structures, counting to numbers higher than 3, etc, LLMs just aren't smart enough.
- anonzzzies 3y ago
  You are right, hence my disclaimer. For paid work, it works fine and that is indeed enterprise kind of mindnumbing plumbing. For other work, I do not use it (but will try every few months if anything changed of course).
withinboredom 3y ago
> So you must have a lot of lol moments in a day. I do! I use code generation for non-work-related things, then spend time refactoring it and cleaning up the mess. AI is pretty decent but lacks the ability to write "maintainable" software, which is easy to extend, and replace. It works as a fantastic starting point in that it fills in all the boilerplate code.