4 ms·

> Not really. Got some code you don't understand? Feed it to a model and ask it to add comments. Absolutely not. An experienced individual in their field can

by rvz 1y ago

> Not really. Got some code you don't understand? Feed it to a model and ask it to add comments. Absolutely not. An experienced individual in their field can tell if the AI made a mistake in the comments / code rather than the typical untrained eye. So no, actually read the code and understand what it does. > Ultimately humans will never need to look at most AI-generated code, any more than we have to look at the machine language emitted by a C compiler. So for safety critical systems, one should not look or check if code has been AI generated?

CamperBob2 1y ago
So for safety critical systems, one should not look or check if code has been AI generated? If you don't review the code your C compiler generates now, why not? Compiler bugs still happen, you know.
- supriyo-biswas 1y ago
  You do understand that LLM output is non-deterministic and tends to have a higher error ratio than compiler bugs, which do not exhibit this “feature”. I see in one of your other posts that you were loudly grumbling about being downvoted. You may want to revisit if taking a combative, bad faith approach while replying to other people is really worth it.
  - CamperBob2 1y ago
    I see in one of your other posts that you were loudly grumbling about being downvoted. You may want to revisit if taking a combative, bad faith approach while replying to other people is really worth it. (Shrug) Tool use is important. People who are better than you at using tools will outcompete you. That's not an opinion or "combative," whatever that means, just the way it works. It's no skin off my nose either way, but HN is not a place where I like to see ignorant, ill-informed opinions paraded with pride.
- rvz 1y ago
  > If you don't review the code your C compiler generates now, why not? That isn't a reason why you should NOT review AI-generated code. Even when comparing the two, a C compiler is far more deterministic in the code that it generates than LLMs, which are non-deterministic and unpredictable by design. > Compiler bugs still happen, you know. The whole point is 'verification' which is extremely important in compiler design and there exists a class of formally-verified compilers that are proven to not generate compiler bugs. There is no equivalent for LLMs. In any case, you still NEED to check if the code's functionality matches the business requirements; AI-generated or not; especially in safety critical systems. Otherwise, it is considered as a logic bug in your implementation.
  - CamperBob2 1y ago
    If you can look at what's happening today, and imagine that code will still be generated the same way in 10-15 years as it is today, then your imagination beats mine. 99.9999% of code is not written with compilers that are "formally verified" as immune to code-generation bugs. It's not likely that any code that you and I run every day is.
    - rvz 1y ago
      > 99.9999% of code is not written with compilers that are "formally verified" as immune to code-generation bugs. Again, that isn't a reason to never check or write tests for your code because an "AI-generated it" or even assuming that an AI will detect all of them. In fact, it means you NEED to do more reviewing, checking and testing than ever before. > It's not likely that any code that you and I run every day is. So millions of phones, cars, control systems, medical devices and planes in use today aren't running formally verified code every day? Are you sure?
      - CamperBob2 1y ago
        Yes, I'm very sure. 99.9999% of the code you are running is not formally proven to be correct, and was not generated by a compiler whose output was formally proven to be correct. Just curious, how much time have you spent in (a) industry, (b) a CS classroom, or (c) both?

by rvz 1y ago

CamperBob2 1y ago

So for safety critical systems, one should not look or check if code has been AI generated? If you don't review the code your C compiler generates now, why not? Compiler bugs still happen, you know.

supriyo-biswas 1y ago
You do understand that LLM output is non-deterministic and tends to have a higher error ratio than compiler bugs, which do not exhibit this “feature”. I see in one of your other posts that you were loudly grumbling about being downvoted. You may want to revisit if taking a combative, bad faith approach while replying to other people is really worth it.
- CamperBob2 1y ago
  I see in one of your other posts that you were loudly grumbling about being downvoted. You may want to revisit if taking a combative, bad faith approach while replying to other people is really worth it. (Shrug) Tool use is important. People who are better than you at using tools will outcompete you. That's not an opinion or "combative," whatever that means, just the way it works. It's no skin off my nose either way, but HN is not a place where I like to see ignorant, ill-informed opinions paraded with pride.
rvz 1y ago
> If you don't review the code your C compiler generates now, why not? That isn't a reason why you should NOT review AI-generated code. Even when comparing the two, a C compiler is far more deterministic in the code that it generates than LLMs, which are non-deterministic and unpredictable by design. > Compiler bugs still happen, you know. The whole point is 'verification' which is extremely important in compiler design and there exists a class of formally-verified compilers that are proven to not generate compiler bugs. There is no equivalent for LLMs. In any case, you still NEED to check if the code's functionality matches the business requirements; AI-generated or not; especially in safety critical systems. Otherwise, it is considered as a logic bug in your implementation.
- CamperBob2 1y ago
  If you can look at what's happening today, and imagine that code will still be generated the same way in 10-15 years as it is today, then your imagination beats mine. 99.9999% of code is not written with compilers that are "formally verified" as immune to code-generation bugs. It's not likely that any code that you and I run every day is.
  - rvz 1y ago
    > 99.9999% of code is not written with compilers that are "formally verified" as immune to code-generation bugs. Again, that isn't a reason to never check or write tests for your code because an "AI-generated it" or even assuming that an AI will detect all of them. In fact, it means you NEED to do more reviewing, checking and testing than ever before. > It's not likely that any code that you and I run every day is. So millions of phones, cars, control systems, medical devices and planes in use today aren't running formally verified code every day? Are you sure?
    - CamperBob2 1y ago
      Yes, I'm very sure. 99.9999% of the code you are running is not formally proven to be correct, and was not generated by a compiler whose output was formally proven to be correct. Just curious, how much time have you spent in (a) industry, (b) a CS classroom, or (c) both?