5 ms·

When's the last time you saw a decent coding model create a buffer-overflow bug while trying to use C strings? Serious question. Anyone else seen this happen

When's the last time you saw a decent coding model create a buffer-overflow bug while trying to use C strings? Serious question. Anyone else seen this happen in the last 12-18 months? If so, which model and version were you using?

macintux 4mo ago
Would you even know? Serious question. The volume of code the models can produce, the subtle ways these bugs can manifest (or even only manifest when under attack), it seems like they would be easy to overlook.
- CamperBob2 4mo ago
  I have a habit of getting GPT 5.5 to review everything Opus writes for me, and vice versa. The model in the reviewer role frequently finds things I overlooked myself. Occasionally in parts of the code I wrote. No modern LLM has found any buffer overflow bugs in parts of my code that originated from another LLM. Again, though, they have found one or two that were my fault.
  - bigstrat2003 4mo ago
    Having one clanker verify the output of another has minimal, almost non-existent, persuasive value as evidence.
    - CamperBob2 4mo ago
      Hopefully you're close enough to retirement age that it's a moot point for you. If not, you're headed for a bad time, a major attitude adjustment, or (most likely) both.
smackeyacky 4mo ago
I had Claude write a bit of stupid C# the other day that had an off by one string truncate. Surprised the hell out of me.
krupan 4mo ago
How many people are writing C code with LLMs? I get the impression it's mostly JavaScript web apps
- CamperBob2 4mo ago
  All the time. C, C++, occasionally some VHDL or Verilog.
smj-edison 4mo ago
I use Zig, which has slices, so so far none. But man, it can't get ref counting right to save its life. There have been remarkably few times it's gotten it right on the first try. My codebase considers OOM recoverable, so it keeps forgetting to clean up memory when OOM is raised. Even in the happy path though it still messes up ref counting. I use Kimi k2.6.

macintux 4mo ago

Would you even know? Serious question. The volume of code the models can produce, the subtle ways these bugs can manifest (or even only manifest when under attack), it seems like they would be easy to overlook.

CamperBob2 4mo ago
I have a habit of getting GPT 5.5 to review everything Opus writes for me, and vice versa. The model in the reviewer role frequently finds things I overlooked myself. Occasionally in parts of the code I wrote. No modern LLM has found any buffer overflow bugs in parts of my code that originated from another LLM. Again, though, they have found one or two that were my fault.
- bigstrat2003 4mo ago
  Having one clanker verify the output of another has minimal, almost non-existent, persuasive value as evidence.
  - CamperBob2 4mo ago
    Hopefully you're close enough to retirement age that it's a moot point for you. If not, you're headed for a bad time, a major attitude adjustment, or (most likely) both.

smackeyacky 4mo ago

I had Claude write a bit of stupid C# the other day that had an off by one string truncate. Surprised the hell out of me.

krupan 4mo ago

How many people are writing C code with LLMs? I get the impression it's mostly JavaScript web apps

CamperBob2 4mo ago
All the time. C, C++, occasionally some VHDL or Verilog.

smj-edison 4mo ago

I use Zig, which has slices, so so far none. But man, it can't get ref counting right to save its life. There have been remarkably few times it's gotten it right on the first try. My codebase considers OOM recoverable, so it keeps forgetting to clean up memory when OOM is raised. Even in the happy path though it still messes up ref counting. I use Kimi k2.6.