4 ms·

I'm with @delichon... I don't "trust" LLM generated code, I review it and decide how appropriate (if at all) it is, possibly modify it, and include it if it mak

I'm with @delichon... I don't "trust" LLM generated code, I review it and decide how appropriate (if at all) it is, possibly modify it, and include it if it makes sense. Trust isn't really part of the equation.

jaredsohn 2y ago
I also don't trust code from stackoverflow, colleagues, or even code I had written myself in the past. Is helpful to always be skeptical of code and try to get an understanding of it. Another way I think of it is I find it easier to revise than to write (both in English and code). So when I write something I just try to get my ideas out there and then skeptically reread it and recompose. LLMs let me just speed up that process.
- mindcrime 2y ago
  or even code I had written myself in the past. Aaawwww hellz no. I hate looking at my old code. I've had too many of those surreal experiences where something breaks, I start looking at how to fix it, and am suddenly stunned to realize "this could never have worked in the first place." There's one or two of those I'm still puzzling over, decades later. :-(
  - zer00eyz 2y ago
    I have been writing go for a decade now. Good go is like brutalist architecture. Strip out all the cute, the smart the stupid coder tricks. Write ugly, blocky, verbose, spelled out, functional code. Test the ever loving shit out of it. Go is fast enough that if your going back to fix a "performance problem" you either did something head smackingly stupid (we all have) or you have a real bottleneck and it's time to roll up your sleeves and write that elegant solution. I tend to revisit code more often to add features or deal with edged case bugs than anything else. That clear and functional code without any magic is a win. Edit: you can do the above with almost any language but those that choose packages over standard libs make it HARD.
nemothekid 2y ago
I don't know if OP meant "trust" as in free from security errors or bugs. I often struggle with "how is this a useful tool?". Fixing bugs in someone else's code seems like more a time sink than just writing the code myself. I assume everyone else must be working on projects/languages that have far more representation in the training set, but for me, especially working in Rust, even trying to write interfaces into C libraries, I would constantly find myself dealing with hallucinated parameters or function calls, incorrectly implemented functions or just broken code. At the very least if I do it myself, I'll have some intuition about how the code works.
- mindcrime 2y ago
  > I assume everyone else must be working on projects/languages that have far more representation in the training set, I can't speak for anyone else, but I've used LLM's mainly for generating Java code. The generated code hasn't necessarily ever been perfect, but IME it's been "close enough" that even counting having to make some manual changes, it's still faster than writing it myself from scratch. That said, I'm typically only using it for relatively short snippets of code, where I "know" how to do the operation in question, but want to avoid having to stop and look up fiddly details that might not be cached "top of mind" because I haven't written that code in a while. I'm sure there are use-cases where LLM's don't save time, but I'm equally sure that there are at least some where they do.

jaredsohn 2y ago

I also don't trust code from stackoverflow, colleagues, or even code I had written myself in the past. Is helpful to always be skeptical of code and try to get an understanding of it. Another way I think of it is I find it easier to revise than to write (both in English and code). So when I write something I just try to get my ideas out there and then skeptically reread it and recompose. LLMs let me just speed up that process.

mindcrime 2y ago
or even code I had written myself in the past. Aaawwww hellz no. I hate looking at my old code. I've had too many of those surreal experiences where something breaks, I start looking at how to fix it, and am suddenly stunned to realize "this could never have worked in the first place." There's one or two of those I'm still puzzling over, decades later. :-(
- zer00eyz 2y ago
  I have been writing go for a decade now. Good go is like brutalist architecture. Strip out all the cute, the smart the stupid coder tricks. Write ugly, blocky, verbose, spelled out, functional code. Test the ever loving shit out of it. Go is fast enough that if your going back to fix a "performance problem" you either did something head smackingly stupid (we all have) or you have a real bottleneck and it's time to roll up your sleeves and write that elegant solution. I tend to revisit code more often to add features or deal with edged case bugs than anything else. That clear and functional code without any magic is a win. Edit: you can do the above with almost any language but those that choose packages over standard libs make it HARD.

nemothekid 2y ago

I don't know if OP meant "trust" as in free from security errors or bugs. I often struggle with "how is this a useful tool?". Fixing bugs in someone else's code seems like more a time sink than just writing the code myself. I assume everyone else must be working on projects/languages that have far more representation in the training set, but for me, especially working in Rust, even trying to write interfaces into C libraries, I would constantly find myself dealing with hallucinated parameters or function calls, incorrectly implemented functions or just broken code. At the very least if I do it myself, I'll have some intuition about how the code works.

mindcrime 2y ago
> I assume everyone else must be working on projects/languages that have far more representation in the training set, I can't speak for anyone else, but I've used LLM's mainly for generating Java code. The generated code hasn't necessarily ever been perfect, but IME it's been "close enough" that even counting having to make some manual changes, it's still faster than writing it myself from scratch. That said, I'm typically only using it for relatively short snippets of code, where I "know" how to do the operation in question, but want to avoid having to stop and look up fiddly details that might not be cached "top of mind" because I haven't written that code in a while. I'm sure there are use-cases where LLM's don't save time, but I'm equally sure that there are at least some where they do.