4 ms·

> Right. If you generate some code with ChatGPT, and then try to find similar code on the web, you usually will. People who "follow" AI, as the latest fad they

by dmd 2y ago

> Right. If you generate some code with ChatGPT, and then try to find similar code on the web, you usually will. People who "follow" AI, as the latest fad they want to comment on and appear intelligent about, repeat things like this constantly, even though they're not actually true for anything but the most trivial hello-world types of problems. I write code all day every day. I use Copilot and the like all day every day (for me, in the medical imaging software field), and all day every day it is incredibly useful and writes nearly exactly the code I would have written, but faster. And none of it appears anywhere else; I've checked.

ngai_aku 2y ago
You’re solving novel problems all day every day?
- dmd 2y ago
  Pretty much, yes. My job is pretty fun; it mostly entails things like "take this horrible file workflow some research assistant came up with while high 15 years ago and turn it into a newer horrible file format a NEW research assistant came up with (also while high) 3 years ago" - and automate this in our data processing pipeline.
  - Der_Einzige 2y ago
    Due to WFH, the weed laws where tech workers live, and the fast tolerance building of cannabis in the body - I estimate that 10% of all code written by west coast tech workers is done “while high” and that estimate is likely low.
    - portaouflop 2y ago
      Do tech workers write better or worse code while high ?
      - throw310822 2y ago
        Should copilot be renamed to "designated driver"?
  - delusional 2y ago
    If I understand that correctly you're converting file formats? That's not exactly "novel"
    - llm_trw 2y ago
      This is exactly the type of novel work that llms are good at. It's tedious and has annoying internal logic, but that logic is quite flat and there are a million examples to generalise from. What they fail at is code with high cyclomatic complexity. Back in the llama 2 finetune days I wrote a script that would break down what each node in the control flow graph into its own prompt using literate programming and the results were amazing for the time. Using the same prompts I'd get correct code in every language I tried.
  - fireflash38 2y ago
    If you've got clearly defined start input format and end output format, sure it seems that it would be a good candidate for heavy LLM use. But I don't know if that's most people.
    - dmd 2y ago
      If it were ever clearly defined or even consistent from input to input I would be overjoyed.
- gitaarik 2y ago
  If we weren't, we (as in developers) weren't needed, right?
tymscar 2y ago
How often did you check?
wokwokwok 2y ago
> even though they're not actually true for anything but the most trivial hello-world types of problems. Um. All the parent post said was: > then try to find similar code on the web, you usually will. Not identical code. Similar code. I think you're really stretching the domain of plausibility to suggest that any code you write is novel enough that you can't find 'similar' code on the internet. To suggest that code generated from a corpus that is not going to be 'similar' to the code from the corpus is just factually and unambiguously false. Of course, it depends on what you interpret 'similar' to mean; but I think it's not unfair to say a lot of code is composed of smaller parts of code that is extremely similar to other examples of code on the internet. Obviously you're not going to find an example similar to your entire code base; but if you're using, for example, copilot where you generate many small snippets of code... welll....
- dmd 2y ago
  Ok, yes. There are other pieces of code on the internet that use a for loop or an if statement. By that logic what you wrote was also composed that way. After all, you’ve used all words that have been used before! I bet even phrases like “that is extremely similar” and “generated from a corpus” and “unambiguously false”. Again, I really find it hard to believe that anyone could make an argument like the one you’re making who has actually used these tools in their work for hundreds of hours, vs. for a couple minutes here or there with made up problems.
  - wokwokwok 2y ago
    > I really find it hard to believe What's true and what's not true is not related to what you personally believe. It is factually and unambiguously false to state that generated code is, in general, not similar to other code from the corpus it is trained on. > And none of it appears anywhere else; I've checked. ^ Even if this statement, is not false (I'm skeptical, but whatever), in general, it would be false for most users of copilot. None of it appears anywhere else? None of it? Really? That's not true of the no-AI code base I'm working on. That's very difficult to believe it would be true on a code base heavily written by copilot and the like. It's probably not true, in general, for AI generated code bases. We can have a different conversation about verbatim copied code, where an AI model generates a large body of verbatim copy from a training source. That's very unusual. ...but to say the generated code wouldn't even be similar? Come on. That's literally what LLMs do.
bobsmooth 2y ago
I've had generated code include comments so specific I was able to find the exact github repo where it came from.
cageface 2y ago
I find specific blog posts ChatGPT is cribbing from all the time when I use it. I think it depends a lot on exactly what you're asking it for.

by dmd 2y ago

ngai_aku 2y ago

You’re solving novel problems all day every day?

dmd 2y ago
Pretty much, yes. My job is pretty fun; it mostly entails things like "take this horrible file workflow some research assistant came up with while high 15 years ago and turn it into a newer horrible file format a NEW research assistant came up with (also while high) 3 years ago" - and automate this in our data processing pipeline.
- Der_Einzige 2y ago
  Due to WFH, the weed laws where tech workers live, and the fast tolerance building of cannabis in the body - I estimate that 10% of all code written by west coast tech workers is done “while high” and that estimate is likely low.
  - portaouflop 2y ago
    Do tech workers write better or worse code while high ?
    - throw310822 2y ago
      Should copilot be renamed to "designated driver"?
- delusional 2y ago
  If I understand that correctly you're converting file formats? That's not exactly "novel"
  - llm_trw 2y ago
    This is exactly the type of novel work that llms are good at. It's tedious and has annoying internal logic, but that logic is quite flat and there are a million examples to generalise from. What they fail at is code with high cyclomatic complexity. Back in the llama 2 finetune days I wrote a script that would break down what each node in the control flow graph into its own prompt using literate programming and the results were amazing for the time. Using the same prompts I'd get correct code in every language I tried.
- fireflash38 2y ago
  If you've got clearly defined start input format and end output format, sure it seems that it would be a good candidate for heavy LLM use. But I don't know if that's most people.
  - dmd 2y ago
    If it were ever clearly defined or even consistent from input to input I would be overjoyed.
gitaarik 2y ago
If we weren't, we (as in developers) weren't needed, right?

tymscar 2y ago

How often did you check?

wokwokwok 2y ago

> even though they're not actually true for anything but the most trivial hello-world types of problems. Um. All the parent post said was: > then try to find similar code on the web, you usually will. Not identical code. Similar code. I think you're really stretching the domain of plausibility to suggest that any code you write is novel enough that you can't find 'similar' code on the internet. To suggest that code generated from a corpus that is not going to be 'similar' to the code from the corpus is just factually and unambiguously false. Of course, it depends on what you interpret 'similar' to mean; but I think it's not unfair to say a lot of code is composed of smaller parts of code that is extremely similar to other examples of code on the internet. Obviously you're not going to find an example similar to your entire code base; but if you're using, for example, copilot where you generate many small snippets of code... welll....

dmd 2y ago
Ok, yes. There are other pieces of code on the internet that use a for loop or an if statement. By that logic what you wrote was also composed that way. After all, you’ve used all words that have been used before! I bet even phrases like “that is extremely similar” and “generated from a corpus” and “unambiguously false”. Again, I really find it hard to believe that anyone could make an argument like the one you’re making who has actually used these tools in their work for hundreds of hours, vs. for a couple minutes here or there with made up problems.
- wokwokwok 2y ago
  > I really find it hard to believe What's true and what's not true is not related to what you personally believe. It is factually and unambiguously false to state that generated code is, in general, not similar to other code from the corpus it is trained on. > And none of it appears anywhere else; I've checked. ^ Even if this statement, is not false (I'm skeptical, but whatever), in general, it would be false for most users of copilot. None of it appears anywhere else? None of it? Really? That's not true of the no-AI code base I'm working on. That's very difficult to believe it would be true on a code base heavily written by copilot and the like. It's probably not true, in general, for AI generated code bases. We can have a different conversation about verbatim copied code, where an AI model generates a large body of verbatim copy from a training source. That's very unusual. ...but to say the generated code wouldn't even be similar? Come on. That's literally what LLMs do.

bobsmooth 2y ago

I've had generated code include comments so specific I was able to find the exact github repo where it came from.

cageface 2y ago

I find specific blog posts ChatGPT is cribbing from all the time when I use it. I think it depends a lot on exactly what you're asking it for.