5 ms·

We have this problem today. I've never worked on a team that didn't have at least one person whose coding ability largely stopped at what you could copy and pas

by tl 3y ago

We have this problem today. I've never worked on a team that didn't have at least one person whose coding ability largely stopped at what you could copy and paste. Pre-internet, they were assigned non-programming tasks or things that could be done via copy-paste-modify of existing code. Collegues on my current team of this caliber have Bard open. This is an observed phenomenon, and I'm not surprised they reach for the less capable but incumbent provided option. I don't have enough data to say this has negatively impacted code quality, but I can rule out most positive results. From the above example, at least one person wants to disable Typescript because it makes code [from them but of unknown origin] harder to execute. That is not promising.

DaiPlusPlus 3y ago
> I've never worked on a team that didn't have at least one person whose coding ability largely stopped at what you could copy and paste. I am fortunate then - I have never been in that situation. You have my sympathy. > Collegues on my current team of this caliber have Bard open. wat ------ I don't like using ChatGPT for code-gen (and I haven't tried any of the others like Bard or LLaMa) because I still don't understand how it's capable of doing-what-it-does. I have thrown prompts at GPT to write my code for me (which is tiresome: as often writing the prompt takes more effort than writing the code myself) but on occassion I'll throw it something that I'll think it can't possibly solve or figure out, and I get blown-away by what it generates (e.g. on a lark I asked it to write a C# program to solve belt-layout problems in Factorio, and the moment I saw what it generated was... a genuinely scary and unsettling experience for me: many people are dismissive of GPT etc, simply saying "oh, that's just because it's been trained to do that" - but I refuse to believe that OpenAI specifically trained GPT to generate C# code for Factorio - and on a whim I asked it to translate that code to Haskell and that, too, was... something-that-just-shouldn't-be-possible. Yes, the code it generated was incomplete, had missing references, and more besides, but remarkably the syntax it generated (for both C# and Haskell) was 100% correct. I avoided ChatGPT for a while after then, not wanting to be rattled again - but I recently tried it again, by prompting it to generate code that I was already writing to see if it knew of a better approach or even as a time-saver (as C# is still not as expressive as I'd like); specifically, I was writing a ModelBinder for ASP.NET Core and it generated code that definitely would have "worked", but had plenty of room for improvement (mostly things that static-analysis would have caught). I wasn't able to observe ChatGPT generating "substantial" bodies of code: it seems best for generating something like a ~200-line class to solve a specific narrow problem (things exactly like that ModelBinder implementation I was working on), especially when I'm stuck for ideas on how to solve a problem entirely by myself. ------ Another example I just remembered as I write this was me asking ChatGPT a straightforward question about MSBuild: what do I need to put in a `.csproj` file to prevent the project from being built when the host environment is non-Windows. This is a screenshot of the interaction I had with it: https://imgur.com/a/TuejAYn https://imgur.com/a/TuejAYn - it starts-off on the right track, but hallucinates small details, or even directly contradicts itself, or generates output that is the precise opposite of what I asked. If ajunior-developer[1] wrote me something like what ChatGPT generated there, then it's easy to attribute that mistake (i.e. that of swapping the `==` and `!=` operators) to simply being a typo (at best), but at worst just absent-minded. But when ChatGPT does it I just don't understand how it's even possible for it to make a mistake like that while it clearly is quite on-top of the rest of the problem-space. And because ChatGPT makes those mistakes unpredictably, all the time, it means I can't trust the code it generates. The other reason, of course, is that it eliminates the challenge (if not thrill?) of me getting to solve a problem by myself, using my own innate reasoning abilities - and being the one who gets to solve problems is, I think, part of every engineer's vocation and sense of identity regardless of field (chemical, civil, mechanical, software, etc). ----- My last thought is that I'm not worried about GPT taking my job or otherwise replacing me: the SW industry already tried that 30 years ago with outsourcing to India, and that didn't kill-off west-coast SWEs - I'm only going to start getting worried about GPT et al. when they do advance to the point where they're almost entirely autonomous and self-directed, while also able to gather-requirements while also challenging assumptions in those requirements, perform its own verification and when the code it generates is at least on-par with what we can write. And I do believe we'll get to that point within 3-5 years from now (e.g. I assume it's straightforward to train an LLM to interrogate incoming requirements and convert that into some kind of normalized domain-model; and already we can wire-up GPT to external APIs, so we can do a conversation-loop between it and compiler error message output, etc). ----- [1] I dislike that term, personally - I feel it's unnecessarily condescending or even infantilising (I'm also fortunate to have never had that as a job-title, and every company I've worked for either had a numeric level nomenclature, or used silly/informal job-titles.
- vba616 3y ago
  >a genuinely scary and unsettling experience for me Every time I think of a good thing to ask ChatGPT (free version), it gives me garbage. Not only for programming tasks, but also for, basically, short essays in English on some topic. By garbage, I mean superficially polished but fundamentally incoherent when you pay attention to the details. It simply doesn't grasp the topic, because, I think, there is no mechanism for that. I don't see a whit of difference between code and plain English - they both require something it doesn't have available. Hypothesis #1 is that I'm just shitty at prompting it. Hypothesis #2, that I like better for reasons of vanity, is that I think of new ideas or questions constantly and most people do not. If you ask anything that exists on the internet then you will be tricked. >when ChatGPT does it I just don't understand how it's even possible for it to make a mistake like that while it clearly is quite on-top of the rest of the problem-space It literally only makes plausible things, true or false. It generates falsehoods that are adjacent to truths or at least to human-produced utterances, in an abstract space. That space naturally has a huge number of falsehoods swirling around every fact it digested. How is that not intuitive, and intuitively so dangerous as to be practically useless? It is inherently the best bullshit, the most superhumanly crafted bullshit, by the much quoted definition of bullshit I first read on HN. But I do keep trying it again and again, at intervals, to see what it has to say.
  - golergka 3y ago
    > ChatGPT (free version) You shouldn't jump to conclusions before you try GPT-4
- tome 3y ago
  > I refuse to believe that OpenAI specifically trained GPT to generate C# code for Factorio Why? Isn't it pretty plausible that someone on GitHub has put up a Factorio belt layout implementation in C#?
- immibis 3y ago
  ChatGPT is easy to understand. It's a bullshit-generator-as-a-service. It's a very complicated probability model which determines which sequences of words are most accepted by humans. You put in some words and it generates the most acceptable next word, and the one after that, and the one after that, ad infinitum. (If you want to play with a GPT model in this style, I suggest NovelAI for a small monthly subscription price). Now, *Chat*GPT is this same thing dressed up to look like a question-and-answer machine. They probably just stick some kind of delimiter between messages. I have read an AI paper somewhere that argued that recent improvements to AI are not from new kinds of AI being better, but from new kinds of AI being able to exploit larger amounts of computational power before hitting the point of diminishing returns. And OpenAI's computational power is gargantuan in AI terms (but probably less than the Pentagon uses to detect nuclear strikes or whatever they do). It happens that correct answers and true facts are very acceptable to humans, so ChatGPT will often produce these, but in fact it doesn't "know" that the answer is correct - only that it's very acceptable. You know what else is acceptable? Confidently stated answers that are subtly wrong. "Auto-GPT" is self-directed ChatGPT. It already exists. It's basically a bullshit generator that generates instructions and then follows them and adds the results back into the question for the next iteration.

by tl 3y ago

DaiPlusPlus 3y ago

> I've never worked on a team that didn't have at least one person whose coding ability largely stopped at what you could copy and paste. I am fortunate then - I have never been in that situation. You have my sympathy. > Collegues on my current team of this caliber have Bard open. wat ------ I don't like using ChatGPT for code-gen (and I haven't tried any of the others like Bard or LLaMa) because I still don't understand how it's capable of doing-what-it-does. I have thrown prompts at GPT to write my code for me (which is tiresome: as often writing the prompt takes more effort than writing the code myself) but on occassion I'll throw it something that I'll think it can't possibly solve or figure out, and I get blown-away by what it generates (e.g. on a lark I asked it to write a C# program to solve belt-layout problems in Factorio, and the moment I saw what it generated was... a genuinely scary and unsettling experience for me: many people are dismissive of GPT etc, simply saying "oh, that's just because it's been trained to do that" - but I refuse to believe that OpenAI specifically trained GPT to generate C# code for Factorio - and on a whim I asked it to translate that code to Haskell and that, too, was... something-that-just-shouldn't-be-possible. Yes, the code it generated was incomplete, had missing references, and more besides, but remarkably the syntax it generated (for both C# and Haskell) was 100% correct. I avoided ChatGPT for a while after then, not wanting to be rattled again - but I recently tried it again, by prompting it to generate code that I was already writing to see if it knew of a better approach or even as a time-saver (as C# is still not as expressive as I'd like); specifically, I was writing a ModelBinder for ASP.NET Core and it generated code that definitely would have "worked", but had plenty of room for improvement (mostly things that static-analysis would have caught). I wasn't able to observe ChatGPT generating "substantial" bodies of code: it seems best for generating something like a ~200-line class to solve a specific narrow problem (things exactly like that ModelBinder implementation I was working on), especially when I'm stuck for ideas on how to solve a problem entirely by myself. ------ Another example I just remembered as I write this was me asking ChatGPT a straightforward question about MSBuild: what do I need to put in a `.csproj` file to prevent the project from being built when the host environment is non-Windows. This is a screenshot of the interaction I had with it: https://imgur.com/a/TuejAYn https://imgur.com/a/TuejAYn - it starts-off on the right track, but hallucinates small details, or even directly contradicts itself, or generates output that is the precise opposite of what I asked. If ajunior-developer[1] wrote me something like what ChatGPT generated there, then it's easy to attribute that mistake (i.e. that of swapping the `==` and `!=` operators) to simply being a typo (at best), but at worst just absent-minded. But when ChatGPT does it I just don't understand how it's even possible for it to make a mistake like that while it clearly is quite on-top of the rest of the problem-space. And because ChatGPT makes those mistakes unpredictably, all the time, it means I can't trust the code it generates. The other reason, of course, is that it eliminates the challenge (if not thrill?) of me getting to solve a problem by myself, using my own innate reasoning abilities - and being the one who gets to solve problems is, I think, part of every engineer's vocation and sense of identity regardless of field (chemical, civil, mechanical, software, etc). ----- My last thought is that I'm not worried about GPT taking my job or otherwise replacing me: the SW industry already tried that 30 years ago with outsourcing to India, and that didn't kill-off west-coast SWEs - I'm only going to start getting worried about GPT et al. when they do advance to the point where they're almost entirely autonomous and self-directed, while also able to gather-requirements while also challenging assumptions in those requirements, perform its own verification and when the code it generates is at least on-par with what we can write. And I do believe we'll get to that point within 3-5 years from now (e.g. I assume it's straightforward to train an LLM to interrogate incoming requirements and convert that into some kind of normalized domain-model; and already we can wire-up GPT to external APIs, so we can do a conversation-loop between it and compiler error message output, etc). ----- [1] I dislike that term, personally - I feel it's unnecessarily condescending or even infantilising (I'm also fortunate to have never had that as a job-title, and every company I've worked for either had a numeric level nomenclature, or used silly/informal job-titles.

vba616 3y ago
>a genuinely scary and unsettling experience for me Every time I think of a good thing to ask ChatGPT (free version), it gives me garbage. Not only for programming tasks, but also for, basically, short essays in English on some topic. By garbage, I mean superficially polished but fundamentally incoherent when you pay attention to the details. It simply doesn't grasp the topic, because, I think, there is no mechanism for that. I don't see a whit of difference between code and plain English - they both require something it doesn't have available. Hypothesis #1 is that I'm just shitty at prompting it. Hypothesis #2, that I like better for reasons of vanity, is that I think of new ideas or questions constantly and most people do not. If you ask anything that exists on the internet then you will be tricked. >when ChatGPT does it I just don't understand how it's even possible for it to make a mistake like that while it clearly is quite on-top of the rest of the problem-space It literally only makes plausible things, true or false. It generates falsehoods that are adjacent to truths or at least to human-produced utterances, in an abstract space. That space naturally has a huge number of falsehoods swirling around every fact it digested. How is that not intuitive, and intuitively so dangerous as to be practically useless? It is inherently the best bullshit, the most superhumanly crafted bullshit, by the much quoted definition of bullshit I first read on HN. But I do keep trying it again and again, at intervals, to see what it has to say.
- golergka 3y ago
  > ChatGPT (free version) You shouldn't jump to conclusions before you try GPT-4
tome 3y ago
> I refuse to believe that OpenAI specifically trained GPT to generate C# code for Factorio Why? Isn't it pretty plausible that someone on GitHub has put up a Factorio belt layout implementation in C#?
immibis 3y ago
ChatGPT is easy to understand. It's a bullshit-generator-as-a-service. It's a very complicated probability model which determines which sequences of words are most accepted by humans. You put in some words and it generates the most acceptable next word, and the one after that, and the one after that, ad infinitum. (If you want to play with a GPT model in this style, I suggest NovelAI for a small monthly subscription price). Now, *Chat*GPT is this same thing dressed up to look like a question-and-answer machine. They probably just stick some kind of delimiter between messages. I have read an AI paper somewhere that argued that recent improvements to AI are not from new kinds of AI being better, but from new kinds of AI being able to exploit larger amounts of computational power before hitting the point of diminishing returns. And OpenAI's computational power is gargantuan in AI terms (but probably less than the Pentagon uses to detect nuclear strikes or whatever they do). It happens that correct answers and true facts are very acceptable to humans, so ChatGPT will often produce these, but in fact it doesn't "know" that the answer is correct - only that it's very acceptable. You know what else is acceptable? Confidently stated answers that are subtly wrong. "Auto-GPT" is self-directed ChatGPT. It already exists. It's basically a bullshit generator that generates instructions and then follows them and adds the results back into the question for the next iteration.