4 ms·

Sometime I think there's something wrong with me. I've used copilot, I'm paying for ChatGPT and we're also having the Jetbrains AI, and there's just something s

by mns 2y ago

Sometime I think there's something wrong with me. I've used copilot, I'm paying for ChatGPT and we're also having the Jetbrains AI, and there's just something so off about all of this. Some basic things are fine, but once you get into specialised things, everything gets just terribly wrong and weird. I can't even put it into words, I see people saying how they are 10x more productive (I'd like to see actual numbers and proof for this), but I just don't see how. Maybe we're working on very custom stuff, or very specific things, but all of these tools seem to give very deep or confident answers that are just plain wrong and shallow. Just yesterday I used GPT 4o for some basic help with Puppet, and the examples it printed, even though I would say it's quite basic, were just wrong, but in the sense of having to debug it for 2 hours just to figure out how ridiculous the error was. I fear the fact that people will end up releasing unsafe, insecure and simply wrong code every day, code that they never debug and not even understand, that maybe works for a basic set of details, but once the real world hits it, it will fail like those self driving cars driving full speed into a trailer that has the same color as the road or sky.

HPsquared 2y ago
It's going to be the same issue as poorly-structured Excel workflows: non-programmers doing programming. Excel itself gets the blame for the buggy poorly-structured spreadsheets made by people who don't have the "programmer mindset" of thinking of the edge cases, exceptions, and extensibility. So it will be with inexperienced people coding with LLMs.
torginus 2y ago
Absolutely the same for me. Whenever I encounter a problem I'm pretty sure nobody else has encountered before (or has not written about it at least), ChatGPT writes complete nonsense. On stuff that's a bit obscure, but not really (like your Vulkan example), ChatGPT tends to write 60-95% correct code, that's flawed in the exact ways a noob wouldn't be able to fix. In this case, nicking code from Github seems to fix the issue, even if I need to adapt it a bit. Then comes the licensing issue. Often, when searching for an obscure topic, the code ChatGPT generates is very close to what's found on Github, but said code often comes with a non-permissive license, unlike what the AI generates.
- ziml77 2y ago
  > Then comes the licensing issue. Often, when searching for an obscure topic, the code ChatGPT generates is very close to what's found on Github, but said code often comes with a non-permissive license, unlike what the AI generates. I think this is a feature for a lot of people. ChatGPT can launder code so they don't have to care about licenses.
cess11 2y ago
Having spent a couple of hours with some llama models and others I've given up on them for code. Code generation from XML or grinding it out ad hoc with stuff like sed on top of a data or config file is faster and more convenient for me. The Jetbrains thing is rather rudely incompetent, it consistently insists on suggestions that use variables and fragments that are supposed to be replaced by what I'm writing and also much more complex than what I actually need. I suffered through at least a hundred mistaken tabbed out shitty suggestions before I disconnected it.
koyote 2y ago
I think there's a vast ocean of different software engineers and the type of code they write on a daily basis, which is why you get such differing views on AI's effectiveness. For me, AI has only ever been useful for very basic tasks and scripts: If I need a quick helper script for something in Python, a language that isn't my daily drier, AI usually gets me there in a couple of prompts. Or maybe I am writing some powershell/bash and forgot the syntax for something and AI is quicker (or has more context) than a web search. However, my main job is trying to come up with "elegant" architectures for complex business logic that interacts with an existing large code base. AI is just completely out of its depth in such cases due to lack of context but also lack of source material to draw from. Even unit tests only work with the most basic of cases, most of the time the setup is so complex it just produces garbage. I've also had very little luck getting it to write performant code. I almost have to feed it the techniques or algorithms before it attempts to write such code, and even then it's usually wrong or not as efficient as it could be.
- hobs 2y ago
  Indeed, if you need a lot of boilerplate that's pretty similar to existing commonly available code, you're set. However... That code is probably buggy, slow, poorly architected, very verbose, and has logical issues where the examples and your needs dont exact match. Generally, the longer the snippet you want your LLM to generate, the more likely its going to go off the rails. I think for some positions this can get you 90% of the code done. For me this usually means I can get started very fast on a new problem, but the last remaining "10%" actually takes significantly longer and more effort to integrate because I dont understand the other 90% off the top :)
- TillE 2y ago
  > my main job is trying to come up with "elegant" architectures for complex business logic that interacts with an existing large code base Right, that's surely the main job of nearly every experienced developer. It's really cool that LLMs can generate code for isolated tasks, but they can barely even begin to do the hard work, and that seems very unlikely to change in the foreseeable future.
vladimirralev 2y ago
I have the same experience. I have my own benchmark, I take a relatively complex project like FreeSWITCH on github, which is part of the training set for all coding LLMs anyway so they should know it and I ask the AI to code small snippets, tests, suggestions and fixes to see how well it understands the codebase and the architecture. I just tried the latest Cursor + Sonnet and it failed in every task. The problem is that there is no way to understand the code without either complete understanding of the domain and the intents or running it in some context. Telecom and media domains in particular are well documented in specs and studied in forum discussions. I am sure they are part of the training data, because I can get most answers if asked directly. So far the LLMs fail to reason about anything useful for me.
zarzavat 2y ago
I don’t use ChatGPT or chat AI for coding, at least not often. It does help me get unstuck on occasion, but most of the time the context switch is too much of a productivity sink. However, I use copilot autocomplete consistently. That makes me much more productive. It’s just really good autocomplete. As an experienced developer I’d put my productivity improvement at 2-3x. It’s huge but not 10x. I’m limited by my decision speed, I need to decide what I want the code to do, AI can’t help with that - it can only do the “how”. Far from introducing more bugs, using Copilot frees some mental cycles for me to be more aware of the code I’m writing.

by mns 2y ago

HPsquared 2y ago

It's going to be the same issue as poorly-structured Excel workflows: non-programmers doing programming. Excel itself gets the blame for the buggy poorly-structured spreadsheets made by people who don't have the "programmer mindset" of thinking of the edge cases, exceptions, and extensibility. So it will be with inexperienced people coding with LLMs.

torginus 2y ago

Absolutely the same for me. Whenever I encounter a problem I'm pretty sure nobody else has encountered before (or has not written about it at least), ChatGPT writes complete nonsense. On stuff that's a bit obscure, but not really (like your Vulkan example), ChatGPT tends to write 60-95% correct code, that's flawed in the exact ways a noob wouldn't be able to fix. In this case, nicking code from Github seems to fix the issue, even if I need to adapt it a bit. Then comes the licensing issue. Often, when searching for an obscure topic, the code ChatGPT generates is very close to what's found on Github, but said code often comes with a non-permissive license, unlike what the AI generates.

ziml77 2y ago
> Then comes the licensing issue. Often, when searching for an obscure topic, the code ChatGPT generates is very close to what's found on Github, but said code often comes with a non-permissive license, unlike what the AI generates. I think this is a feature for a lot of people. ChatGPT can launder code so they don't have to care about licenses.

cess11 2y ago

Having spent a couple of hours with some llama models and others I've given up on them for code. Code generation from XML or grinding it out ad hoc with stuff like sed on top of a data or config file is faster and more convenient for me. The Jetbrains thing is rather rudely incompetent, it consistently insists on suggestions that use variables and fragments that are supposed to be replaced by what I'm writing and also much more complex than what I actually need. I suffered through at least a hundred mistaken tabbed out shitty suggestions before I disconnected it.

koyote 2y ago

I think there's a vast ocean of different software engineers and the type of code they write on a daily basis, which is why you get such differing views on AI's effectiveness. For me, AI has only ever been useful for very basic tasks and scripts: If I need a quick helper script for something in Python, a language that isn't my daily drier, AI usually gets me there in a couple of prompts. Or maybe I am writing some powershell/bash and forgot the syntax for something and AI is quicker (or has more context) than a web search. However, my main job is trying to come up with "elegant" architectures for complex business logic that interacts with an existing large code base. AI is just completely out of its depth in such cases due to lack of context but also lack of source material to draw from. Even unit tests only work with the most basic of cases, most of the time the setup is so complex it just produces garbage. I've also had very little luck getting it to write performant code. I almost have to feed it the techniques or algorithms before it attempts to write such code, and even then it's usually wrong or not as efficient as it could be.

hobs 2y ago
Indeed, if you need a lot of boilerplate that's pretty similar to existing commonly available code, you're set. However... That code is probably buggy, slow, poorly architected, very verbose, and has logical issues where the examples and your needs dont exact match. Generally, the longer the snippet you want your LLM to generate, the more likely its going to go off the rails. I think for some positions this can get you 90% of the code done. For me this usually means I can get started very fast on a new problem, but the last remaining "10%" actually takes significantly longer and more effort to integrate because I dont understand the other 90% off the top :)
TillE 2y ago
> my main job is trying to come up with "elegant" architectures for complex business logic that interacts with an existing large code base Right, that's surely the main job of nearly every experienced developer. It's really cool that LLMs can generate code for isolated tasks, but they can barely even begin to do the hard work, and that seems very unlikely to change in the foreseeable future.

vladimirralev 2y ago

I have the same experience. I have my own benchmark, I take a relatively complex project like FreeSWITCH on github, which is part of the training set for all coding LLMs anyway so they should know it and I ask the AI to code small snippets, tests, suggestions and fixes to see how well it understands the codebase and the architecture. I just tried the latest Cursor + Sonnet and it failed in every task. The problem is that there is no way to understand the code without either complete understanding of the domain and the intents or running it in some context. Telecom and media domains in particular are well documented in specs and studied in forum discussions. I am sure they are part of the training data, because I can get most answers if asked directly. So far the LLMs fail to reason about anything useful for me.

zarzavat 2y ago

I don’t use ChatGPT or chat AI for coding, at least not often. It does help me get unstuck on occasion, but most of the time the context switch is too much of a productivity sink. However, I use copilot autocomplete consistently. That makes me much more productive. It’s just really good autocomplete. As an experienced developer I’d put my productivity improvement at 2-3x. It’s huge but not 10x. I’m limited by my decision speed, I need to decide what I want the code to do, AI can’t help with that - it can only do the “how”. Far from introducing more bugs, using Copilot frees some mental cycles for me to be more aware of the code I’m writing.