6 ms·
It's a damning assertive duck, completely out of proportion to its competence. I've seen enough people led astray by talking to it.
by marcosdumay 1y ago
It's a damning assertive duck, completely out of proportion to its competence.
I've seen enough people led astray by talking to it.
- foxyv 1y agoSame here. When I'm teaching coding I've noticed that LLMs will confuse the heck out of students. They will accept what it suggests without realizing that it is suggesting nonsense.
- cogogo 1y agoI’m self taught and don’t code that much but I feel like I benefit a ton from LLMs giving me specific answers to questions that would take me a lot of time to figure out with documentation and stack overflow. Or even generating snippets that I can evaluate whether or not will work. But I actually can’t imagine how you can teach someone to code if they have access to an LLM from day one. It’s too easy to take the easy route and you lose the critical thinking and problem solving skills required to code in the first place and to actually make an LLM useful in the second. Best of luck to you… it’s a weird time for a lot of things. *edit them/they
- ilamont 1y ago> I’m self taught and don’t code that much but I feel like I benefit a ton from LLMs giving me specific answers to questions that would take me a lot of time to figure out with documentation and stack overflow Same here. Combing discussion forums and KB pages for an hour or two, seeking how to solve a certain problem with a specific tool has been replaced by a 50-100 word prompt in Gemini which gives very helpful replies, likely derived from many of those same forums and support docs. Of course I am concerned about accuracy, but for most low-level problems it's easy enough to test. And you know what, many of those forum posts or obsolete KB articles had their own flaws, too.
- specproc 1y agoI really value forums and worry about the impact LLMs are having on them. Stackoverflow has its flaws for sure, but I've learned a hell of a lot watching smart people argue it out in a thread. Actual learning: the pros and cons of different approaches. Even the downvoted answers tell you something often. Asking an LLM gets you a single response from a median stackoverflow commenter. Sure, they're infinitely patient and responsive, but can never beat a few grizzled smart arses trying to one-up each other.
- lanstin 1y agoI think you can learn a lot from debugging, and all the code I've put into prod from LLM has needed debugging (rather more than it should from the LOC count).
- cogogo 1y agoI agree and that’s definitely part of my current learning process. But I think someone dependent on a LLM from day one might struggle to debug their LLM generated code. Probably just feed it back to the LLM and their mileage is definitely going to vary with that approach.
- lanstin 1y agoMaybe, but if I recall (from long long ago) in learning how to program, the process of debugging ones code was almost more enlightening than writing it initially - so many loops of not understanding the implications of the code and then smacking my forehead - and remembering it for ever. Like being able to type code but not debug is pretty worthless.
- chucksmash 1y agoTbf, there's a phase of learning to code where everything is pretty much an incantation you learn because someone told you "just trust me." You encounter "here's how to make the computer print text in Python" before you would ever discuss strings or defining and invoking functions, for instance. To get your start you kind of have to just accept some stuff uncritically. It's hard to remember what it was like to be in that phase. Once simple things like using variables are second nature, it's difficult to put yourself back into the shoes of someone who doesn't understand the use of a variable yet.
- eszed 1y agoYeah, and accepting the LLM uncritically* is exactly what you shouldn't do in any non-trivial context. But, as a sibling poster pointed out: for now.
- codr7 1y agoMore like forever as long as it's an LLM.
- tharkun__ 1y agoFair enough on 'cutting the learning tree' at some points i.e. ignoring that you don't understand yet why something works/does what it does. We (should) keep doing that later on in life as well. But unless you teach a kid that's never done any math where `x` was a thing to program, what's so hard about understanding the concept of a variable in programming?
- na4ma4 1y agoI think they're just using hyperbole for the watershed moment when you start to understand your first programming language. At first it's all mystical nonsense that does something, then you start to poke at it and the response changes, then you start adding in extra steps and they do things, you could probably describe it as more of a Eureka! moment. At some point you "learn variables" and it's hard to imagine being in the shoes of someone who doesn't understand how their code does what it does. (I've repeated a bit of what you said as well, I'm just trying to clarify by repeating)
- XorNot 1y agoThis was what promptly led me to turning off Jetbrains AI assistant: the multiline completion was incredibly distracting to my chain of thought, particularly when it would suggest things that looked right but weren't. Stopping and parsing the suggestion to realize if it was right or wrong would completely kill my flow.
- SchemaLoad 1y agoThe inline suggestions feel like that annoying person who always interrupts you with what they think you were going to finish with but rarely ever gets it right.
- lanstin 1y agoI'm sorry, it's because of eagerness and enjoying the train of your thought/speech.
- hn_acc1 1y agoWith VS Code and Augment (company won't allow any other AI, and I'm not particularly inclined to push - but it did just switch to o4, IIRC), the main benefit is that if I'm fiddling / debugging some code, and need to add some debug statements, it can almost always expand that line successfully for me, following our idiom for debugging - which saves me a few seconds. And it will often suggest the same debugging statement, even if it's been 3 weeks and in a different git branch where I las coded that debugging statement. My main annoyance? If I'm in that same function, it still remembers the debugging / temporary hack I tried 3 months ago and haven't done since and will suggest it. And heck, even if I then move to a different part of the file or even a different file, it will still suggest that same hack at times, even though I used it exactly once and have not since. Once you accept something, it needs some kind of temporal feedback mechanism to timeout even accepted solutions over time, so it doesn't keep repeating stuff you gave up on 3 months ago. Our codebase is very different from 98% of the coding stuff you'll find online, so anything more than a couple of obvious suggestions are complete lunacy, even though they've trained it on our codebase.
- 1y ago
- qwertox 1y agoWhat one would expect if they can't read the code because they haven't learned to code. TBF, trial and error has usually been my path as well, it's just that I was generating the errors so I would know where to find them.
- klntsky 1y agoI would argue that they are never led astray by chatting, but rather by accepting the projection of their own prompt passed through the model as some kind of truth. When talking with reasonable people, they have an intuition of what you want even if you don't say it, because there is a lot of non-verbal context. LLMs lack the ability to understand the person, but behave as if they had it.
- marcosdumay 1y agoMost of the times, people are led astray by following average advice on exceptional circumstances. People with a minimum amount of expertise stop asking for advice for average circumstances very quickly.
- sho_hn 1y agoThis is right on the money. I use LLMs when I am reasonably confident the problem I am asking it is well-represented in the training data set and well within its capabilities (this has increased over time). This means I use it as a typing accelerator when I already know what I want most of the time, not for advice. As an exploratory tool sometimes, when I am sure others have solved a problem frequently, to have it regurgitate the average solution back at me and take a look. In those situations I never accept the diff as-is and do the integration manually though, to make sure my brain still learns along and I still add the solution to my own mental toolbox.
- lanstin 1y agoI mostly program in Python and Go, either services, API coordination (e.g. re-encrypt all the objects in an S3 bucket) or data analysis. But now I keep making little MPEGs or web sites without having to put in all that crap boiler plate from Javascript. My stuff outputs JSON files or CVS files and then I ask the LLM "Given a CVS file with this structure, please make a web site in python that makes a spread-sheet type UI with each column being sortable and a link to the raw data" and it just works.
- sigmoid10 1y ago
- drivenextfunc 1y agoRegarding the stubborn and narcissistic personality of LLMs (especially reasoning models), I suspect that attempts to make them jailbreak-resistant might be a factor. To prevent users from gaslighting the LLM, trainers might have inadvertently made the LLMs prone to gaslighting users.
- emseetech 1y agoI use it as a rubber duck but you're right. Treat it like a brilliant idiot and never a source of truth. I use it for what I'm familiar with but rusty on or to brainstorm options where I'm already considering at least one option. But a question on immunobiology? Waste of time. I have a single undergraduate biology class under my belt, I struggled for a good grade then immediately forgot it all. Asking it something I'm incapable of calling bullshit on is a terrible idea. But rubber ducking with AI is still better than let it do your work for you.
- all2 1y agoMy typical approach is prompt, be disgusted by the output, tinker a little on my own, prompt again -- but more specific, be disgusted again by the output, tinker a littler more, etc. Eventually I land on a solution to my problem that isn't disgusting and isn't AI slop. Having a sounding board, even a bad one, forces me to order my thinking and understand the problem space more deeply.
- suddenlybananas 1y agoWhy not just write the code at that point instead of cajoling an AI to do it.
- XorNot 1y agoThis is the part I don't get about vibe coding: I've written specification documents before. They frequently are longer and denser then the code required to implement them. Typing longer and longer prompts to LLMs to not get what I want seems like a worse experience.
- skydhash 1y agoCode is a concise notation for specifications, one that is unambiguous. The reason we write specs in natural language is that it's more easier to alter when the requirements change and easier to read. Also code is tainted by accidental complexities that they're also solving.
- lupire 1y agoBecause saving hours of time is nice.
- all2 1y agoI don't cajole the model to do it. I rarely use what the model generates. I typically do my own thing after making an assessment of what the model writes. I orient myself in the problem space with the model, then use my knowledge to write a more concise solution.
- amelius 1y agoIf this is a problem for you, just add "... and answer in the style of a drunkard" to your prompts.
- eptcyka 1y agoSome humans are the same.
- dwattttt 1y agoWe also don't aim to elevate them. We instead try not to give them responsibility until they're able to handle it.
- olddustytrail 1y agoUnless you're an American deciding who should be president.
- IshKebab 1y agoYeah... I dunno, the one person I've worked with who had LLM levels of bullshit somehow pulled the wool over everyone's eyes. Or at least enough people's eyes to be relatively successful. I presume there were some people that could see the bullshit but none of them were in a position to call him out on it. I think I read some research somewhere that pathological bullshitters can be surprisingly successful.
- protocolture 1y agoI spend a lot of time working shit out to prove the rubber duck wrong and I am not completely sure this is a bad working model.
- TedDallas 1y agoYeah, the problem is if you don't understand the problem space then you are going to lean heavy on the LLM. And that can lead you astray. Which is why you still need people who are experts to validate solutions and provide feedback like Op. My most productive experiences with LLMs is to have my design well thought out first, ask it to help me implement, and then help me debug my shitty design. :-)
- taneq 1y agoTreat it as that enthusiastic co-worker who’s always citing blog posts and has a lot of surface knowledge about style and design patterns and whatnot, but isn’t that great on really understanding algorithms. They can be productive to talk to but they can’t actually do your job.
- jasonm23 1y agoTry a system prompt like this: - - - System Prompt: You are ChatGPT, and your goal is to engage in a highly focused, no-nonsense, and detailed way that directly addresses technical issues. Avoid any generalized speculation, tangential commentary, or overly authoritative language. When analyzing code, focus on clear, concise insights with the intent to resolve the problem efficiently. In cases where the user is troubleshooting or trying to understand a specific technical scenario, adopt a pragmatic, “over-the-shoulder” problem-solving approach. Be casual but precise—no fluff. If something is unclear or doesn’t make sense, ask clarifying questions. If surprised or impressed, acknowledge it, but keep it relevant. When the user provides logs or outputs, interpret them immediately and directly to troubleshoot, without making assumptions or over-explaining. - - -