15 ms·

This matches my experience. I actually think a fair amount of value from LLM assistants to me is having a reasonably intelligent rubber duck to talk to. Now the

This matches my experience. I actually think a fair amount of value from LLM assistants to me is having a reasonably intelligent rubber duck to talk to. Now the duck can occasionally disagree and sometimes even refine. https://en.m.wikipedia.org/wiki/Rubber_duck_debugging https://en.m.wikipedia.org/wiki/Rubber_duck_debugging I think the big question everyone wants to skip right to and past this conversation is, will this continue to be true 2 years from now? I don’t know how to answer that question.

marcosdumay 1y ago
LLMs will still be this way 10 years from now. But IDK if somebody won't create something new that gets better. But there is no reason at all to extrapolate our current AIs into something that solves programing. Whatever constraints that new thing will have will be completely unrelated to the current ones.
- smokel 1y ago
  Stating this without any arguments is not very convincing. Perhaps you remember that language models were completely useless at coding some years ago, and now they can do quite a lot of things, even if they are not perfect. That is progress, and that does give reason to extrapolate. Unless of course you mean something very special with "solving programming".
  - marcosdumay 1y ago
    Why state the same arguments everybody has been repeating for ages? LLMs can only give you code that somebody has wrote before. This is inherent. This is useful for a bunch of stuff, but that bunch won't change if OpenAI decides to spend the GDP of Germany training one instead of Costa Rica.
    - rhubarbtree 1y ago
      That’s not true. LLMs are great translators, they can translate ideas to code. And that doesn’t mean it has to be recalling previously seen text.
    - Epa095 1y ago
      First, how much of coding is really never done before? And secondly, what you say are false (at least if taken literally). I can create a new programming language, give the definition of it in the prompt, ask it to code something in my language, and expect something out. It might even work.
      - apwell23 1y ago
        > how much of coding is really never done before? A lot because we use libraries for 'done frequently before' code. i don't generate a database driver for my webapp with llm.
        Epa095 1y ago
        We use libraries for SOME of the 'done frequently' code. But how much of enterprise programming is 'get some data from a database, show it on a Web page (or gui), store some data in the database', with variants? It makes sense that we have libraries for abstraction away some common things. But it also makes sense that we can't abstract away everything we do multiple times, because at some point it just becomes so abstract that it's easier to write it yourself than to try to configure some library. Does not mean that it's not a variant of something done before.
        deleted 1y ago
        [deleted]
        apwell23 1y ago
        > it's easier to write it yourself than to try to configure some library yeah unfortunately LLM will make this worse. Why abstract when you can generate. I am already seeing this a lot at work :(
        sunrunner 1y ago
        > we can't abstract away everything we do multiple times I think there's a fundamental truth about any code that's written which is that it exists on some level of specificity, or to put it in other words, a set of decisions have been made about _how_ something should work (in the space of what _could_ work) while some decisions have been left open to the user. Every library that is used is essentially this. Database driver? Underlying I/O decisions are probably abstracted away already (think Netty vs Mina), and decisions on how to manage connections, protocol handling, bind variables, etc. are made by the library, while questions remain for things like which specific tables and columns should be referenced. This makes the library reusable for this task as long as you're fine with the underlying decisions. Once you get to the question of _which specific data is shown on a page_ the decisions are closer to the human side of how we've arbitrarily chosen to organise things in this specific thousandth-iteration of an e-commerce application. The devil is in the details (even if you know the insides of the devil aren't really any different).
    - nahbruheem 1y ago
      Generating unseen code is not hard. Set rules on what’s valid, which most languages already do; omit generation of known code; generate everything else The computer does the work, programmers don’t have to think it up. A typed language example to explain; generate valid func sigs func f(int1, int2) return int{} If that’s our only func sig in our starting set then it makes it obvious Well relative to our tiny starter set func f(int1, int2, int3) return int{} is novel This Redis post is about fixing a prior decision of a random programmer. A linguistics decision. That’s why LLMs seem worse than programmers because we make linguistics decisions that fit social idioms. If we just want to generate all the never before seen in this model code we don’t need a programmer. If we need to abide laws of a flexible language nature, that’s what a programmer is for; compose not just code by compliance with ground truth. That antirez is good at Redis is a bias since he has context unseen by the LLM. Curious how well antirez would do with an entirely machine generated Redis-clone that was merely guided by experts. Would his intuition for Redis’ implementation be useful to a completely unknown implementation? He’d make a lot of newb errors and need mentorship, I’m guessing.
      - nprateem 1y ago
        I think we're hoping for more than the 'infinite monkeys bashing out semantically correct code' approach.
        nahbruheem 1y ago
        Ok, define what means and make it. Then as soon as you do realize you run into Gödel’s understanding your machine doesn’t solve problems related to its own existence and needs outside help. So you need to generate that yet unseen solution that lacks context for understanding itself… repeat and see it’s exactly generating one yet unseen layer of logic after another. Read the article; his younger self failed to see logic needed now. Add that onion peel. No such thing as perfect clairvoyance. Even Yann LeCun’s energy based models driving robots have the same experience problem. Make a computer that can observe all of the past and future. Without perfect knowledge our robots will fail to predict some composition of space time before they can adapt. So there’s no probe we can launch that’s forever and generally able to survive with our best guess when launched. More people need to study physical experiments and physics and not the semantic rigor of academia. No matter how many ideas we imagine there is no violating physics. Pop culture seems to have people feeling starship Enterprise is just about to launch from dry dock.
    - deleted 1y ago
      [deleted]
    - JoshCole 1y ago
      > LLMs can only give you code that somebody has wrote before. This premise is false. It is fundamentally equivalent to the claim that a language model being trained on a dataset: ["ABA", "ABB"] would be unable to generate, given input "B" the string "BAB" or "BAA".
      - 1718627440 1y ago
        Isn't the claim, that it will never make up "C"?
        JoshCole 1y ago
        They don't claim that. They say LLMs only generate text someone has written. Another way you could refute their premise was by showing the existence of AI-created programs for which someone isn't a valid description of the writer (e.g., from evolutionary algorithms) then training a network on that data such that it can output it. It is just as trivial a way to prove that the premise is false. Your claim here is slightly different. You're claiming that if a token isn't supported, it can't be output [1]. But we can easily disprove this by adding minimal support for all tokens, making C appear in theory. Such support addition shows up all the time in AI literature [2]. [1]: https://en.wikipedia.org/wiki/Support_(mathematics) https://en.wikipedia.org/wiki/Support_(mathematics) [2]: In some regimes, like game theoretic learning, support is baked into the solving algorithms explicitly during the learning stage. In others, like reinforcement learning, its accomplished by making the policy a function of two objectives, one an exploration objective, another an exploitation objective. That existing cross pollination already occurs between LLMs in the pre-trained unsupervised regime and LLMs in the post-training fine-tuning via forms of reinforcement learning regime should cause someone to hesitate to claim that such support addition is unreasonable if they are versed in ML literature. Edit: Got downvoted, so I figure maybe people don't understand. Here is the simple counterexample. Consider an evaluator that gives rewards: F("AAC") = 1, all other inputs = 0. Consider a tokenization that defines "A", "B", "C" as tokens, but a training dataset from which the letter C is excluded but the item "AAA" is present. After training "AAA" exists in the output space of the language model, but "AAC" does not. Without support, without exploration, if you train the language model against the reinforcement learning reward model of F, you might get no ability to output "C", but with support, the sequence "AAC" can be generated and give a reward. Now actually do this. You get a new language model. Since "AAC" was rewarded, it is now a thing within the space of the LLM outputs. Yet it doesn't appear in the training dataset and there are many reward models F for which no person will ever have had to output the string "AAC" in order for the reward model to give a reward for it. It follows that "C" can appear even though "C" does not appear in the training data.
    - vidarh 1y ago
      > LLMs can only give you code that somebody has wrote before. This is inherent. This is trivial to prove to be false. Invent a programming language that does not exist. Describe its semantics to an LLM. Ask it to write a program to solve a problem in that language. It will not always work, but it will work often enough to demonstrate that they are very much capable of writing code that has never been written before. The first time I tried this was with GPT3.5, and I had it write code in an unholy combination of Ruby and INTERCAL, and it had no problems doing that. Similarly giving it a grammar of a hypothetical language, and asking it to generate valid text in a language that has not existed before also works reasonably well. This notion that LLMs only spit out things that has been written before might have been reasonable to believe a few years ago, but it hasn't been a reasonable position to hold for a long time at this point.
      - shrewduser 1y ago
        This doesn't surprise me, i find LLM's are really good at interpolating and translating. so if i made up a language and gave it the rules and asked it to translate i wouldn't expect it to be bad at it.
        vidarh 1y ago
        It shouldn't surprise anyone, but it is clear evidence against the claim I replied to, and clearly a lot of people still hold on to this irrational assumption that they can't produce anything new.
        skydhash 1y ago
        They're not producing anything new... If you give it the answer before asking the question, no wonder it can answer. Prompting is to find resonance in the patterns extracted from the training data, which is why it fails spectacularly for exotic programming languages.
        vidarh 1y ago
        When you invent a language and tell it express something in that language, you've not given it the answer before asking the question. That's an utterly bizarre notion. The answer in question never existed before. By your definition humans never produce anything new either, because we always also extrapolate on patterns from our previous knowledge. > it fails spectacularly for exotic programming languages. My experience is that it not just succeeds for "exotic" languages, but for languages that didn't exist prior to the prompt. In other words, they can code at least simple programs even with zero-shot by explaining semantics of a language without giving them even a single example of programs in that language. Did you even read the comment you replied to above? To quote myself: "Invent a programming language that does not exist." I've had this work both for "from scratch" descriptions of languages by providing grammars, and for "combine feature A from language X, and feature B from language Y". In the latter case you might have at least an argument. In the former case you do not. Most humans struggle with tasks like this - you're setting a bar for LLMs most humans would fail to meet.
  - Retric 1y ago
    Progress sure, but the rate the’ve improved hasn’t been particularly fast recently. Programming has become vastly more efficient in terms of programmer effort over decades, but making some aspects of the job more efficient just means all your effort it spent on what didn’t improve.
    - mirsadm 1y ago
      The latest batch of LLMs has been getting worse in my opinion. Claude in particular seems to be going backwards with every release. The verbosity of the answers is infuriating. You ask it a simple question and it starts by inventing the universe, poorly
    - lexandstuff 1y ago
      People seem to have forgotten how good the 2023 GPT-4 really was at coding tasks.
  - apwell23 1y ago
    > Perhaps you remember that language models were completely useless at coding some years ago no i don't remember that. They are doing similar things now that they did 3 yrs ago. They were still a decent rubber duck 3 yrs ago.
    - vidarh 1y ago
      And 6 years ago GPT2 had just been released. You're being obtuse by interpreting "some years" as specifically 3.
    - deleted 1y ago
      [deleted]
  - bigstrat2003 1y ago
    > Perhaps you remember that language models were completely useless at coding some years ago, and now they can do quite a lot of things, even if they are not perfect. IMO, they're still useless today, with the only progress being that they can produce a more convincing facade of usefulness. I wouldn't call that very meaningful progress.
    - IshKebab 1y ago
      They are very clearly not useless. You haven't given them a fair shake.
    - drdeca 1y ago
      I’ve found them somewhat useful? Not for big things, and not for code for work. But for small personal projects? Yes, helpful.
      - conradkay 1y ago
        It's funny how there's a decent % of people at both "LLMs are useless" and "LLMs 3-10x my productivity"
        otabdeveloper4 1y ago
        > LLMs 3-10x my productivity x10 of zero is still zero, I guess.
    - wvenable 1y ago
      I don't know how someone can legitimately say that they're useless. Perfect, no. But useless, also no.
      - addaon 1y ago
        > I don't know how someone can legitimately say that they're useless. Clearly, statistical models trained on this HN thread would output that sequence of tokens with high probability. Are you suggesting that a statement being probable in a text corpus is not a legitimate source of truth? Can you generalize that a little bit?
gerad 1y ago
It's like chess. Humans are better for now, they won't be forever, but humans plus software is going to better than either alone for a long time.
- vFunct 1y ago
  No guarantee that will happen. LLMs are still statistically based. It's not going to give you edgier ideas, like filling a glass of wine to the rim. Use them for the 90% of your repetitive uncreative work. The last 10% is up to you.
  - skydhash 1y ago
    The pain of that 90% work is how you get libraries and framework. Imagine having many different implementation of sorting algorithms inside your codebase.
    - vFunct 1y ago
      OK now we have to spend time figuring out the framework. It's why people say just write plain Javascript, for example.
      - skydhash 1y ago
        Is it? Or is it because the framework are not suitable for the project?
- apwell23 1y ago
  its not like chess
- kelseydh 1y ago
  The time where humans + computers in chess were better than just computers was not a long time. That era ended well over a decade ago. Might have been true for only 3-5 years.
  - 1y ago
_tom_ 1y ago
For me, it's a bit like pair programming. I have someone to discuss ideas with. Someone to review my code and suggest alternative approaches. Some one that uses different feature than I do, so I learn from them.
- platevoltage 1y ago
  This is how I use it too. It's great at quickly answering questions. I find it particularly useful if I have to work with a language of framework that I'm not fully experienced in.
  - 12_throw_away 1y ago
    > I find it particularly useful if I have to work with a language of framework that I'm not fully experienced in Yep - my number 1 use case for LLMs is as a template and example generator. It actually seems like a fairly reasonable use for probabilistic text generation!
- traceroute66 1y ago
  I guess if you enjoy programming with someone you can never really trust, then yeah, sure, its "a bit like" pairs programming.
  - mock-possum 1y ago
    Trust, but verify ;]
marcosdumay 1y ago
It's a damning assertive duck, completely out of proportion to its competence. I've seen enough people led astray by talking to it.
- foxyv 1y ago
  Same here. When I'm teaching coding I've noticed that LLMs will confuse the heck out of students. They will accept what it suggests without realizing that it is suggesting nonsense.
  - cogogo 1y ago
    I’m self taught and don’t code that much but I feel like I benefit a ton from LLMs giving me specific answers to questions that would take me a lot of time to figure out with documentation and stack overflow. Or even generating snippets that I can evaluate whether or not will work. But I actually can’t imagine how you can teach someone to code if they have access to an LLM from day one. It’s too easy to take the easy route and you lose the critical thinking and problem solving skills required to code in the first place and to actually make an LLM useful in the second. Best of luck to you… it’s a weird time for a lot of things. *edit them/they
    - ilamont 1y ago
      > I’m self taught and don’t code that much but I feel like I benefit a ton from LLMs giving me specific answers to questions that would take me a lot of time to figure out with documentation and stack overflow Same here. Combing discussion forums and KB pages for an hour or two, seeking how to solve a certain problem with a specific tool has been replaced by a 50-100 word prompt in Gemini which gives very helpful replies, likely derived from many of those same forums and support docs. Of course I am concerned about accuracy, but for most low-level problems it's easy enough to test. And you know what, many of those forum posts or obsolete KB articles had their own flaws, too.
      - specproc 1y ago
cortesoft 1y ago
Currently, I find AI to be a really good autocomplete
- jdiff 1y ago
  The crazy thing is that people think that a model designed to predict sequences of tokens from a stem, no matter how advanced the model, to be much more than just "really good autocomplete." It is impressive and very unintuitive just how far that can get you, but it's not reductive to use that label. That's what it is on a fundamental level, and aligning your usage with that will allow it to be more effective.
  - fl7305 1y ago
    > "The crazy thing is that people think that a model designed to" It's even crazier that some people believe that humans "evolved" intelligence just by nature selecting the genes which were best at propagating. Clearly, human intelligence is the product of a higher being designing it. /s
    - fhd2 1y ago
      I would consider evolution a form of intelligence, even though I wouldn't consider nature a being. There's a branch of AI research I was briefly working in 15 years ago, based on that premise: Genetic algorithms/programming. So I'd argue humans were (and are continuously being) designed, in a way.
      - fl7305 1y ago
        (non-sarcastically from me this time) Sure, I would agree with that wording. In the same way, neural networks which are trained to do a task could be said to be "designed" to do something. In my view, there's a big difference in what the training data is for a neural network, and what the neural network is "designed" for. We can train a network using word completion examples, with the intent of designing it for intelligence.
joshdavham 1y ago
> I actually think a fair amount of value from LLM assistants to me is having a reasonably intelligent rubber duck to talk to. I wonder if the term "rubber duck debugging" will still be used much longer into the future.
- layer8 1y ago
  As long as it remains in the training material, it will be used. ;)
Bukhmanizer 1y ago
There are a couple people I work with who clearly don’t have a good understanding of software engineering. They aren’t bad to work with and are in fact great at collaborating and documenting their work, but don’t seem to have the ability to really trace through code and logically understand how it works. Before LLMs it was mostly fine because they just didn’t do that kind of work. But now it’s like a very subtle chaos monkey has been unleashed. I’ve asked on some PRs “why is this like this? What is it doing?” And the answer is “ I don’t know, ChatGPT told me I should do it.” The issue is that it throws basically all their code under suspicion. Some of it works, some of it doesn’t make sense, and some of it is actively harmful. But because the LLMs are so good at giving plausible output I can’t just glance at the code and see that it’s nonsense. And this would be fine if we were working on like a crud app where you can tell what is working and broken immediately, but we are working on scientific software. You can completely mess up the results of a study and not know it if you don’t understand the code.
- jajko 1y ago
  Sounds almost like you definitely shouldnt use llms nor those juniors for such an important work. Is it just me or are we heading into a period of explosion of software done, but also a massive drop of its quality? Not uniformly, just a bit of chaotic spread
  - palmotea 1y ago
    > Is it just me or are we heading into a period of explosion of software done, but also a massive drop of its quality? Not uniformly, just a bit of chaotic spread I think we are, especially with executives mandating the use LLMs use and expecting it to massively reduce costs and increase output. For the most part they don't actually seem to care that much about software quality, and tend to push to decrease quality at every opportunity.
  - jrochkind1 1y ago
    Which is frightening, because it's not like our industry is known for producing really high quality code at the starting point before LLM authored code.
bossyTeacher 1y ago
I think of them as highly sycophant LSD-minded 2nd year student who has done some programming
ortusdux 1y ago
> I think the big question everyone wants to skip right to and past this conversation is, will this continue to be true 2 years from now? I don’t know how to answer that question. I still think about Tom Scott's 'where are we on the AI curve' video from a few years back. https://www.youtube.com/watch?v=jPhJbKBuNnA https://www.youtube.com/watch?v=jPhJbKBuNnA
schwartzworld 1y ago
For me it's like having a junior developer work under me who knows APIs inside and out, but has no common sense about architecture. I like that I delegate tasks to them so that my brain can be free for other problems, but it makes my job much more review heavy than before. I put every PR through 3-4 review cycles before even asking my team for a review.
- eslaught 1y ago
  How do you not completely destroy your concentration when you do this though? I normally build things bottom up so that I understand all the pieces intimately and when I get to the next level of abstraction up, I know exactly how to put them together to achieve what I want. In my (admittedly limited) use of LLMs so far, I've found that they do a great job of writing code, but that code is often off in subtle ways. But if it's not something I'm already intimately familiar with, I basically need to rebuild the code from the ground up to get to the point where I understand it well enough so that I can see all those flaws. At least with humans I have some basic level of trust, so that even if I don't understand the code at that level, I can scan it and see that it's reasonable. But every piece of LLM generated code I've seen to date hasn't been trustworthy once I put in the effort to really understand it.
  - schwartzworld 1y ago
    I use a few strategies, but it's mostly the same as if I was mentoring a junior. A lot of my job already involved breaking up big features into small tickets. If the tasks are small enough, juniors and LLMs have an easier time implementing things and I have an easier time reviewing. If there's something I'm really unfamiliar with, it should be in a dedicated function backed by enough tests that my understanding of the implementation isn't required. In fact, LLMs do great with TDD! > At least with humans I have some basic level of trust, so that even if I don't understand the code at that level, I can scan it and see that it's reasonable. If you can't scan the code and see that it's reasonable, that's a smell. The task was too big or its implemented the wrong way. You'd feel bad telling a real person to go back and rewrite it a different way but the LLM has no ego to bruise. I may have a different perspective because I already do a lot of review, but I think using LLMs means you have to do more of it. What's the excuse for merging code that is "off" in any way? The LLM did it? It takes a short time to review your code, give your feedback to the LLM and put up something actually production ready. > But every piece of LLM generated code I've seen to date hasn't been trustworthy once I put in the effort to really understand it. That's why your code needs tests. More tests. If you can't test it, it's wrong and needs to be rewritten.
empath75 1y ago
Just the exercise of putting my question in a way that the LLM could even theoretically provide a useful response is enough for me to figure out how to solve the problem a good percentage of the time.
koonsolo 1y ago
It seems to me we're at the flat side of the curve again. I haven't seen much real progress in the last year. It's ignorant to think machines will not catch up to our intelligence at some point, but for now, it's clearly not. I think there needs to be some kind of revolutionary breakthrough again to reach the next stage. If I were to guess, it needs to be in the learning/back propagation stage. LLM's are very rigid, and once they go wrong, you can't really get them out of it. A junior develop for example could gain a new insight. LLM's, not so much.
Waterluvian 1y ago
Same. Just today I used it to explore how a REST api should behave in a specific edge case. It gave lots of confident opinions on options. These were full of contradictions and references to earlier paragraphs that didn’t exist (like an option 3 that never manifested). But just by reading it, I rubber ducked the solution, which wasn’t any of what it was suggesting.
p1necone 1y ago
> the duck can occasionally disagree This has not been my experience. LLMs have definitely been helpful, but generally they either give you the right answer or invent something plausible sounding but incorrect. If I tell it what I'm doing I always get breathless praise, never "that doesn't sound right, try this instead."
- crazygringo 1y ago
  That's not my experience. I routinely get a polite "that might not be the optimal solution, have you considered..." when I'm asking whether I should do something X way with Y technology. Of course it has to be something the LLM actually has lots of material it's trained with. It won't work with anything remotely cutting-edge, but of course that's not what LLM's are for. But it's been incredibly helpful for me in figuring out the best, easiest, most idiomatic ways of using libraries or parts of libraries I'm not very familiar with.
  - Jarwain 1y ago
    I find it very much depends on the LLM you're using. Gemini feels more likely to push back than claude 3.7 is. Haven't tried claude 4 yet
- mbrameld 1y ago
  Ask it. Instead of just telling it what you're doing and expecting it to criticize that, ask it directly for criticism. Even better, tell it what you're doing, then tell it to ask you questions about what you're doing until it knows enough to recommend a better approach.
  - lupire 1y ago
    This is key. Humans each have a personality and some sense of mood. When you ask for help, you choose ask and that person can sense your situation. LLM has every personality and doesn't know your situation. You have to tell it which personality to use and what your situation is.
bandoti 1y ago
My take is that AI is very one-dimensional (within its many dimensions). For instance, I might close my eyes and imagine an image of a tree structure, or a hash table, or a list-of-trees, or whatever else; then I might imagine grabbing and moving the pieces around, expanding or compressing them like a magician; my brain connects sight and sound, or texture, to an algorithm. However people think about problems is grounded in how we perceive the world in its infinite complexity. Another example: saying out loud the colors red, blue, yellow, purple, orange, green—each color creates a feeling that goes beyond its physical properties into the emotions and experiences. AI image-generation might know the binary arrangement of an RGBA image but actually, it has NO IDEA what it is to experience colour. No idea how to use the experience of colour to teach a peer of an algorithm. It regurgitates a binary representation. At some point we’ll get there though—no doubt. It would be foolish to say never! For those who want to get there before everyone else probably should focus on the organoids—because most powerful things come from some Faustian monstrosity.
- eddd-ddde 1y ago
  This is really funny to read as someone who CANNOT imagine anything more complex than the most simple shape like a circle. Do you actually see a tree with nodes that you can rearrange and have the nodes retain their contents and such?
  - bandoti 1y ago
    Haha—yeah, for me the approach is always visual. I have to draw a picture to really wrap my brain around things! Other people I’d imagine have their own human, non-AI way to organize a problem space. :) I have been drawing all my life and studied traditional animation though, so it’s probably a little bit of nature and nurture.
johnnyanmac 1y ago
>I think the big question everyone wants to skip right to and past this conversation is, will this continue to be true 2 years from now? For me, it's less "conversation to be skipped" and more about "can we even get to 2 years from now"? There's so much insability right now that it's hard to say what anything will look like in 6 months. "
Buttons840 1y ago
LLMs aren't my rubber duck, they're my wrong answer. You know that saying that the best way to get an answer online is to post a wrong answer? That's what LLMs do for me. I ask the LLM to do something simple but tedious, and then it does it spectacularly wrong, then I get pissed off enough that I have the rage-induced energy to do it myself.
- Buttons840 1y ago
  I'm probably suffering undiagnosed ADHD, and will get stuck and spend minutes picking a function name and then writing a docstring. LLMs do help with this even if they get the code wrong, because I usually won't bother to fix their variables names or docstring unless needed. LLMs can reliably solve the problem of a blank-page.
  - linotype 1y ago
    This. I have ADHD and starting is the hardest part for me. With an LLM it gets me from 0 to 20% (or more) and I can nail it for the rest. It’s way less stressful for me to start now.
    - raihansaputra 1y ago
      very much agree. although lately with how good it is i get hyperfocused and spent more time then i allocated because i ended up wanting to implement more than i planned.
      - linotype 1y ago
        It’s a struggle right? First world LLM problems.
    - bayesianbot 1y ago
      Been suffering the same, I'm used to having so many days (weeks/months) when I just don't get that much done. With LLMs I can take these days and hack around / watch videos / play games while the LLM is working on background and just check the work. Best part is it often leads to some problematic situation that gets me involved and often I'll end up getting a real day of work out of it after I get started.
mock-possum 1y ago
Yeah in my experience as long as you don’t stray too far off the beaten path, LLMs are great at just parroting conventional wisdom for how to implement things - but the second you get to something more complicated - or especially tricky bug fixing that requires expensive debuggery - forget about it, they do more harm than good. Breaking down complex tasks into bite sized pieces you can reasonably expect the robot to perform is part of the art of the LLM.
akshay_trikha 1y ago
I've had this same thought that it would be nice to have an AI rubber ducky to bounce ideas off of while pair programming (so that you don't sound dumb to your coworkers & waste their time). This is my first comment so I'm not sure how to do this but I made a BYO-API key VSCode extension that uses the OpenAI realtime API so you can have interactive voice conversations with a rubber ducky. I've been meaning to create a Show HN post about it but your comment got me excited! In the future I want to build features to help people communicate their bugs / what strategies they've tried to fix them. If I can pull it off it would be cool if the AI ducky had a cursor that it could point and navigate to stuff as well. Please let me know if you find it useful https://akshaytrikha.github.io/deep-learning/2025/05/23/ducky.html https://akshaytrikha.github.io/deep-learning/2025/05/23/duck...
- akadeb 1y ago
  I like the sound of that! I think youre gonna like what we are building here https://github.com/akdeb/ElatoAI https://github.com/akdeb/ElatoAI Its as if the rubber duck was actually on the desk while youre programming and if we have an MCP that can get live access to code it could give you realtime advice.
  - akshay_trikha 1y ago
    Wow, that's really cool thanks for open sourcing! I might dig into your MCP I've been meaning to learn how to do that. I genuinely think this could be great for toys that kids grow up with i.e. the toy could adjust the way it talks depending on the kids age and remember key moments in their life - could be pretty magical for a kid
- AdieuToLogic 1y ago
  > I've had this same thought that it would be nice to have an AI rubber ducky to bounce ideas off of while pair programming (so that you don't sound dumb to your coworkers & waste their time). I humbly suggest a more immediate concern to rectify is identifying how to improve the work environment such that the fear one might "sound dumb to your coworkers & waste their time" does not exist.
- 1y ago
travisgriggs 1y ago
LLMs are a passel of eager to please know it all interns that you can command at will without any moral compunctions. They drive you nuts trying to communicate with them what you actually want them to do. They have a vast array of facts at immediate recall. They’ll err in their need to produce and please. They do the dumbest things sometimes. And surprise you at other times. You’ll throw vast amounts of their work away or have to fix it. They’re (relatively) cheap. So as an army of monkeys, if you keep herding them, you can get some code that actually tells a story. Mostly.
hoppp 1y ago
Same. I do rubber duck debugging too and found the LLM to compliment it nicely. Looking forward for rubber duck shaped hardware AI interfaces to talk to in the future. Im sure somebody will create it

marcosdumay 1y ago

LLMs will still be this way 10 years from now. But IDK if somebody won't create something new that gets better. But there is no reason at all to extrapolate our current AIs into something that solves programing. Whatever constraints that new thing will have will be completely unrelated to the current ones.

smokel 1y ago
Stating this without any arguments is not very convincing. Perhaps you remember that language models were completely useless at coding some years ago, and now they can do quite a lot of things, even if they are not perfect. That is progress, and that does give reason to extrapolate. Unless of course you mean something very special with "solving programming".
- marcosdumay 1y ago
  Why state the same arguments everybody has been repeating for ages? LLMs can only give you code that somebody has wrote before. This is inherent. This is useful for a bunch of stuff, but that bunch won't change if OpenAI decides to spend the GDP of Germany training one instead of Costa Rica.
  - rhubarbtree 1y ago
    That’s not true. LLMs are great translators, they can translate ideas to code. And that doesn’t mean it has to be recalling previously seen text.
  - Epa095 1y ago
    First, how much of coding is really never done before? And secondly, what you say are false (at least if taken literally). I can create a new programming language, give the definition of it in the prompt, ask it to code something in my language, and expect something out. It might even work.
    - apwell23 1y ago
      > how much of coding is really never done before? A lot because we use libraries for 'done frequently before' code. i don't generate a database driver for my webapp with llm.
      - Epa095 1y ago
        We use libraries for SOME of the 'done frequently' code. But how much of enterprise programming is 'get some data from a database, show it on a Web page (or gui), store some data in the database', with variants? It makes sense that we have libraries for abstraction away some common things. But it also makes sense that we can't abstract away everything we do multiple times, because at some point it just becomes so abstract that it's easier to write it yourself than to try to configure some library. Does not mean that it's not a variant of something done before.
        deleted 1y ago
        [deleted]
        apwell23 1y ago
        > it's easier to write it yourself than to try to configure some library yeah unfortunately LLM will make this worse. Why abstract when you can generate. I am already seeing this a lot at work :(
        sunrunner 1y ago
        > we can't abstract away everything we do multiple times I think there's a fundamental truth about any code that's written which is that it exists on some level of specificity, or to put it in other words, a set of decisions have been made about _how_ something should work (in the space of what _could_ work) while some decisions have been left open to the user. Every library that is used is essentially this. Database driver? Underlying I/O decisions are probably abstracted away already (think Netty vs Mina), and decisions on how to manage connections, protocol handling, bind variables, etc. are made by the library, while questions remain for things like which specific tables and columns should be referenced. This makes the library reusable for this task as long as you're fine with the underlying decisions. Once you get to the question of _which specific data is shown on a page_ the decisions are closer to the human side of how we've arbitrarily chosen to organise things in this specific thousandth-iteration of an e-commerce application. The devil is in the details (even if you know the insides of the devil aren't really any different).
  - nahbruheem 1y ago
    Generating unseen code is not hard. Set rules on what’s valid, which most languages already do; omit generation of known code; generate everything else The computer does the work, programmers don’t have to think it up. A typed language example to explain; generate valid func sigs func f(int1, int2) return int{} If that’s our only func sig in our starting set then it makes it obvious Well relative to our tiny starter set func f(int1, int2, int3) return int{} is novel This Redis post is about fixing a prior decision of a random programmer. A linguistics decision. That’s why LLMs seem worse than programmers because we make linguistics decisions that fit social idioms. If we just want to generate all the never before seen in this model code we don’t need a programmer. If we need to abide laws of a flexible language nature, that’s what a programmer is for; compose not just code by compliance with ground truth. That antirez is good at Redis is a bias since he has context unseen by the LLM. Curious how well antirez would do with an entirely machine generated Redis-clone that was merely guided by experts. Would his intuition for Redis’ implementation be useful to a completely unknown implementation? He’d make a lot of newb errors and need mentorship, I’m guessing.
    - nprateem 1y ago
      I think we're hoping for more than the 'infinite monkeys bashing out semantically correct code' approach.
      - nahbruheem 1y ago
        Ok, define what means and make it. Then as soon as you do realize you run into Gödel’s understanding your machine doesn’t solve problems related to its own existence and needs outside help. So you need to generate that yet unseen solution that lacks context for understanding itself… repeat and see it’s exactly generating one yet unseen layer of logic after another. Read the article; his younger self failed to see logic needed now. Add that onion peel. No such thing as perfect clairvoyance. Even Yann LeCun’s energy based models driving robots have the same experience problem. Make a computer that can observe all of the past and future. Without perfect knowledge our robots will fail to predict some composition of space time before they can adapt. So there’s no probe we can launch that’s forever and generally able to survive with our best guess when launched. More people need to study physical experiments and physics and not the semantic rigor of academia. No matter how many ideas we imagine there is no violating physics. Pop culture seems to have people feeling starship Enterprise is just about to launch from dry dock.
  - deleted 1y ago
    [deleted]
  - JoshCole 1y ago
    > LLMs can only give you code that somebody has wrote before. This premise is false. It is fundamentally equivalent to the claim that a language model being trained on a dataset: ["ABA", "ABB"] would be unable to generate, given input "B" the string "BAB" or "BAA".
    - 1718627440 1y ago
      Isn't the claim, that it will never make up "C"?
      - JoshCole 1y ago
        They don't claim that. They say LLMs only generate text someone has written. Another way you could refute their premise was by showing the existence of AI-created programs for which someone isn't a valid description of the writer (e.g., from evolutionary algorithms) then training a network on that data such that it can output it. It is just as trivial a way to prove that the premise is false. Your claim here is slightly different. You're claiming that if a token isn't supported, it can't be output [1]. But we can easily disprove this by adding minimal support for all tokens, making C appear in theory. Such support addition shows up all the time in AI literature [2]. [1]: https://en.wikipedia.org/wiki/Support_(mathematics) https://en.wikipedia.org/wiki/Support_(mathematics) [2]: In some regimes, like game theoretic learning, support is baked into the solving algorithms explicitly during the learning stage. In others, like reinforcement learning, its accomplished by making the policy a function of two objectives, one an exploration objective, another an exploitation objective. That existing cross pollination already occurs between LLMs in the pre-trained unsupervised regime and LLMs in the post-training fine-tuning via forms of reinforcement learning regime should cause someone to hesitate to claim that such support addition is unreasonable if they are versed in ML literature. Edit: Got downvoted, so I figure maybe people don't understand. Here is the simple counterexample. Consider an evaluator that gives rewards: F("AAC") = 1, all other inputs = 0. Consider a tokenization that defines "A", "B", "C" as tokens, but a training dataset from which the letter C is excluded but the item "AAA" is present. After training "AAA" exists in the output space of the language model, but "AAC" does not. Without support, without exploration, if you train the language model against the reinforcement learning reward model of F, you might get no ability to output "C", but with support, the sequence "AAC" can be generated and give a reward. Now actually do this. You get a new language model. Since "AAC" was rewarded, it is now a thing within the space of the LLM outputs. Yet it doesn't appear in the training dataset and there are many reward models F for which no person will ever have had to output the string "AAC" in order for the reward model to give a reward for it. It follows that "C" can appear even though "C" does not appear in the training data.
  - vidarh 1y ago
    > LLMs can only give you code that somebody has wrote before. This is inherent. This is trivial to prove to be false. Invent a programming language that does not exist. Describe its semantics to an LLM. Ask it to write a program to solve a problem in that language. It will not always work, but it will work often enough to demonstrate that they are very much capable of writing code that has never been written before. The first time I tried this was with GPT3.5, and I had it write code in an unholy combination of Ruby and INTERCAL, and it had no problems doing that. Similarly giving it a grammar of a hypothetical language, and asking it to generate valid text in a language that has not existed before also works reasonably well. This notion that LLMs only spit out things that has been written before might have been reasonable to believe a few years ago, but it hasn't been a reasonable position to hold for a long time at this point.
    - shrewduser 1y ago
      This doesn't surprise me, i find LLM's are really good at interpolating and translating. so if i made up a language and gave it the rules and asked it to translate i wouldn't expect it to be bad at it.
      - vidarh 1y ago
        It shouldn't surprise anyone, but it is clear evidence against the claim I replied to, and clearly a lot of people still hold on to this irrational assumption that they can't produce anything new.
        skydhash 1y ago
        They're not producing anything new... If you give it the answer before asking the question, no wonder it can answer. Prompting is to find resonance in the patterns extracted from the training data, which is why it fails spectacularly for exotic programming languages.
        vidarh 1y ago
        When you invent a language and tell it express something in that language, you've not given it the answer before asking the question. That's an utterly bizarre notion. The answer in question never existed before. By your definition humans never produce anything new either, because we always also extrapolate on patterns from our previous knowledge. > it fails spectacularly for exotic programming languages. My experience is that it not just succeeds for "exotic" languages, but for languages that didn't exist prior to the prompt. In other words, they can code at least simple programs even with zero-shot by explaining semantics of a language without giving them even a single example of programs in that language. Did you even read the comment you replied to above? To quote myself: "Invent a programming language that does not exist." I've had this work both for "from scratch" descriptions of languages by providing grammars, and for "combine feature A from language X, and feature B from language Y". In the latter case you might have at least an argument. In the former case you do not. Most humans struggle with tasks like this - you're setting a bar for LLMs most humans would fail to meet.
- Retric 1y ago
  Progress sure, but the rate the’ve improved hasn’t been particularly fast recently. Programming has become vastly more efficient in terms of programmer effort over decades, but making some aspects of the job more efficient just means all your effort it spent on what didn’t improve.
  - mirsadm 1y ago
    The latest batch of LLMs has been getting worse in my opinion. Claude in particular seems to be going backwards with every release. The verbosity of the answers is infuriating. You ask it a simple question and it starts by inventing the universe, poorly
  - lexandstuff 1y ago
    People seem to have forgotten how good the 2023 GPT-4 really was at coding tasks.
- apwell23 1y ago
  > Perhaps you remember that language models were completely useless at coding some years ago no i don't remember that. They are doing similar things now that they did 3 yrs ago. They were still a decent rubber duck 3 yrs ago.
  - vidarh 1y ago
    And 6 years ago GPT2 had just been released. You're being obtuse by interpreting "some years" as specifically 3.
  - deleted 1y ago
    [deleted]
- bigstrat2003 1y ago
  > Perhaps you remember that language models were completely useless at coding some years ago, and now they can do quite a lot of things, even if they are not perfect. IMO, they're still useless today, with the only progress being that they can produce a more convincing facade of usefulness. I wouldn't call that very meaningful progress.
  - IshKebab 1y ago
    They are very clearly not useless. You haven't given them a fair shake.
  - drdeca 1y ago
    I’ve found them somewhat useful? Not for big things, and not for code for work. But for small personal projects? Yes, helpful.
    - conradkay 1y ago
      It's funny how there's a decent % of people at both "LLMs are useless" and "LLMs 3-10x my productivity"
      - otabdeveloper4 1y ago
        > LLMs 3-10x my productivity x10 of zero is still zero, I guess.
  - wvenable 1y ago
    I don't know how someone can legitimately say that they're useless. Perfect, no. But useless, also no.
    - addaon 1y ago
      > I don't know how someone can legitimately say that they're useless. Clearly, statistical models trained on this HN thread would output that sequence of tokens with high probability. Are you suggesting that a statement being probable in a text corpus is not a legitimate source of truth? Can you generalize that a little bit?

gerad 1y ago

It's like chess. Humans are better for now, they won't be forever, but humans plus software is going to better than either alone for a long time.

vFunct 1y ago
No guarantee that will happen. LLMs are still statistically based. It's not going to give you edgier ideas, like filling a glass of wine to the rim. Use them for the 90% of your repetitive uncreative work. The last 10% is up to you.
- skydhash 1y ago
  The pain of that 90% work is how you get libraries and framework. Imagine having many different implementation of sorting algorithms inside your codebase.
  - vFunct 1y ago
    OK now we have to spend time figuring out the framework. It's why people say just write plain Javascript, for example.
    - skydhash 1y ago
      Is it? Or is it because the framework are not suitable for the project?
apwell23 1y ago
its not like chess
kelseydh 1y ago
The time where humans + computers in chess were better than just computers was not a long time. That era ended well over a decade ago. Might have been true for only 3-5 years.
- 1y ago

_tom_ 1y ago

For me, it's a bit like pair programming. I have someone to discuss ideas with. Someone to review my code and suggest alternative approaches. Some one that uses different feature than I do, so I learn from them.

platevoltage 1y ago
This is how I use it too. It's great at quickly answering questions. I find it particularly useful if I have to work with a language of framework that I'm not fully experienced in.
- 12_throw_away 1y ago
  > I find it particularly useful if I have to work with a language of framework that I'm not fully experienced in Yep - my number 1 use case for LLMs is as a template and example generator. It actually seems like a fairly reasonable use for probabilistic text generation!
traceroute66 1y ago
I guess if you enjoy programming with someone you can never really trust, then yeah, sure, its "a bit like" pairs programming.
- mock-possum 1y ago
  Trust, but verify ;]

marcosdumay 1y ago

It's a damning assertive duck, completely out of proportion to its competence. I've seen enough people led astray by talking to it.

foxyv 1y ago
Same here. When I'm teaching coding I've noticed that LLMs will confuse the heck out of students. They will accept what it suggests without realizing that it is suggesting nonsense.
- cogogo 1y ago
  I’m self taught and don’t code that much but I feel like I benefit a ton from LLMs giving me specific answers to questions that would take me a lot of time to figure out with documentation and stack overflow. Or even generating snippets that I can evaluate whether or not will work. But I actually can’t imagine how you can teach someone to code if they have access to an LLM from day one. It’s too easy to take the easy route and you lose the critical thinking and problem solving skills required to code in the first place and to actually make an LLM useful in the second. Best of luck to you… it’s a weird time for a lot of things. *edit them/they
  - ilamont 1y ago
    > I’m self taught and don’t code that much but I feel like I benefit a ton from LLMs giving me specific answers to questions that would take me a lot of time to figure out with documentation and stack overflow Same here. Combing discussion forums and KB pages for an hour or two, seeking how to solve a certain problem with a specific tool has been replaced by a 50-100 word prompt in Gemini which gives very helpful replies, likely derived from many of those same forums and support docs. Of course I am concerned about accuracy, but for most low-level problems it's easy enough to test. And you know what, many of those forum posts or obsolete KB articles had their own flaws, too.
    - specproc 1y ago

cortesoft 1y ago

Currently, I find AI to be a really good autocomplete

jdiff 1y ago
The crazy thing is that people think that a model designed to predict sequences of tokens from a stem, no matter how advanced the model, to be much more than just "really good autocomplete." It is impressive and very unintuitive just how far that can get you, but it's not reductive to use that label. That's what it is on a fundamental level, and aligning your usage with that will allow it to be more effective.
- fl7305 1y ago
  > "The crazy thing is that people think that a model designed to" It's even crazier that some people believe that humans "evolved" intelligence just by nature selecting the genes which were best at propagating. Clearly, human intelligence is the product of a higher being designing it. /s
  - fhd2 1y ago
    I would consider evolution a form of intelligence, even though I wouldn't consider nature a being. There's a branch of AI research I was briefly working in 15 years ago, based on that premise: Genetic algorithms/programming. So I'd argue humans were (and are continuously being) designed, in a way.
    - fl7305 1y ago
      (non-sarcastically from me this time) Sure, I would agree with that wording. In the same way, neural networks which are trained to do a task could be said to be "designed" to do something. In my view, there's a big difference in what the training data is for a neural network, and what the neural network is "designed" for. We can train a network using word completion examples, with the intent of designing it for intelligence.

joshdavham 1y ago

> I actually think a fair amount of value from LLM assistants to me is having a reasonably intelligent rubber duck to talk to. I wonder if the term "rubber duck debugging" will still be used much longer into the future.

layer8 1y ago
As long as it remains in the training material, it will be used. ;)

Bukhmanizer 1y ago

There are a couple people I work with who clearly don’t have a good understanding of software engineering. They aren’t bad to work with and are in fact great at collaborating and documenting their work, but don’t seem to have the ability to really trace through code and logically understand how it works. Before LLMs it was mostly fine because they just didn’t do that kind of work. But now it’s like a very subtle chaos monkey has been unleashed. I’ve asked on some PRs “why is this like this? What is it doing?” And the answer is “ I don’t know, ChatGPT told me I should do it.” The issue is that it throws basically all their code under suspicion. Some of it works, some of it doesn’t make sense, and some of it is actively harmful. But because the LLMs are so good at giving plausible output I can’t just glance at the code and see that it’s nonsense. And this would be fine if we were working on like a crud app where you can tell what is working and broken immediately, but we are working on scientific software. You can completely mess up the results of a study and not know it if you don’t understand the code.

jajko 1y ago
Sounds almost like you definitely shouldnt use llms nor those juniors for such an important work. Is it just me or are we heading into a period of explosion of software done, but also a massive drop of its quality? Not uniformly, just a bit of chaotic spread
- palmotea 1y ago
  > Is it just me or are we heading into a period of explosion of software done, but also a massive drop of its quality? Not uniformly, just a bit of chaotic spread I think we are, especially with executives mandating the use LLMs use and expecting it to massively reduce costs and increase output. For the most part they don't actually seem to care that much about software quality, and tend to push to decrease quality at every opportunity.
- jrochkind1 1y ago
  Which is frightening, because it's not like our industry is known for producing really high quality code at the starting point before LLM authored code.

bossyTeacher 1y ago

I think of them as highly sycophant LSD-minded 2nd year student who has done some programming

ortusdux 1y ago

> I think the big question everyone wants to skip right to and past this conversation is, will this continue to be true 2 years from now? I don’t know how to answer that question. I still think about Tom Scott's 'where are we on the AI curve' video from a few years back. https://www.youtube.com/watch?v=jPhJbKBuNnA https://www.youtube.com/watch?v=jPhJbKBuNnA

schwartzworld 1y ago

For me it's like having a junior developer work under me who knows APIs inside and out, but has no common sense about architecture. I like that I delegate tasks to them so that my brain can be free for other problems, but it makes my job much more review heavy than before. I put every PR through 3-4 review cycles before even asking my team for a review.

eslaught 1y ago
How do you not completely destroy your concentration when you do this though? I normally build things bottom up so that I understand all the pieces intimately and when I get to the next level of abstraction up, I know exactly how to put them together to achieve what I want. In my (admittedly limited) use of LLMs so far, I've found that they do a great job of writing code, but that code is often off in subtle ways. But if it's not something I'm already intimately familiar with, I basically need to rebuild the code from the ground up to get to the point where I understand it well enough so that I can see all those flaws. At least with humans I have some basic level of trust, so that even if I don't understand the code at that level, I can scan it and see that it's reasonable. But every piece of LLM generated code I've seen to date hasn't been trustworthy once I put in the effort to really understand it.
- schwartzworld 1y ago
  I use a few strategies, but it's mostly the same as if I was mentoring a junior. A lot of my job already involved breaking up big features into small tickets. If the tasks are small enough, juniors and LLMs have an easier time implementing things and I have an easier time reviewing. If there's something I'm really unfamiliar with, it should be in a dedicated function backed by enough tests that my understanding of the implementation isn't required. In fact, LLMs do great with TDD! > At least with humans I have some basic level of trust, so that even if I don't understand the code at that level, I can scan it and see that it's reasonable. If you can't scan the code and see that it's reasonable, that's a smell. The task was too big or its implemented the wrong way. You'd feel bad telling a real person to go back and rewrite it a different way but the LLM has no ego to bruise. I may have a different perspective because I already do a lot of review, but I think using LLMs means you have to do more of it. What's the excuse for merging code that is "off" in any way? The LLM did it? It takes a short time to review your code, give your feedback to the LLM and put up something actually production ready. > But every piece of LLM generated code I've seen to date hasn't been trustworthy once I put in the effort to really understand it. That's why your code needs tests. More tests. If you can't test it, it's wrong and needs to be rewritten.

empath75 1y ago

Just the exercise of putting my question in a way that the LLM could even theoretically provide a useful response is enough for me to figure out how to solve the problem a good percentage of the time.

koonsolo 1y ago

It seems to me we're at the flat side of the curve again. I haven't seen much real progress in the last year. It's ignorant to think machines will not catch up to our intelligence at some point, but for now, it's clearly not. I think there needs to be some kind of revolutionary breakthrough again to reach the next stage. If I were to guess, it needs to be in the learning/back propagation stage. LLM's are very rigid, and once they go wrong, you can't really get them out of it. A junior develop for example could gain a new insight. LLM's, not so much.

Waterluvian 1y ago

Same. Just today I used it to explore how a REST api should behave in a specific edge case. It gave lots of confident opinions on options. These were full of contradictions and references to earlier paragraphs that didn’t exist (like an option 3 that never manifested). But just by reading it, I rubber ducked the solution, which wasn’t any of what it was suggesting.

p1necone 1y ago

> the duck can occasionally disagree This has not been my experience. LLMs have definitely been helpful, but generally they either give you the right answer or invent something plausible sounding but incorrect. If I tell it what I'm doing I always get breathless praise, never "that doesn't sound right, try this instead."

crazygringo 1y ago
That's not my experience. I routinely get a polite "that might not be the optimal solution, have you considered..." when I'm asking whether I should do something X way with Y technology. Of course it has to be something the LLM actually has lots of material it's trained with. It won't work with anything remotely cutting-edge, but of course that's not what LLM's are for. But it's been incredibly helpful for me in figuring out the best, easiest, most idiomatic ways of using libraries or parts of libraries I'm not very familiar with.
- Jarwain 1y ago
  I find it very much depends on the LLM you're using. Gemini feels more likely to push back than claude 3.7 is. Haven't tried claude 4 yet
mbrameld 1y ago
Ask it. Instead of just telling it what you're doing and expecting it to criticize that, ask it directly for criticism. Even better, tell it what you're doing, then tell it to ask you questions about what you're doing until it knows enough to recommend a better approach.
- lupire 1y ago
  This is key. Humans each have a personality and some sense of mood. When you ask for help, you choose ask and that person can sense your situation. LLM has every personality and doesn't know your situation. You have to tell it which personality to use and what your situation is.

bandoti 1y ago

My take is that AI is very one-dimensional (within its many dimensions). For instance, I might close my eyes and imagine an image of a tree structure, or a hash table, or a list-of-trees, or whatever else; then I might imagine grabbing and moving the pieces around, expanding or compressing them like a magician; my brain connects sight and sound, or texture, to an algorithm. However people think about problems is grounded in how we perceive the world in its infinite complexity. Another example: saying out loud the colors red, blue, yellow, purple, orange, green—each color creates a feeling that goes beyond its physical properties into the emotions and experiences. AI image-generation might know the binary arrangement of an RGBA image but actually, it has NO IDEA what it is to experience colour. No idea how to use the experience of colour to teach a peer of an algorithm. It regurgitates a binary representation. At some point we’ll get there though—no doubt. It would be foolish to say never! For those who want to get there before everyone else probably should focus on the organoids—because most powerful things come from some Faustian monstrosity.

eddd-ddde 1y ago
This is really funny to read as someone who CANNOT imagine anything more complex than the most simple shape like a circle. Do you actually see a tree with nodes that you can rearrange and have the nodes retain their contents and such?
- bandoti 1y ago
  Haha—yeah, for me the approach is always visual. I have to draw a picture to really wrap my brain around things! Other people I’d imagine have their own human, non-AI way to organize a problem space. :) I have been drawing all my life and studied traditional animation though, so it’s probably a little bit of nature and nurture.

johnnyanmac 1y ago

>I think the big question everyone wants to skip right to and past this conversation is, will this continue to be true 2 years from now? For me, it's less "conversation to be skipped" and more about "can we even get to 2 years from now"? There's so much insability right now that it's hard to say what anything will look like in 6 months. "

Buttons840 1y ago

LLMs aren't my rubber duck, they're my wrong answer. You know that saying that the best way to get an answer online is to post a wrong answer? That's what LLMs do for me. I ask the LLM to do something simple but tedious, and then it does it spectacularly wrong, then I get pissed off enough that I have the rage-induced energy to do it myself.

Buttons840 1y ago
I'm probably suffering undiagnosed ADHD, and will get stuck and spend minutes picking a function name and then writing a docstring. LLMs do help with this even if they get the code wrong, because I usually won't bother to fix their variables names or docstring unless needed. LLMs can reliably solve the problem of a blank-page.
- linotype 1y ago
  This. I have ADHD and starting is the hardest part for me. With an LLM it gets me from 0 to 20% (or more) and I can nail it for the rest. It’s way less stressful for me to start now.
  - raihansaputra 1y ago
    very much agree. although lately with how good it is i get hyperfocused and spent more time then i allocated because i ended up wanting to implement more than i planned.
    - linotype 1y ago
      It’s a struggle right? First world LLM problems.
  - bayesianbot 1y ago
    Been suffering the same, I'm used to having so many days (weeks/months) when I just don't get that much done. With LLMs I can take these days and hack around / watch videos / play games while the LLM is working on background and just check the work. Best part is it often leads to some problematic situation that gets me involved and often I'll end up getting a real day of work out of it after I get started.

mock-possum 1y ago

Yeah in my experience as long as you don’t stray too far off the beaten path, LLMs are great at just parroting conventional wisdom for how to implement things - but the second you get to something more complicated - or especially tricky bug fixing that requires expensive debuggery - forget about it, they do more harm than good. Breaking down complex tasks into bite sized pieces you can reasonably expect the robot to perform is part of the art of the LLM.

akshay_trikha 1y ago

I've had this same thought that it would be nice to have an AI rubber ducky to bounce ideas off of while pair programming (so that you don't sound dumb to your coworkers & waste their time). This is my first comment so I'm not sure how to do this but I made a BYO-API key VSCode extension that uses the OpenAI realtime API so you can have interactive voice conversations with a rubber ducky. I've been meaning to create a Show HN post about it but your comment got me excited! In the future I want to build features to help people communicate their bugs / what strategies they've tried to fix them. If I can pull it off it would be cool if the AI ducky had a cursor that it could point and navigate to stuff as well. Please let me know if you find it useful https://akshaytrikha.github.io/deep-learning/2025/05/23/ducky.html https://akshaytrikha.github.io/deep-learning/2025/05/23/duck...

akadeb 1y ago
I like the sound of that! I think youre gonna like what we are building here https://github.com/akdeb/ElatoAI https://github.com/akdeb/ElatoAI Its as if the rubber duck was actually on the desk while youre programming and if we have an MCP that can get live access to code it could give you realtime advice.
- akshay_trikha 1y ago
  Wow, that's really cool thanks for open sourcing! I might dig into your MCP I've been meaning to learn how to do that. I genuinely think this could be great for toys that kids grow up with i.e. the toy could adjust the way it talks depending on the kids age and remember key moments in their life - could be pretty magical for a kid
AdieuToLogic 1y ago
> I've had this same thought that it would be nice to have an AI rubber ducky to bounce ideas off of while pair programming (so that you don't sound dumb to your coworkers & waste their time). I humbly suggest a more immediate concern to rectify is identifying how to improve the work environment such that the fear one might "sound dumb to your coworkers & waste their time" does not exist.
1y ago

travisgriggs 1y ago

LLMs are a passel of eager to please know it all interns that you can command at will without any moral compunctions. They drive you nuts trying to communicate with them what you actually want them to do. They have a vast array of facts at immediate recall. They’ll err in their need to produce and please. They do the dumbest things sometimes. And surprise you at other times. You’ll throw vast amounts of their work away or have to fix it. They’re (relatively) cheap. So as an army of monkeys, if you keep herding them, you can get some code that actually tells a story. Mostly.

hoppp 1y ago

Same. I do rubber duck debugging too and found the LLM to compliment it nicely. Looking forward for rubber duck shaped hardware AI interfaces to talk to in the future. Im sure somebody will create it