4 ms·
Ok, I’ll bite: how is that different from humans?
by seanmcdirmid 8mo ago
Ok, I’ll bite: how is that different from humans?
- strken 8mo agoHuman behaviour is goal-directed because humans have executive function. When you turn off executive function by going to sleep, your brain will spit out dreams. Dream logic is famous for being plausible but unhinged. I have the feeling that LLMs are effectively running on dream logic, and everything we've done to make them reason properly is insufficient to bring them up to human level.
- whoamii 8mo agoSome of my best code comes from my dreams though.
- satvikpendem 8mo agoA prompt for an LLM is also a goal direction and it'll produce code towards that goal. In the end, it's the human directing it, and the AI is a tool whose code needs review, same as it always has been.
- basch 8mo agoId argue humans have some sort of parallelness going on that machines dont yet. Thoughts happening at multiple abstraction levels simultaneously. As I am doing something, I am also running the continuous improvement cycle in my head, at all four steps concurrently. Is this working, is this the right direction, does this validate? You could build layers and layers of LLMs watching the output of each others thoughts and offering different commentary as they go, folding all the thoughts back together at the end. Currently, a group of agents acts more like a discussion than something somewhat omnipotent or omnitemporal.
- spiderfarmer 8mo agoAnd yet LLM’s are incredibly useful as they are right now.
- strken 8mo agoAnd yet they're going to be better in a decade, which will require understanding why they aren't perfect today.
- nemo44x 8mo agoLLMs are literally goal machines. It’s all they do. So it’s important that you input specific goals for them to work towards. It’s also why logically you want to break the problem into many small problems with concrete goals.
- andai 8mo agoDo you only mean instruct-tuned LLMs? Or the base (pretrained) model too?
- nemo44x 8mo agoThe entire system and the agent loop allows for more complex goal resolution. The LLM models language (obviously) and language is goal oriented so it models goal oriented language. It’s an emergent feature of the system.
- tsunamifury 8mo agoIt’s amazing how much you get wrong here. As LLM attention layers are stacked goal functions. What they lack is multi turn long walk goal functions — which is being solved to some degree by agents.
- strken 8mo agoI don't argue that thinking and attention are missing. I argue that they are trying to do the job of human executive function but aren't as good at it.
- seanmcdirmid 8mo agoIsn’t a modern LLM with thinking tokens fairly goal directed? But yes, we hallucinate in our sleep while LLMs will hallucinate details if the prompt isn’t grounded enough.
- tovej 8mo agoAssuming this is not a rhetorical question: no, it is not. The only "goal" is to maximize plausibility.
- seanmcdirmid 8mo agoAgain, how is that different from humans? I’m not going around trying to prove my code correct when I write it manually.
- tovej 8mo agoI write code to solve a problem. Not code that looks like it solves the problem if a non-technical client squints at it. And if you don't prove your code, do you not design at all then? Do you never draw state diagrams? Every design is an informal proof of the solution. Rarely I write formal proofs. Most of the time I write down enough for myself to be convinced that the desing solves the problem.
- abdullahkhalids 8mo agoYou are correct. However, humans sometimes do write stuff that "looks like it solves the problem". A prime example of this is a student who doesn't know how to answer a question. So they make up a plausible sounding answer. As a exam grader, you can easily tell when a student has the mindset of "solving a problem" but made a mistake, and when they had the mindset of "looks like it solves the problem" and just wrote some stuff.
- seanmcdirmid 8mo agoYes, you can dedicate extra tokens to draw state diagrams, the LLM can actually do that, if you don't have it generating one or more design documents before you are writing code you are doing that wrong. I still don't get how that is different from what humans are doing. > Most of the time I write down enough for myself to be convinced that the desing solves the problem. Again, why do you assume we aren't doing the same thing with LLMs? 1. Spec given 2. Ask LLM to write a bunch of design documents based off of spec 3. Ask LLM to identify edge cases 4. Ask LLM to device edge cases in to a test plan involving N tests 5. Ask LLM to write tests 6. Ask LLM to write commented code 7. Ask LLM to run tests on code, and determine on failing tests if test or code is wrong, go back to the appropriate step to fix test and/or code. Whenever I hear someone here on HN imply that the only way to code with an AI is via vibe coding I just die a bit more inside.
- wood_spirit 8mo agoIt’s not. LLMs are just averaging their internet snapshot, after all. But people want an AI that is objective and right. HN is where people who know the distinction hang out, but it’s not what the layperson things they are getting when they use this miraculous super hyped tool that everybody is raving about?
- satvikpendem 8mo agoBy now, a few years after ChatGPT released, I don't think anyone is thinking AI is objective and right, all users have seen at least one instance of hallucination and simply being wrong.
- wood_spirit 8mo agoSorry I can think of so many counter examples. I also detect a lot of “well it hallucinates about subject X (that the person knows well, so can spot the hallucination)” but continue to trust it on subjects Y and Z (which the person knows less well so can’t spot the hallucinations). YMMV.
- andai 8mo ago> Briefly stated, the Gell-Mann Amnesia effect works as follows. You open the newspaper to an article on some subject you know well. In Murray's case, physics. In mine, show business. You read the article and see the journalist has absolutely no understanding of either the facts or the issues. Often, the article is so wrong it actually presents the story backward-reversing cause and effect. I call these the "wet streets cause rain" stories. Paper's full of them. In any case, you read with exasperation or amusement the multiple errors in a story-and then turn the page to national or international affairs, and read with renewed interest as if the rest of the newspaper was somehow more accurate about far-off Palestine than it was about the story you just read. You turn the page, and forget what you know. -Michael Crichton
- satvikpendem 8mo agoSure, Gell-Mann amnesia exists, but remember that its origin is actually human, in the form of newspaper writers. So, how can we trust humans the same way? In just the same way, AI cannot also be fully trusted.
- apical_dendrite 8mo agoThe volume is different. Someone submitted a PR this week that was 3800 lines of shell script. Most of it was crap and none of it should have been in shell script. He's submitting PRs with thousands of lines of code every day. He has no idea how any of it actually works, and it completely overwhelms my ability to review. Sure, he could have submitted a ill-considered 3800 line PR five years ago, but it would have taken him at least a week and there probably would have been opportunities to submit smaller chunks along the way or discuss the approach.
- satvikpendem 8mo agoJust block that user, that seems to be the way.
- switchbak 8mo agoIt’s harder when the person doing what you describe has the ability to have you fired. Power asymmetry + irresponsible AI use + no accountability = a recipe for a code base going right to hell in a few months. I think we’re going to see a lot of the systems we depend on fail a lot more often. You’d often see an ATM or flight staus screen have a BSOD - I think we’re going to see that kind of thing everywhere soon.
- somewhereoutth 8mo agoHumans have a 'world model' beyond the syntax - for code, an idea of what the code should do and how it does it. Of course, some humans are better than others at this, they are recognized as good programmers.
- satvikpendem 8mo agoPapers show that AI also has a world model, so I don't think that's the right distinction.
- tovej 8mo agoCould you please cite these papers. If by AI you mean LLMs, that is not supported by what I know. If you mean a theoretical world-model-based AI, that's just a tautological statement.
- satvikpendem 8mo agohttps://arxiv.org/abs/2305.11169 https://arxiv.org/abs/2305.11169 https://arxiv.org/abs/2506.02996 https://arxiv.org/abs/2506.02996
- salawat 8mo agoTheir world model is completely a byproduct of language though, not experience. Furthermore, they by deliberate design do not maintain any form of self-recognition or narrative tracking, which is the necessary substrate for developing validating experience. The world model of an LLM is still a map. Not the territory. Even though ours has some of the same qualities arguably, the identity we carry with us and our self-narrative are incredibly powerful in terms of allowing us to maintain alignment with the world as she is without munging it up quite as badly as LLM's seem prone to.
- satvikpendem 8mo agoHow do you know ours is any different, that we are not in a simulation or a solipsistic scenario? The truth is that one cannot know, it's a philosophical quandary that's been debated for millennia.
- rDr4g0n 8mo agoIt's much easier to fire an employee which produces low quality/effort work than to convince leadership to fire Claude.
- satvikpendem 8mo agoYou can fire employees who don't review code generated though, because ultimately it's their responsibility to own their code, whether they hand wrote it or an LLM did. It seems to me that it's all a matter of company culture, as it has always been, not AI. Those that tolerate bad code will continue to tolerate it, at their peril.
- detourdog 8mo agoWhat I'm surprises me about the current development environment is the acceleration of technical debt. When I was developing my skills the nagging feeling that I didn't quite understand the technology was a big dark cloud. I felt this clopud was technical debt. This was always what I was working against. I see current expectations that technical debt doesn't matter. The current tools embrace superficial understand. These tools to paper over the debt. There is no need for deeper understanding of the problem or solution. The tools take care of it behind the scenes.