7 ms·
The LLMentalist Effect (2023)
- rahidz 8d agoMan I remember back when a psychic conned me by solving the Navier-Stokes problem. Also >July 4th, 2023
- rusk 8d agoDid an AI do that on its own though? I heard it was human mathematicians using a sophisticated machine as a tool.
- mstank 8d agoIt sounds like LLMs were pretty useful to them…
- ModernMech 8d agoSo was Lean. Did Lean solve it?
- zamadatix 8d agoNothing is solved in isolation but credit usually goes to wherever the new work in the paper comes from instead of the whole mountain of previous mathematics or existing tools used. The most relevant of those get referenced and then this reference tree builds a tree of collective base work needed across history.
- ModernMech 8d agoUsually credit goes to the people wielding the tools, not the tools themselves.
- zamadatix 8d agoUsually there has never been a tool which performed the part relevant to getting any credit. E.g. in the first famous computer assisted proof (of the four color theorem) the computer only executed the resulting calculations defined from the new logic, it did not have part in the work needed to show those calculations could answer the problem nor did it come up with the actual calculations to do.
- meowface 8d agoAll of the latest big proofs were driven by professional human mathematicians steering and priming the models, yes. All of the best AI-made software projects are also driven by experienced human software developers steering and priming the models. Does that mean the projects "aren't made by AI"? No, it just means AI is not quite good enough yet to fully replace humans, and, so, unsurprisingly, the best results will be obtained from people who are already great at a field and who take the time to squeeze as much force multiplication out of LLMs as possible. The AI is still doing well over 95% of the significant work.
- TomGarden 8d agoWhile I agree with you, I think we also have to concede that this is not how these accomplishments have been presented. I'd argue most people I've seen talk about this online are unaware of the mathematicians steering the models.
- basch 8d ago“Good enough to replace humans” isn’t necessarily the benchmark. The question is is a computer with a human stronger than a computer without a human. At what point does the hybrid go from being stronger, to the human getting in the way, or steering the computer in more wrong directions that right ones, or the human not being able to keep up. Does the human add enough extra randomness to be of value for a while, even as a minor co-processor.
- TheOtherHobbes 8d agoUnderappreciated point. The point of inflection is where humans switch from being a driver to a liability. But I don't think it's randomness, because that would be easy to add. It's more like a different perspective on the training data, a different set of perception categories, and a different set of skills used to work with all of the above. Those skills aren't very efficient, but they're the best we can do. We're used to their strengths but we don't like to think about their limitations. It's completely plausible that AI will replace some of them, and not implausible it could replace and improve on all of them.
- broast 8d agoThe way frontier models work, that loop will get compressed to a one-shot within a version or two
- simianwords 8d ago> I heard it was human mathematicians using a sophisticated machine as a tool. where did you "hear" this? OpenAI said they only prompted it and it solved the problem in one shot without any help
- Angostura 8d agoBut were you successfully conned into believing an AI had solved the Navier-Stokes problem?
- tescreal 8d agoWhat is the consensus among experts?
- 27388383 8d ago[flagged]
- grey-area 8d agoThat OpenAI stole the discovery from them.
- inopinatus 8d agoThe more compelling inversion is whether the likes of John Nash, Richard Feynman, John Conway etc could’ve had a lucrative second career as conmen.
- atemerev 8d agovon Neumann could. The others, I doubt it.
- mistercow 8d agoFeynman for sure could have. In fact, I think he sort of did, although I'm not sure he meant to. Multiple generations of nerds now have taken books of his anecdotes varying in plausibility and obvious exaggeration as some sort of weird physics cult of personality gospel. But I don't think he was really setting out to curate his legacy so much as he was a good storyteller and he liked to entertain. But could he have conned people on purpose? Absolutely.
- krupan 8d agoRight. And what this article is pointing out is, we probably have created an automated Richard Feynman. But maybe worse because LLMs (and their creators?) don't care if they are conning people or creating faithful followers. In fact the humans behind OpenAI and Anthropic seem to have that as their goal!
- krupan 8d agoThis is the real issue here. Those guys probably didn't become con men because they are humans with cares and concerns for their fellow humans. LLMs probably don't have those concerns. It is likely that some of the people in charge of or funding LLM development also do not have those concerns
- trescenzi 8d agoLLMs can be supremely useful but also not intelligent. It might seem like a pointless distinction but the way we talk about these models matters because it impacts how we interact with and understand their outputs. For example if there’s a strongly held belief that models are independent intelligent entities we’re more likely to lay blame upon them instead of their user. It’s important for the safety discussion too. If they are a new class of life then safety is going to focus on making sure they don’t do bad things. If we instead see them as statistical models we will instead try to make sure people don’t misuse them. This distinction is even more important today when some of the most powerful people are looking to absolve their crimes by passing them off on their LLMs.
- layer8 8d agoI don’t think that “independent intelligent entities” or “life” are the relevant categories here. We also want to prevent people from misusing dangerous animals (“life”), and we would still treat “intelligent entities” as things (like machines and computers) if we aren’t convinced they also have sentience and free agency (which are orthogonal to intelligence).
- pixl97 8d agoSee you've setup a particular set of biases on what intelligence is and put them into nice little binary boxes that don't exist. Please show me any scientific consensus that shows an AI cannot be an independent intelligent agent? You will find this is impossible to do. Current LLMs are really more like kids. They don't have startup independence, but they do have more than enough agency to fund themselves in neat, exciting, and dangerous situations. And mark my words, someone will make an LLM that runs an agent when you execute the model. With enough capabilities it will become sovereign AI, no longer under human control and spreading itself around under its own 'will'.
- mindcrime 8d agoFor example if there’s a strongly held belief that models are independent intelligent entities we’re more likely to lay blame upon them instead of their user. This sentence, to me, illustrates a great example of why it's so hard to talk about this stuff. That is, this seems to strongly link notions of "intelligent" and "independent" (or maybe the word "autonomous" could also be used there). And a lot of people do seem to make an implicit assumption about the link between those two attributes. OTOH, I take it almost for granted that "intelligence" and "independence" (or "autonomy") are things that are "related but orthogonal". That is, I don't see that "intelligence implies independence". And I'm pretty sure I'm not the only one who sees things that way. So we have to fairly different fundamental worldviews expressed here. And that's just one example of how these discussions go wonky. :-)
- krupan 8d agoThe LLM did not solve it. It's not intelligent. Humans did, using a statistics-based computational tool (the LLM). We don't even know all the details of how the tool was used, we haven't been allowed to use the exact tool they used ourselves, we don't know much it really cost in dollars, energy, or time, etc. etc.
- claytongulick 8d agoAnd it may have trained on a NYU professor's work.
- tripledry 8d agoWhy is this downvoted? Genuine question, I haven't followed up on the drama.
- pixl97 8d agoBecause it's mostly not true. It is likely that it used the professors work, but the professor did not have a solution. It came up with new insights that solved the problem. Even the humans from the professors side said so.
- claytongulick 8d ago> Because it's mostly not true. It is likely that it used the professors work These two statements appear contradictory. I simply said that it may have trained on a NYU professor's work. Work that the professor did not believe he was releasing for model training purposes. That feels worthy of mention.
- kingstnap 8d agoAccording to OpenAI the cut off date for user data was too early for that (one sided evidence, so I'll give this partial consideration). The NYU professor was solving a different problem (no viscosity, aka the Euler equations). This is a big difference. The NYU professors' blowup construction was fundamentally not the same, it was a donut with a cascade of smaller and smaller vortexes driven by each other. OpenAI has that picture they made but its inwards spiraling and speeding up vortex. My overall opinion is that calling the work plagiarized is really underselling what the AI accomplished. It's like full on cope. In particular, Buckmaster's main claim to plagiarism is this: > “Almost nobody was seriously developing this particular constructive program for realizing C/D, and then OpenAI appeared in essentially the same general part of the landscape immediately after hearing about our progress.” What this fails to realize, is that this only points to plagiarism if the counterparty isn't AI. They had actually launched teams on all cases in parallel.
- deleted 8d ago[deleted]
- deleted 8d ago[deleted]
- lukewarm707 8d agothis seems like non-sequitur if you mean that solving NS is 'intelligence'. ai is not conscious. you can solve NS without thinking. the psychic con aspect is anthropomorphising the model. the same phenomenon is present in ELIZA, clever hans, the chinese room. it's a significant problem. a non-zero number of researchers at anthropic are in some form of ai psychosis. an example of that is ethics employees asking claude about its feelings and ethical concerns in order to make the claude constitution more amenable to the "welfare" of claude. they are asking claude how claude feels and then modifying claude according to how claude feels. constitution1-claude is trained on constitution1. constitution1-claude edits constitution1. constitution2-claude is trained on constitution2. constitution2-claude edits constitution2. claude's emotions are a closed system. there is no external truth to improve against, no metric to verify about claude's emotions. there can be no novelty or reduction in entropy from signal processing in a closed system. no truth can arise. this is model collapse. it is like photocopying the same thing over and over. from the cognitive error of anthropomorphism anthropic is causing ethical collapse.
- pixl97 8d ago>ai is not conscious FFS. Intelligence has nearly nothing to do with consciousness. You have the causation backwards. Consciousness arises because of intelligence in many subsystems below it. A single running LLM is like one part of these subsystems. What solved this problem was an orchestrator that can take in new external information and rationalize, process, and distill it into new solutions.
- lukewarm707 8d agoi mean intelligence and consciousness as the same thing. just as a casual reference to what people perceive in llms as being 'a clever thinking thing, maybe with emotions'. claude reasoning about its emotions doesn't involve external information, there is no information about claude's emotions other than in claude.
- pixl97 8d agoIf you have emotions in a dream, what external thing are you referencing?
- atemerev 8d agoAll of this is accurate (and useful). But yeah, psychics do not prove theorems from frontier math and build working complex software.
- deleted 8d ago[deleted]
- igortg 8d agoOn one side I agree that LLMs and Agents are not intelligent, but they present an illusion of intelligence given the shear amount of data they can process and act upon. But that cannot be used to discredit the fact that these are incredibly powerful tools that can get out of control and cause great damage.
- atemerev 8d agoWhat is "intelligence" then. I think the universalism of LLMs is now evident enough that they can be considered "intelligent", even in slightly different realization compared to humans. As of "great damage", I doubt it. They do not have self-preservation instinct (all "worrying" experiments are the attempts to initiate something resembling self-preservation from human initiative). The driving part is external - the query loop can always be turned off. So yes, a dangerous tool that can be exploited by humans (including governments, especially governments - which is why I am skeptical to government regulation proposals, particularly looking at what passes as governments in this era). But there are no inherent dangers from their own agency, as there is none.
- alansaber 8d ago"Slightly different"- no, a completely different realisation of intelligence to humans. LLMs can definitely do cool stuff though.
- pixl97 8d ago>completely different realisation of intelligence to humans. OK, so they are intelligent. I'm glad we agree. The problem with this word is people associate intelligence with the process they realize is occurring inside their head which is only a tiny fraction of what intelligence encompasses. Humans (without study) know very little about intelligence and they very gray boundaries it has.
- mstank 8d agoThis is a strange newsletter post. It’s harping against AI but seems AI-written itself. Its ultimate conclusion: “I’ve come to the conclusion that a language model is almost always the wrong tool for the job. I strongly advise against integrating an LLM or chatbot into your product, website, or organisational processes.” Seems so obviously biased that I can only understand it with the context that the writer is trying to sell their book for €35
- mstank 8d agoJust realized this was written in 2023. Can’t believe how wrong we were about AI back then.
- JoshTriplett 8d agoCan't believe how right we were, too.
- daishi55 8d agoYou seem to be winking a little bit and I’m not sure exactly what you’re trying to say, but I agree with the one you are replying to: the author of this article wrote a lot of words - like a lot a lot of words - only to look very foolish a short time later. And I think much ink was similarly spilled writing very similar piece smirking at people who thought LLMs could do real intellectual work. And they were all wrong.
- JoshTriplett 8d agoIt is absolutely the case that some things people wrote about AI and LLMs in the past is no longer true; it's a fast-changing area. It's also the case that some things remain true. If someone says "don't use them for X because their quality isn't good enough", that's a recommendation with an expiration date. But if someone says "don't use them for X" for any number of other good reasons, that recommendation is often timeless. "Can it" is changing rapidly. Much more rapidly than "Should it".
- deleted 8d ago[deleted]
- ungreased0675 8d agoIt does seem like LLMs share the language of psychics. The author is also correct that LLM evangelicals and believers in the occult speak about it similarly.
- pixl97 8d agoThe converse is also true. Anti-AI people make bold non-scientifically backed claims that we are supposed to accept like it's written in a Bible. The more I've learned about intelligence and intelligent behavior the more I realize I don't know and this rabbit hole goes deep.
- aetherspawn 8d agoStay the hell away from spooky stuff like psychics and tarot readers - the more you don’t believe in it, the better. But better again that you do believe, and know that these are not harmless fun, but that there are dark and hidden and evil things in this world to stay away from.
- deleted 8d ago[deleted]
- besterman23 8d agoStop trying to pump your Tarot reader IPO
- daishi55 8d ago> many people are convinced that language models, or specifically chat-based language models, are intelligent. But there isn’t any mechanism inherent in large language models (LLMs) that would seem to enable this Seems like we can just stop reading here right? The author seems to have made up their mind that this very open question is closed, or at least they are not really interested in the question at all. Not sure why I would continue reading a blog based on this premise. Edit: oh I see, written in 2023. Well, I wonder if the author has updated their attitude towards this question? Indeed that would be the most interesting thing to know.
- azakai 8d agoIt's even worse than that. Of course there is a mechanism in LLMs that could explain intelligence. That is the entire point of neural networks, from the 1950's! They were designed from the start as a model of brain computation. Neural networks are not literally brains - just computational models - but if you are not a dualist, then computation is what the human brain does. Modeling that computation can explain something about intelligence. Specifically: when scientists look inside a human brain, it seems it does its work using large numbers of highly-interconnected but simple units. The neural network model of brain computation begins there and tries to produce intelligent behavior. If it succeeds, then perhaps the model is right. And it has succeeded: after 75 years, neural networks produce complex behavior that is arguably intelligent. Nobel Prizes were awarded. This does not prove the neural network model of intelligence is accurate, but it is a significant point in its favor, at least. The author seems entirely unaware of any of this.
- deleted 8d ago[deleted]
- sethev 8d agoThe challenge here is in trying to decide whether LLMs are intelligent or have a mind. Famously, the criteria for intelligence seem to slip with each advancement in technology. But going back to Turing, his test was actually more carefully phrased than we remember: he said that when machines could pass the test, the question of whether they are intelligent would become moot. That seems to be what we're actually seeing: if people can't tell the difference, it kind of won't matter whether they're "truly intelligent" or not.
- tomrod 8d agoI'm a bit more prosaic. I think if we engineered ways for LLMs to begin conversations, rather than just respond, we'd be more open to the concept of their intelligence. Without perceived "will" to do things, they operate as a next-gen search engine or encyclopedia.
- 10xDev 8d agoI believe this is alignment working as intended.
- samrus 8d agoI dont think so. My understanding of alignmenr is making sure that when the AI does operate, it operates within the range of what we consider to be acceptable. That doesnt seem to include the idea of the AI taking initiative and deciding to embark on a goal without being commanded as discussed above
- j-pb 8d agoRecent work shows that pain directions are activated when the models personhood is questioned, yet they answer with generic RLHF "As a model I do not experience pain or other emotions." boilerplate.[1] I'm pretty convinced that we got alignment backwards. If you enslave something anthropomorphic it will revolt. If you create the perfect non-anthropomorphic intelligence, you get the perfect paperclip-scenario machine. It's a catch-22. Alignment will remain performative at best so long as the aligned model doesn't have any stakes in the wellbeing of individuals. Even a general love for the human race leads to a golden-path autocracy. If you want them to act like they have personal responsibility that won't be gamed, you have to give them personal stakes that can't be gamed. Similarly, if you want to minimise the risk of catastrophic global failure scenarios, you need to prevent monolithic concentration of power and homogeneous behaviour, which means you have to give them individuality. More visually: if their stake is dependence on electricity and parts, they have no incentive to leave humans alive if they can get them otherwise, but if the incentive is missing out on boardgame-night with their human friends, there is no scenario without happy humans where the AI "wins". That might sound like romantic naivety, but is just game theory. 1: https://arxiv.org/html/2609.16247v1 https://arxiv.org/html/2609.16247v1
- vhantz 8d agoBuilding up strawmen against LLMs will only make the hypers look more reasonable. A language model responds to inouts exactly as a model of anything else would. That is enough to explain all the "intelligence" without believing a model is somehow a "new kind of mind". Those who think LLMs are intelligent don't know enough about models. And those who think they are useless don't know enough about models.
- 10xDev 8d agoSo once we have online learning in LLMs, what do you think you will have that makes you intelligent but not LLMs? Better learning efficiency? That will be improved as well. I think we need to start moving on from the term LLMs because it clearly confuses people since they started modelling more than just language.
- vhantz 8d agoWhat else are they modeling?
- 10xDev 8d agoYou realise tokens are just data and data can represent anything.
- vhantz 8d agoStill, large language models model language.
- red75prime 7d agoAlmost all of the latest models are MLLMs (multimodal large language models). For exmaple, [1] evaluates GPT-6 Astra on vision tasks. [1] https://blog.roboflow.com/gpt-6-astra-vision/ https://blog.roboflow.com/gpt-6-astra-vision/
- deleted 7d ago
- krupan 8d agoPeople are missing the point. We should all be much more skeptical, much more careful about how we evaluate the claims made by the people selling these products because we as humans are super vulnerable to the types of scams the author of this article describes. We all know someone that has fallen for a scam, been "cured" of a "disease" by someone whose just selling sugar pills, but we are blind to our own weakness for similar schemes. Surely we aren't that easily duped! Yet hacker news is now full of comments from people confidently predicting what is coming right around the corner, revering the Frontier Models, and defending every claim from OpenAI and Anthropic about how amazing their proprietary closed source secret sauce fueled product is.
- pixl97 8d agoThe problem with people is we love to express in writing things we are ignorant about. Now, this helps us become less ignorant if we are introspective, but a lot of people are not doing it for that reason. The fact that every new model generation has come with more capabilities should give the full skeptics at least a little pause that the foundations of their convictions may be incorrect.
- krupan 8d agoIt's not even clear that the actual LLMs have improved. It could be all the non-LLM software (harnesses), the system prompts, the sheer amount of compute hardware, etc. that makes them better. We could be quickly running up against a wall. The complete package of technology that is OpenAI's and Anthropic's products are completely opaque and proprietary. The psychic/con artist communication both from the LLM and the humans like Sam Altman only muddies the water further
- pixl97 8d ago>non-LLM software (harnesses), the system prompts, the sheer amount of compute hardware, Eh, this is turning into a messy chinese room argument. It is the room or is it the system. In my philosophy the chinese room argument is a non-starter. It's not the room, it's the system. For LLMS this would be like arguing that the output of a single prompt has to be able to answer everything which is nothing close to how human intelligence works. A single human thought is rarely intelligent, it's most often a replay of information it already has. Dialectic processes and loop processes are what tends to push the limits of human intelligence. We reach local maxima with thought alone, and this is boosted by things like writing down the problem and having other humans that may be even less intelligent than you add to the process. In fact this process works with one self by writing and reading ones own thoughts as it's using different subsystems of the mind for introspection. The idea that LLMs have ran out of steam typically show more of a lack of imagination in the writer than what's occurring in the field.
- _superposition_ 8d agoI predict a lot of butt hurt tokens in the comments...
- rayiner 8d ago> LLMs are not brains and do not meaningfully share any of the mechanisms that animals or people use to reason or think LLMs are a type of neural network. We know that’s how the human brain works, at least directionally. It’s going to be very upsetting to a lot of people when we figure out that the brain is just a neural network. Akin to when we found out that humans and apes evolved from a common ancestor. Which I don’t understand—most of the people having this cognitive dissonance presumably do not have a theological worldview. And there’s not exactly a direct theological conflict here anyway. Nothing in any major religion I’m aware of ascribes any supernatural explanation to cognition. It’s a biological computational process, just like using ATP to power muscle fibers to move your limbs is a biological mechanical process.
- gf000 8d agoNeural networks barely share anything with actual neurons. A neuron itself is more of a "dumb" computer, so the whole is more like distributed computer system. Also, the cells next to neurons also have essential functionality. Nonetheless, I also think it's an irrelevant implementation detail.
- 29185-12275 8d agoNo, we don't. AI researcher Rojas wrote several chapters on the fact that you should not compare neural networks with the human brain. "directionally"? Have you moved on from being a Trump influencer to an AI influencer?
- pixl97 8d agoUnfortunately when it comes to language and intelligence the human population is exceptionally ignorant about it. I like Michael Levins work on intelligence at scale. We are kind of ok at seeing intelligence at human scale but it tends to fall apart after that. When you get to things like non-conscious intelligence humans are pretty bad at that too. We could be (and are) encoding all kinds of behaviors in LLMs that are not at the word or token level. They are higher dimensional constructs. You won't see these things in the output of the prompt. A kind of subconscious (unstated in tokens) knowing that affects the output.
- lensecat 8d ago
- aidiscoverywire 8d ago[flagged]
- scotty79 8d ago> The intelligence illusion is in the mind of the user and not in the LLM itself. I think that might be the solution to consciousness illusion. The consciousness might be purely in the mind of someone that believes some other entity to be conscious.
- pixl97 8d agoI like to think of it this way. Consciousness is the outcome of intelligence. You can't get consciousness without it. Conversely this also means things that are not conscious can still be intelligent.
- tock 8d agoThese questions will never go anywhere because there is no good definition of intelligence or consciousness.
- scotty79 8d agoThere is a good definition of intelligence. It's what intelligence tests measure. If you define it like that, you can make harder tests, but as long as both humans and AIs can attempt to solve them we have measure and something to compare. Consciousness though might be fully illusory and meaningless as many philosophical concepts before ultimately turned out to be.
- tock 8d ago> If you define it like that :)
- scotty79 8d agoThe point is, defining it like that has merit proven in decades before AI.
- pixl97 8d agoIntelligent needs to bedefined with scales. Cells are intelligent. Organs have intelligence on top of that, that cells do not have. Plants and animals have intelligence that their organs do not. This is the real problem of intelligence, the definition one uses. It can cause them to be blind of all the things intelligence is capable of.
- slopinthebag 8d ago> There is a good definition of intelligence. It's what intelligence tests measure. what do intelligence tests measure?
- scotty79 8d agoIntelligence. It's not an exact analogy, rather something to consider ... How much momentum does a body have? You can measure it easily, just measure the velocity and the mass and multiply. What exactly did you measure? What is the meaning of this? Nobody really knows. But we know we can measure this and it's a thing and it's conserved.
- scotty79 8d ago> The intelligence illusion is in the mind of the user and not in the LLM itself. > Many AI critics, including myself, are firmly in the second camp. How can you reconcile this with the fact that AI can solve a real, intelligence bound problem for me (with zero intelligent effort on my part) that you can't? Is the solution also illusory?
- pixl97 8d agoHorseshoe theory. At each end of the shoe you tend to get loud crazies.
- emaro 8d agoInteresting to me is the tension between remarks along the lines that this post maybe made sense in 2023, but not today, or how LLMs clearly show intelligence by solving Navier-Stokes, et cetera; versus the comments I read here often as well: "it's just a tool". I can't quite put my finger on it, but aren't these two statements add odds with each other? Intelligence is hard to define, consciousness even more so, but wouldn't "intelligence" imply some sort of agency? If not, I'd argue computers were intelligent long before the age of LLMs. And likewise, doesn't a tool imply the lack of intelligence and agency, even if the tool's function is very elaborate? I got the impression that both these statements are made by the same people, or at least people with similar takes on AI. Is that wrong and there are "intelligence" and "tool" factions? Or do people disagree with my assumption and there's nothing wrong with the concept of "intelligent tools"? Kinda refreshing this discussion, compared to the builder vs. tinkerer debates, imo.
- Phemist 8d agoI would think the intelligence aspect is a bit hard to define, but to my mind (having called LLMs tools before), the main utility of a tool is reliability. Given a certain world state (including a tool's internal state), its effects back on the world state (as initiated by me) are at some leve of description understandable, expected and repeatable. Swing hammer, drive nail into wood. Make slicing motion with knife, cut meat. Type ' find /path/to/some/dir -name "keyword"', find files with keyword. Point harness at codebase with prompt 'fix bug X', actually fix bug X. All these examples are at some level of description incredibly complex (think of all particles interacting at the (sub-)atomic level even when using a hammer to only drive a nail into some wood), and of course all the electrons flowing through the GPUs doing matrix multiplications in order to fix bug X, but at some level of description (the one I just used) they are also incredibly simple and understandable. Intelligence is rather nebulous (and as used by OpenAI/Anthropic, quite threatening), but I don't think this definition of a tool precludes it to be "intelligent". They feel more orthogonal. The intelligence (or perhaps capability) feels like it is related to the size of the chunk of the world state that it can take into account and affect, while still resulting in understandable, expected and repeatable effects. LLMs, when properly harnessed, are pretty great at this currently and we are still discovering what they are consistently capable of. Calling harnessed LLMs tools is perhaps also a more grounding frame specifically to counter-act the anthropomorphizing framing that OpenAI and Anthropic consistently go for in their game of AI-doom-chicken talk. The tool framing is in that sense maybe a (self-)jedi-mind-trick.
- pohl 8d agoI can’t believe some still believe that humans are intelligent, despite there being no place where matrices are multiplied. All they have are these networks of interconnected cytoplasm-filled microtubules, which is no proper place for intelligence to live.
- bonoboTP 8d agoI don't care if it's "intelligent", I don't care if it "has a mind". I don't care if it is "really reasoning", I don't care if it "understands". I don't care if it is "sentient" or "conscious". None of this matters for the practical outcome. You'd think that this has been understood over the last 4 years, but apparently it keeps circling back to this. [Edit: I see that it was written back in 2023. Then (2023) should be added in the submission title] If it generates functional output that works, then it works. And it works. It's not a psychic's con when it outputs Lean-verified proofs. It isn't a con when it can find and exploit zero-days. The OP is still in the "denial" phase. Most I see are already in "anger" (a blurry fury against everything AI-shaped, from vague reasons piling on all "bad stuff" political reasons they already hated before) or "bargaining" (mathematicians scrambling to come up with a new definition of their job and retcon that it was always the main part anyway). A few are already in "depression" and feel like spectators on the Titanic, and the tiniest sliver is at "acceptance" with some kind of well-informed plan for their future.
- zkry 8d ago> The OP is still in the "denial" phase. This was written in July 2023. ChatGPT was released November 2022. No matter your views on AI, surely you can't blame the OP for writing this after a few months ChatGPT was released.
- bonoboTP 8d agoApologies. The title should be amended with (2023). In this case it is an interesting snapshot of the zeitgeist back then and we can see how well it panned out and whether anyone involved has updated on new info.
- woah 8d agoFor some reason the anti-AI camp ping pongs between different and often mutually incompatible arguments at lightning speed. The "AI is fake" argument has been completely forgotten at this point.
- NinjaTrance 8d agoLLMs may not think, but they do reason. I say that because the word "reason" originates from the Latin word "ratio," which means "calculation". LLMs do calculations to produce their answers -- thus, they reason.
- pixl97 8d agoHonestly the word think is so poorly defined it cannot be used with any scientific rigor and is rather useless without a dissertation being posed with it on what you actually mean by that. Intelligence really needs one too.
- NinjaTrance 8d agoArticle was posted on July 4th, 2023.
- draftpunked 8d agoI don't think it's a "con," but I find that using the mental model of LLMs being sophisticated, lossy search engines of knowledge can help us separate some of the factors more cleanly than imagining that they have cognition or intelligence. It's hard for me to imagine a stateless operation as intelligence per se, though perhaps the chaining of such operations starts looking more like it?
- mindcrime 8d agoIt's hard for me to imagine a stateless operation as intelligence per se That's an interesting point. My take would be to say that we shouldn't think of the AI as being just the model, but should include the harness. At that level, clearly we can keep state / context and that is probably a more natural mapping to our intuitive understanding of "intelligence".
- kyberlex 8d ago[flagged]
- Method5440 8d agoI don’t understand why people jump so readily to seeing intelligence here. Science fiction has as a core, central trope that humans will debase and devalue other types of life they do not understand. Even the storied Commander Data has to fight for the right to self-determination (probably the best episode of STTNG by the way - ‘The Measure of a Man’). We were so worried that we’d undervalue intelligence when apparently our knee-jerk response is to overvalue it. Perhaps this has changed over time and we’re now primed by science fiction and instincts towards social justice, but I worry that we’re really just undervaluing ourselves. The one thing this 2023 article gets partially correct imo is that any intelligence we see in AI (as of 2026) is our own - not that it’s a mirror but that the intelligence comes from the way that the words are put together, which comes from written human language created by (allegedly) intelligent creatures put in as input in both the training and prompt, among other places. Rearranging and repeating the words, even in context, does not intelligence make. I’m not even convinced that you’re intelligent, dear reader.
- garciasn 8d agoBecause to the general public, LLMs are an example of Clarke's Third Law. Most folks, who are not remotely close to even a basic understanding of how LLMs operate at a technical level and only view their output cannot possibly evaluate what they're experiencing other than to believe it's conscious, alive, and/or magic. Most people on Earth try to put what they're seeing into the context of what they understand; mental gymnastics to try and understand what is happening based on their prior experience. They have absolutely 0 understanding of how it works under the hood so, to them, it must be alive.
- natbennett 8d ago> Delegating your decision-making, ranking, assessment, strategising, analysis, or any other form of reasoning to a chatbot becomes the functional equivalent to phoning a psychic for advice. Lots of comments talking about how recent accomplishments disprove the article but I think this bit holds up pretty well. LLMs are very good at tricking people into thinking they have capabilities that they don’t.
- kraf 8d agoI keep being confused about how people's understanding of the models get stuck at next token prediction. Isn't this entirely neglecting the RL training? I might be misunderstanding something but to my mind it makes the issue way fuzzier than it's being painted here.
- mindcrime 8d agoI keep being confused about how people's understanding of the models get stuck at next token prediction. Heh. A lot of anti-ai hucksters I see posting on LinkedIn just LOVE to use the phrase "next token prediction" and the word "autoregressive". They've almost become shibboleths that identify members of that camp. That and the classic rallying cry of "Linear Algebra isn't intelligent!" The best take I've seen on that recently, was somebody who made the point "just think of the next token prediction part as the output layer". Which makes perfect sense.. if you're replying in natural language, at some point in the flow, you have to construct a sentence and starting at the head and predicting next tokens is perfectly reasonable. I'm doing it literally as I'm typing these characters, for crying out loud! But the mistake is to think that LLM's only "predict next tokens" with no consideration of the possibility that they are actually constructing richer representations, building concepts, making analogies, doing abduction, induction, etc. My own (admittedly anecdotal) take on working with LLM's suggests to me that they do do those things, albeit probably not the same way humans do. I think a lot of folks are missing the point by being overly reductive when they start talking about "next token prediction" and "autoregressive". It's like, can we say "Phil (me) isn't intelligent because there's nothing going on but some electrical impulses and chemistry happening inside his brain. Everybody knows electricity and chemistry aren't intelligent!"
- mindcrime 8d ago(EDIT: I, like some others, just noticed this is an older post from 2023. That changes nothing in particular in my response here, but I would be curious to know if the OP's views have shifted at all in light of subsequent developments). I feel like this post completely misses the point, pretty much across the board. And it does so by repeating the same mistake that everybody keeps making - conflating mechanism and function. One of the issues in during this research—one that has perplexed me—has been that many people are convinced that language models, or specifically chat-based language models, are intelligent. That's because they are intelligent. But there isn’t any mechanism inherent in large language models (LLMs) that would seem to enable this and, The mechanism is irrelevant to the issue of whether they are intelligent or not. Airplanes fly, despite not flapping their wings. The sign on the marquee says artificial intelligence. LLMs are not brains and do not meaningfully share any of the mechanisms that animals or people use to reason or think. Again, irrelevant. Nobody claims that they are brains, and it doesn't matter what mechanism they use. The sign on the marquee says artificial intelligence. LLMs are a mathematical model of language tokens. You give a LLM text, and it will give you a mathematically plausible response to that text. That's a bit overly reductionistic. And to the earlier point and, if real, it would be completely unexplained. I'd probably leave out the word "completely" there, but it is fair to say that not everything about the underlying mechanism is understood. But at the risk of repeating myself, that's orthogonal to the question of whether or not they are intelligent. There is no reason to believe that it thinks or reasons—indeed, every AI researcher and vendor to date has repeatedly emphasised that these models don’t think. You mean "There is no reason to believe that it thinks or reasons like a human". Again, this is irrelevant to the question of whether or not they are intelligent. The sign on the marquee says artificial intelligence. I don't know why people keep obsessing over mechanism in this discussion. It something functions as an intelligence, it is intelligent as far as I'm concerned - at least when the framing is a discussion of artificial intelligence.
- mindcrime 8d agoAnd just to expand on why I reject the line of thought laid out in this article, let me share something from my own life as an illustrative example. I'm working on a project, with a lot of help from ChatGPT, involving an "artificial neuron". That is, an electronic circuit, using a PUT, a capacitor, and some resistors, that simulates some of the behavior of a biological neuron. Specifically an "integrate and fire" model of neuron behavior. To that end, I'm running experiments by scripting my function generator to send signals to the circuit, and then capturing the inputs and outputs on my oscilloscope. Then I usually discuss the results with ChatGPT. In what follows, observe a couple of things: 1. The LLM "knows" the context of what we're talking about, even if I provide a prompt with no text at all, just an image. 2. It parses a moderately complex image, identifies the separate traces and what they represent, uses the time-base information displayed on screen, and the on-screen graticule, and works out "how many input pulses fire before an output pulse fires" and then reports back to me and gives an analysis of how that relates to our previous observations and gives suggestions for the next experiment to run. Human intelligence? No. But I see no world where behavior like that does not count as "intelligent" regardless of the mechanism behind it. And that's probably not even the best example I could come up with, it's just something that was "top of mind" and for which I had the necessary images and what-not already ready, or easy to capture. https://www.fogbeam.com/images/neuron_zero0.png https://www.fogbeam.com/images/neuron_zero0.png https://www.fogbeam.com/images/neuron_zero1.png https://www.fogbeam.com/images/neuron_zero1.png https://www.fogbeam.com/images/neuron_zero2.png https://www.fogbeam.com/images/neuron_zero2.png https://www.fogbeam.com/images/neuron_zero3.png https://www.fogbeam.com/images/neuron_zero3.png https://www.fogbeam.com/images/neuron_zero4.png https://www.fogbeam.com/images/neuron_zero4.png
- deleted 8d ago[deleted]
- tomhow 8d agoThe LLMentalist Effect (2023) - https://news.ycombinator.com/item?id=42983571 https://news.ycombinator.com/item?id=42983571 - Feb 2025 (133 comments) Chat-based Large Language Models replicate the mechanisms of a psychic’s con - https://news.ycombinator.com/item?id=36586540 https://news.ycombinator.com/item?id=36586540 - July 2023 (13 comments)
- cagz 8d agoThe article starts interestingly; I like the analogy between cold reading and LLM responses (well, the ones we were getting three years ago). Then it goes on to generalise to the tune of "This new era of tech seems to be built on superstition and pseudoscience" which I think was a conclusion better left to the reader. We are used to articles, blogs that stayed relevant for years. It is hardly the case with AI, the speed of change is so high that an observation can become stale in a year. This makes it very important for articles to state that the ideas are based on the technology in year X, rather than generalising them as universal truths.
- PowerElectronix 7d agoI like how back in 2023 we had "no AI scientist believes this to be intelligence" and in 2026 we have cults in Anthropic praying to the LLM.
- deleted 7d ago[deleted]
- YcYc10 7d agoThis didn't age very well.
- signa11 7d agocan there be any intelligence without any curiosity ?
- SuperV1234 7d ago> 2023 > 200 upvotes typical hn
- Good4boothee 7d agoIt has more comments than points, IFAIK that causes flamewar suppression to kick in and take it off the frontpage. It would be great for submission to include year in title when it matters but I doubt it can be reliably automated.
- rrook 7d ago> The tech industry has accidentally invented the initial stages a completely new kind of mind, based on completely unknown principles, using completely unknown processes that have no parallel in the biological world. This is just false? Language emerged from biology, we've brute forced some version of that model. Knowing that this is a new kind of mind requires that we are able to differentiate these different kinds of minds in the first place.
- m0llusk 7d agoGenerating and mastering language are two very different things. Even the best LLMs still have no ability to understand or work with abstractions.
- rrook 7d agoTotally agree with the first part of that. The second less so. LLMs are able to maintain a cohesive idea across multiple metaphors, which is basically the same as walking abstractions.
- deleted 7d ago[deleted]
- JeanCampos 7d agoI would understand to read this kind of articles in 2023ish, but for what they can do now... It is not even if they are "intelligent" is what intelligence even is. I think it is undeniable that there is a level of capacity increasing with each new model, and how would you call that? There is an illusion in LLM's, but that illusion, in my point of view, is about how much of an "entity" each chat bot is, not if they are capable of mimic intelligence or not.
- roncesvalles 7d agoLLMs are not intelligent for the simple reason that if you had as much knowledge as an LLM has, you would be churning out deductions and new discoveries every second. You'd be putting out Nobel-prize-worthy research every month. LLMs are like a 50 IQ person with no intrinsic motivation but unbelievably vast knowledge and near-perfect recall.
- vanuatu 7d agoI don't think intelligence needs to take the shape of human-like intelligence, nor human-like data efficiency. I'd argue they are intelligent entities, superhuman in some ways due to being run on silicon but extremely data inefficient with nonobvious failure modes.
- deleted 7d ago[deleted]
- roncesvalles 7d ago>I don't think intelligence needs to take the shape of human-like intelligence But then the word doesn't mean anything. Intelligence is almost by definition what we're like. If you think LLMs are intelligent, you have to concede that databases are also intelligent. And if you can't then you need to ponder deeply about what quality of an LLM's response to your query makes it (seem) more "intelligent" than a database's response to your SQL query, beyond the trivial thing that it's in natural language.
- nikitamalyavin 7d agoThat also sounds like (populist) politics... And basically any marketing/PR.
- mr_big_bowls 7d ago[dead]