9 ms·

Thinking Fast, Slow, and Artificial: How AI Is Reshaping Human Reasoning

199 points by Anon84 7mo ago

ashwinnair99 7mo ago
[flagged]
- pepperoni_pizza 7mo ago
  I already noticed that. When I feel lazy, I feel like reaching for the AI. Exactly the same laziness voice that nudges me to drive instead of walking. But then I go running and swimming for fun, and there is no laziness voice there, telling me to stop, because I enjoy it. And similarly with AI, I only use it for things where I don't care about, like various corporate bs. Maybe the cure for AI-brain is to care about and be passionate about things. Conversely, does this mean that the kind of people who use AI for everything don't care about anything?
  - delijati 7mo ago
    That is why i compare it to fast-food. From time to time you enjoy it but you should not consume it too much ;)
  - necrotic_comp 7mo ago
    There's something interesting I've found about my interactions with the AI - I use it as a thought-partner. I don't ask it to solve a problem for me (well, first at least!) I think about it as a tool to work with, engage with the problem, and spit out a result that I then test and review. I see it as part of the feedback loop, and it speeds up some of the mechanical drudgery, while not removing any of the semantic problems inherent in problem solving. In other words, there's things machines are good at, and things humans are good at - if we each stick to our strengths, we can move incredibly fast.
  - throaway197512 7mo ago
    I've been using Claude to vibe code my game ideas for the past months (iterated with docs). I find when I think of it as a being named "Claude," like a juniour partner who's there to eagerly help me, I get lazy. I think of it as if it's a real almost slave-like creature, who's there to make everything for me without any regards to himself. But, when I think of it as a tool, as if its a hammer or something, I feel much less lazy. I think of it as "building something" using a program, not telling "Claude" what to do and expecting it to happen. I even turn off Claude's verbal responses completely sometimes to help this. 100% impersonal.
- n_u 7mo ago
  Are you a LLM? This comment is written twice in this thread and of your last 10 comments, 6 use the pattern "X isn't Y" or "X didn't Y, Z did" https://news.ycombinator.com/item?id=47469767 https://news.ycombinator.com/item?id=47469767 > The concern isn't that AI reasons differently. https://news.ycombinator.com/item?id=47469834 https://news.ycombinator.com/item?id=47469834 > The concern isn't that AI reasons differently. https://news.ycombinator.com/item?id=47470111 https://news.ycombinator.com/item?id=47470111 > The problem isn't time. https://news.ycombinator.com/item?id=47469760 https://news.ycombinator.com/item?id=47469760 > Airlines have been quietly expanding what they can remove you for. This isn't really about headphones. https://news.ycombinator.com/item?id=47469448 https://news.ycombinator.com/item?id=47469448 > Good tech losing isn't new, it's just always a bit sad when it happens slowly https://news.ycombinator.com/item?id=47469437 https://news.ycombinator.com/item?id=47469437 > The tool didn't fail here, the person did
  - christophilus 7mo ago
    Definitely AI. Every comment sounds like GPT.
  - eslaught 7mo ago
    Please don't take up space in the comment section with accusations. You can report this at the email below and the mods will look at it: > Please don't post insinuations about astroturfing, shilling, brigading, foreign agents, and the like. It degrades discussion and is usually mistaken. If you're worried about abuse, email hn@ycombinator.com and we'll look at the data. > https://news.ycombinator.com/newsguidelines.html https://news.ycombinator.com/newsguidelines.html
    - dgacmu 7mo ago
      I find it kind of helpful and interesting to see a subset of these called out with a bit of data. Helps keep my LLM detector trained (the one in my brain, that is) and I think it helps a little about expressing the community consensus against this crap. In this case, I'm glad the GP posted something, as it's definitely not mistaken.
- keiferski 7mo ago
  ”Which is why the Matrix was redesigned to this, the peak of your civilization. I say your civilization because as soon as we started thinking for you it really became our civilization which is of course what this is all about.“
ashwinnair99 7mo ago
[flagged]
gmuslera 7mo ago
The main problem with "System 3" is that it have its own kind of "cognitive biases", like System 1, but those new cognitive biases are designed by marketing, politics, culture and whatever censor or makes visible the original training. Even if the process, the processing and whatever else around was perfect (that is not, i.e. hallucinations) But, we still have the System 1, and survived and reached this stage because of it, because even a bad guess is better than the slowness of doing things right. It have its problems, but sometimes you must reach a compromise.
- HPsquared 7mo ago
  I suppose the publishing process has always existed as system 3. It's just that now we have a new way to read and write with an abstract "rest of the world".
kikkupico 7mo ago
Contrary to the general opinion, I feel that AI has IMPROVED my cognitive skills. I find myself discovering solutions to problems I've always struggled with (without asking AI about it, of course). I also find myself becoming much better at thinking on my feet during regular conversations. I believe I'm spending more time deep thinking than ever before because I can leave the boring cognitive stuff to AI, and that's giving my mind tougher workouts and making it stronger; but I could be completely wrong.
- siva7 7mo ago
  It's so fascinating, i feel the same but at the same i feel like most people get dumber than before ai (and most seem to struggle adapting ai)
  - deleted 7mo ago
    [deleted]
  - mayukh 7mo ago
    Because most people either don't know how to use it (multiple reasons, that ai itself can help them solve) or don't have the right mindset going into it (deeper work needed)
- eslaught 7mo ago
  Without an empirical methodology it's hard to know how true this is. There are known and well-documented human biases (e.g., placebo effect) that could easily be involved here. And besides that, there's a convincing (but often overlooked on HN) argument to be made that modern LLMs are optimized in the same manner as other attention economy technologies. That is to say, they're addictive in the same general way that the YouTube/TikTok/Facebook/etc. feed algorithms are. They may be useful, but they also manipulate your attention, and it's difficult to disentangle those when the person evaluating the claims is the same person (potentially) being manipulated. I'd love to see an empirical study that actually dives into this and attempts to show one way or another how true it is. Otherwise it's just all anecdotes.
  - 7mo ago
Ozzie_osman 7mo ago
When humans have an easy way to do something that is almost as good, we choose that easy way. Call it laziness, energy conservation, coddling, etc. The hard thing then becomes hard to do even when the easy thing isn't available, because the cognitive muscle and the discipline atrophy. Like kids who are never taught to do things for themselves.
- tac19 7mo ago
  Do you refuse to use a calculator or spreadsheet, because doing long hand division helps you exercise your mental muscle? Do you refuse to use a database, because it will make your memory weaker? Or, do you refuse to use a car, because it makes you less able to walk when the car is unavailable? No. Because the car empowers you to do something that, at the very least, takes a lot longer on foot. People have worried with every single new technology that it will enfeeble the masses, rather than empower them, and yet in the end, we usually find ourselves better off.
  - bluefirebrand 7mo ago
    > Do you refuse to use a calculator or spreadsheet, because doing long hand division helps you exercise your mental muscle Yeah when I was learning in school we weren't allowed electronics for division, and I think I absolutely would be dumber if I had never done that > People have worried with every single new technology that it will enfeeble the masses, rather than empower them, and yet in the end, we usually find ourselves better off. If you're posting this from America, you're living in a society that is fatter than ever thanks to cars. So there's surely some nuance here, not every technology upgrade is strictly better with no downsides
  - wongarsu 7mo ago
    The car seems like a great example of a technology with a lot of problematic side effects. Places that had a more measured adoption ended up a lot better than those that replaced all public transit with cars and routinely demolished neighborhoods to make space for bigger highways Cars are an essential part of modern life, but the sweetspot for car adoption isn't on either of the extremes
HarHarVeryFunny 7mo ago
> Across studies, participants with higher trust in AI and lower need for cognition and fluid intelligence showed greater surrender to System 3 So the smart get smarter and the dumb get dumber? Well, not exactly, but at least for now with AI "highly jagged", and unreliable, it pays to know enough to NOT trust it, and indeed be mentally capable enough that you don't need to surrender to it, and can spot the failures. I think the potential problems come later, when AI is more capable/reliable, and even the intelligentsia perhaps stop questioning it's output, and stop exercising/developing their own reasoning skills. Maybe AI accelerates us towards some version of "Idiocracy" where human intelligence is even less relevant to evolutionary success (i.e. having/supporting lots of kids) than it is today, and gets bred out of the human species? Maybe this is the inevitable trajectory: species gets smarter when they develop language and tool creation, then peak, and get dumber after having created tools that do the thinking for them? Pre-AI, a long time ago, I used to think/joke we might go in the other direction - evolve into a pulsating brain, eyes, genitalia and vestigial limbs, as mental work took over from physical, but maybe I got that reversed!
- RodgerTheGreat 7mo ago
  I think everyone who believes that they can personally resist the detrimental psychological effects of exposure to LLMs by "remaining aware" or "being careful", because they have cultivated an understanding of how language models work, is falling into precisely the same fallacy as people who think they can't be conned or that marketing doesn't work on them. Don't kid yourself. If you use this junk, it's making you dumber and damaging your critical thinking skills, full-stop. This is delegation of core competency. You may feel smarter, or that you're learning faster, of that you're more productive, but to people who aren't addicted to LLMs it sounds exactly like gamblers insisting they have a foolproof system for slots, or alcoholics insisting that a few beers make them a better driver. Nobody outside the bubble is impressed with the results.
  - thesumofall 7mo ago
    I fully agree that it’s close to impossible to not eventually fall into the trap of overrelying on them. However, it’s also true that I was able to do things with them that I would never have done otherwise for a lack of time or skill (all sorts of small personal apps, tools, and scripts for my hobbies). Maybe it’s a bit similar to only reading the comment section in a newspaper instead of the news? They will introduce you to new perspectives but if you stop reading the underlying news you’ll harm your own critical thinking? So it’s maybe a bit more grey than black & white?
andai 7mo ago
Damn. I came up with a hypothetical "System 3" last year! I didn't find AI very helpful in that regard though. Current status: partially solved. Problem: System 2 is supposed to be rational, but I found this to be far from the case. Massive unnecessary suffering. Solution (WIP): Ask: What is the goal? What are my assumptions? Is there anything I am missing? -- So, I repeatedly found myself getting into lots of trouble due to unquestioned assumptions. System 2 is supposed to be rational, but I found this to be far from the case. So I tried inventing an "actually rational system" that I could "operate manually", or with a little help. I called it System 3, a system where you use a Thinking Tool to help you think more effectively. Initial attempt was a "rational LLM prompt", but these mostly devolve into unhelpful nitpicking. (Maybe it's solvable, but I didn't get very far.) Then I realized, wouldn't you get better results with a bunch of questions on pen and paper? Guided writing exercises? So here are my attempts so far: reflect.py - https://gist.github.com/a-n-d-a-i/d54bc03b0ceeb06b4cd61ed173512f1f#file-reflect-py https://gist.github.com/a-n-d-a-i/d54bc03b0ceeb06b4cd61ed173... unstuck.py - https://gist.github.com/a-n-d-a-i/d54bc03b0ceeb06b4cd61ed173512f1f#file-unstuck-py https://gist.github.com/a-n-d-a-i/d54bc03b0ceeb06b4cd61ed173... -- I'm not sure what's a good way to get yourself "out of a rut" in terms of thinking about a problem. It seems like the longer you've thought about it, the less likely you are to explore beyond the confines of the "known" (i.e. your probably dodgy/incomplete assumptions). I haven't solved System 3 yet, but a few months later found myself in an even more harrowing situation which could have been avoided if I had a System 3. The solution turned out to be trivial, but I missed it for weeks... In this case, I had incorrectly named the project, and thus doomed it to limbo. Turns out naming things is just as important in real life as it is in programming! So I joked "if being pedantic didn't solve the problem, you weren't being pedantic enough." But it's not a joke! It's about clear thinking. (The negative aspect of pedantry is inappropriate communication. But the positive aspect is "seeing the situation clearly", which is obviously the part you want to keep!)
nasretdinov 7mo ago
I mean... I don't really check calculations made by a computer (e.g. by my own programs) all that often either and I think I'm completely fine :). But I guess the difference is that we kind of know how computers work and that they're generally super accurate and make mistakes incredibly rarely. The "AI" (although I disagree with "I" part) is wrong incredibly often, and I don't think people appreciate that the difference to the "traditional" approach isn't just significant, it's astronomical: LLMs make things up at least 5% of the time, whereas CPUs male mistakes maybe (10^-12)% of time or less. It's 12 orders of magnitude or so.
- danielbln 7mo ago
  How often do humans make mistakes? That's the better comparison.
thr0waway001 7mo ago
AI reminds of listening to any person who seems like an intellectual authority on multiple subjects on YouTube and is not afraid to wax confidently on any topic. They seem very intelligent and knowledgable until they actually talk about something you know. In other words, I try to learn from it whenever it does something I can't do but when it does something I can do or something I'm really good at it I find myself wanting to correct it cause it doesn't do it that well. It just seems like a really quick thinking and fast executing but, ultimately, mid skilled / novice person.
- ahd94 7mo ago
  And it starts showing impatience when its about to run out of context, more like someone who wants to get out of the office exactly at 5.
  - kykat 7mo ago
    Not just when running out of context, it's always. Once it fixates on a goal, all hell breaks loose and there's nothing that it won't be sacrificed to get there. At least that's my experience with Claude Code, I am pressing the figurative breaks all the time.
    - bitwize 7mo ago
      "I'm Mr. Meeseeks, look at meeee!"
      - thr0waway001 7mo ago
        Claude, I need you to help me take two strokes off my golf game.
  - xg15 7mo ago
    It was sort of funny when codex switched mid-session from "patch complete, I'll now automatically run the tests and verify results" to "patch complete, if you want to run the tests, just paste this instruction into a terminal somewhere: ..." Apparently this was caused by the context window getting full! (At least I assume that because it went back to the old behavior after I triggered a compaction)
johnnymonster 7mo ago
blocking access to a site because you don't enable javascript is diabolical
bjourne 7mo ago
"Time pressure (Study 2) and per-item incentives and feedback (Study 3) shifted baseline performance but did not eliminate this pattern: when accurate, AI buffered time-pressure costs and amplified incentive gains; when faulty, it consistently reduced accuracy regardless of situational moderators." I LOLed.
bobokaytop 7mo ago
[dead]
danilor 7mo ago
I couldn't figure if this was published to a journal? Or is it only published to a pre-print server?
- tastroder 7mo ago
  SSRN is a preprint server and that is the only published version.
deevelton 7mo ago
Have been curious what it could look like (and whether it might be an interesting new type of “post” people make) if readers could see the human prompts and pivots and steering of the LLM inline within the final polished AI output.
woopsn 7mo ago
In the technophile's future people aren't just getting dumber, not wanting to think or forgetting how - they aren't allowed to think. Maybe about anything. It's too big liability, costs too much to support, moreover detracts from the product. Like Sam A telling those Indian students they aren't worth the energy and water. That's what we're dealing with.
- windowliker 7mo ago
  That's reminiscent of Kurt Vonnegut's vision of the future in Harrison Bergeron.
vicchenai 7mo ago
[dead]
- memolife23 7mo ago
  [dead]
pink_eye 7mo ago
Can it design and implement a plutonium electric fuel cell with a 24,000 year half life? We have yet to witness it. Can it automate Farming and Agriculture? These are the real questions. #Born-Crusty
andrewssobral 7mo ago
[dead]
meander_water 7mo ago
I'm conflicted about this. As I was reading the paper, my AI detector senses were tingling all over the place. Large parts of the paper score very high probability of being written entirely by AI in gptzero. I'm not sure if I could trust anything written in it.
yubainu 7mo ago
[flagged]
psybrg-prtcls 7mo ago
Anyone else get the distinct impression that parts of this paper were written by AI?
- nemo44x 7mo ago
  Why shouldn’t it be? You can gather and form your thoughts, create a draft, and then have a LLM rewrite it for you. You can write in the style you prefer so you can focus on thoughts and then have the LLM rewrite it in the appropriate style for the audience.
  - gwern 7mo ago
    One might worry that it would increase the authors' confidence even following their LLM rewrite errors and reduce accuracy overall regardless of moderators.
    - nemo44x 7mo ago
      You still need to review and edit.
      - gwern 7mo ago
        I was being sarcastic, because that point obviously also applies to the subjects in the experiment as well.
Null-Set 7mo ago
The original reseaech around thinking fast and slow (aka system 1 system 2 thinking) failed to be replicated when researchers tried.
seu 7mo ago
There's a very interesting critique of Kahneman's "Thinking fast and slow" from German psychologist Gerd Gigerenzen: https://www.researchgate.net/publication/397923694_The_Legacy_of_Daniel_Kahneman_A_Personal_View https://www.researchgate.net/publication/397923694_The_Legac... I suggest everyone interested in learning how these theories emerge, and how the social sciences work, to give it a read. Also, it kind of dismantles the whole idea of System 1 and 2, which then I guess would question the theoretical foundations of this paper too.
- fhi 7mo ago
  I've read the work, Gerd Gigerenzen criticises the approaches to scientific experiments conducted by Daniel Kahneman and Amos Tversky, and the lack of desire, to put it mildly, to properly define the terms they were introducing. Critique to System 1 and 2 is based mostly on using System 1 and 2 to excuse the alleged deficiencies in the experiments. I think the original article in this discussion is using "Systems 1 and 2" as intuitive and rational modes for problem-solving and interestingly enough, Gerd Gigerenzen also has a reference in this work "accuracy-effort trade-off (Payne, Bettman, and Johnson 1993): The less effort one takes, the less accurate one will be." which aligns with the broader idea of Systems 1 and 2.
alexchengyuli 7mo ago
The paper puts AI next to System 1 and 2, but those are ways you think. With AI the thinking still happens, you just can't see or control it anymore. When you googled something and got five contradictory results, that told you the question was hard. A clean AI answer doesn't give you that signal. Coherence looks the same whether the answer is right or wrong. The failure mode didn't get worse. It got quieter.
bsenftner 7mo ago
A major problem with LLM AIs is their core nature is not understood by the vast majority of everyone - developers included. They are an embodiment of literature, and if that confuses you you're probably operating on an incorrect definition of them. I like to think of them as idiot savants with exponential more savant than your typical fictional idiot savant. They pivot on every word you use, each word in your series activating areas of training knowledge, until your prompt completes and then the LLM is logically located at some biased perspective of the topic you seek (if your wording was not vague and using implied references). Few seem to realize there is no "one topic" for each topic an LLM knows, there are numerous perspectives on every topic. Those perspectives reflect the reason one person/group is using that topic, and their technical seriousness within that topic. How you word your prompts dictates which of these perspectives your ultimate answer is generated. When people say their use of AI reflects a mid level understanding of whatever they prompted, that is because the prompt is worded with the language used by "mid level understanding persons". If you want the LLM to respond with expert guidance, you have to prompt it using the same language and terms that the expert you want would use. That is how you activate their area of training to generate a response from them. This goes further when using coding AI. If your code has the coding structure of a mid level developer, that causes a strong preference for mid level developer guidance - because that is relevant to your code structure. It requires a well written prompt using PhD/Professorial terminology in computer science to operate with a mid level code base and then get advice that would improve that code above it's mid level architecture.
- ben_w 7mo ago
  In two words "book smart". In more words, "of course it's stupid, it's as complex as a mid-sized rodent where we taught it purely by selective breeding on getting answers right while carefully preventing any mutations which made their brains any bigger".
  - cucumber3732842 7mo ago
    It's basically plotting the dots of all easily accessible written word to find your words, finding the words that are answers to your words and then charting a line through them no matter how scattered those points may be, and spitting that back out. It doesn't "know" anything nor is it reasoning even if the results are similar. You have to come into it with the same "these people are only stupid and lack the experience to answer my questions despite thinking they do, they lack the world view to even process how I arrived at the parameters of my question(s)" apprehension like you would if asking reddit about some hazardous thing that would make them all screech. AI is the margarine to that butter. It's a technology with potential to deliver great value, but there are limitations...
BANRONFANTHE 7mo ago
[dead]
tim333 7mo ago
I'm not sure if this is saying people were given a task and the option to consult an AI. When they did they were influenced by its answer. Which is kind of duh? Of course. They have some cool language like calling the AI system 3 and calling taking advice 'cognitive surrender' but I'm not sure how this differs from asking your mate Bob and taking his advice?
pugchat 7mo ago
[dead]
gregfrank 7mo ago
This framing points at something important that I think the alignment evaluation literature often misses: the distinction between what a model represents internally and what it does behaviorally. Probing can tell you what's in the representations, and linear probes can be surprisingly accurate. But in experiments I've run on DeepSeek and Qwen models, high probe accuracy for a given behavior doesn't predict whether the model actually routes through that behavior at inference time. The detection layer and the routing layer are architecturally separable, and most evaluation benchmarks are measuring the former while claiming to measure the latter.
michelejoseph 7mo ago
contradish catches when 2 users get different answers to the same question
iydr 7mo ago
This paper is irrelevant: "AI assistant (ChatGPT; GPT‑4o)" (I found no others, including 0 "claude") It's also probably bad about something else important.

199 points by Anon84 7mo ago

ashwinnair99 7mo ago

[flagged]

pepperoni_pizza 7mo ago
I already noticed that. When I feel lazy, I feel like reaching for the AI. Exactly the same laziness voice that nudges me to drive instead of walking. But then I go running and swimming for fun, and there is no laziness voice there, telling me to stop, because I enjoy it. And similarly with AI, I only use it for things where I don't care about, like various corporate bs. Maybe the cure for AI-brain is to care about and be passionate about things. Conversely, does this mean that the kind of people who use AI for everything don't care about anything?
- delijati 7mo ago
  That is why i compare it to fast-food. From time to time you enjoy it but you should not consume it too much ;)
- necrotic_comp 7mo ago
  There's something interesting I've found about my interactions with the AI - I use it as a thought-partner. I don't ask it to solve a problem for me (well, first at least!) I think about it as a tool to work with, engage with the problem, and spit out a result that I then test and review. I see it as part of the feedback loop, and it speeds up some of the mechanical drudgery, while not removing any of the semantic problems inherent in problem solving. In other words, there's things machines are good at, and things humans are good at - if we each stick to our strengths, we can move incredibly fast.
- throaway197512 7mo ago
  I've been using Claude to vibe code my game ideas for the past months (iterated with docs). I find when I think of it as a being named "Claude," like a juniour partner who's there to eagerly help me, I get lazy. I think of it as if it's a real almost slave-like creature, who's there to make everything for me without any regards to himself. But, when I think of it as a tool, as if its a hammer or something, I feel much less lazy. I think of it as "building something" using a program, not telling "Claude" what to do and expecting it to happen. I even turn off Claude's verbal responses completely sometimes to help this. 100% impersonal.
n_u 7mo ago
Are you a LLM? This comment is written twice in this thread and of your last 10 comments, 6 use the pattern "X isn't Y" or "X didn't Y, Z did" https://news.ycombinator.com/item?id=47469767 https://news.ycombinator.com/item?id=47469767 > The concern isn't that AI reasons differently. https://news.ycombinator.com/item?id=47469834 https://news.ycombinator.com/item?id=47469834 > The concern isn't that AI reasons differently. https://news.ycombinator.com/item?id=47470111 https://news.ycombinator.com/item?id=47470111 > The problem isn't time. https://news.ycombinator.com/item?id=47469760 https://news.ycombinator.com/item?id=47469760 > Airlines have been quietly expanding what they can remove you for. This isn't really about headphones. https://news.ycombinator.com/item?id=47469448 https://news.ycombinator.com/item?id=47469448 > Good tech losing isn't new, it's just always a bit sad when it happens slowly https://news.ycombinator.com/item?id=47469437 https://news.ycombinator.com/item?id=47469437 > The tool didn't fail here, the person did
- christophilus 7mo ago
  Definitely AI. Every comment sounds like GPT.
- eslaught 7mo ago
  Please don't take up space in the comment section with accusations. You can report this at the email below and the mods will look at it: > Please don't post insinuations about astroturfing, shilling, brigading, foreign agents, and the like. It degrades discussion and is usually mistaken. If you're worried about abuse, email hn@ycombinator.com and we'll look at the data. > https://news.ycombinator.com/newsguidelines.html https://news.ycombinator.com/newsguidelines.html
  - dgacmu 7mo ago
    I find it kind of helpful and interesting to see a subset of these called out with a bit of data. Helps keep my LLM detector trained (the one in my brain, that is) and I think it helps a little about expressing the community consensus against this crap. In this case, I'm glad the GP posted something, as it's definitely not mistaken.
keiferski 7mo ago
”Which is why the Matrix was redesigned to this, the peak of your civilization. I say your civilization because as soon as we started thinking for you it really became our civilization which is of course what this is all about.“

ashwinnair99 7mo ago

[flagged]

gmuslera 7mo ago

The main problem with "System 3" is that it have its own kind of "cognitive biases", like System 1, but those new cognitive biases are designed by marketing, politics, culture and whatever censor or makes visible the original training. Even if the process, the processing and whatever else around was perfect (that is not, i.e. hallucinations) But, we still have the System 1, and survived and reached this stage because of it, because even a bad guess is better than the slowness of doing things right. It have its problems, but sometimes you must reach a compromise.

HPsquared 7mo ago
I suppose the publishing process has always existed as system 3. It's just that now we have a new way to read and write with an abstract "rest of the world".

kikkupico 7mo ago

Contrary to the general opinion, I feel that AI has IMPROVED my cognitive skills. I find myself discovering solutions to problems I've always struggled with (without asking AI about it, of course). I also find myself becoming much better at thinking on my feet during regular conversations. I believe I'm spending more time deep thinking than ever before because I can leave the boring cognitive stuff to AI, and that's giving my mind tougher workouts and making it stronger; but I could be completely wrong.

siva7 7mo ago
It's so fascinating, i feel the same but at the same i feel like most people get dumber than before ai (and most seem to struggle adapting ai)
- deleted 7mo ago
  [deleted]
- mayukh 7mo ago
  Because most people either don't know how to use it (multiple reasons, that ai itself can help them solve) or don't have the right mindset going into it (deeper work needed)
eslaught 7mo ago
Without an empirical methodology it's hard to know how true this is. There are known and well-documented human biases (e.g., placebo effect) that could easily be involved here. And besides that, there's a convincing (but often overlooked on HN) argument to be made that modern LLMs are optimized in the same manner as other attention economy technologies. That is to say, they're addictive in the same general way that the YouTube/TikTok/Facebook/etc. feed algorithms are. They may be useful, but they also manipulate your attention, and it's difficult to disentangle those when the person evaluating the claims is the same person (potentially) being manipulated. I'd love to see an empirical study that actually dives into this and attempts to show one way or another how true it is. Otherwise it's just all anecdotes.
- 7mo ago

Ozzie_osman 7mo ago

When humans have an easy way to do something that is almost as good, we choose that easy way. Call it laziness, energy conservation, coddling, etc. The hard thing then becomes hard to do even when the easy thing isn't available, because the cognitive muscle and the discipline atrophy. Like kids who are never taught to do things for themselves.

tac19 7mo ago
Do you refuse to use a calculator or spreadsheet, because doing long hand division helps you exercise your mental muscle? Do you refuse to use a database, because it will make your memory weaker? Or, do you refuse to use a car, because it makes you less able to walk when the car is unavailable? No. Because the car empowers you to do something that, at the very least, takes a lot longer on foot. People have worried with every single new technology that it will enfeeble the masses, rather than empower them, and yet in the end, we usually find ourselves better off.
- bluefirebrand 7mo ago
  > Do you refuse to use a calculator or spreadsheet, because doing long hand division helps you exercise your mental muscle Yeah when I was learning in school we weren't allowed electronics for division, and I think I absolutely would be dumber if I had never done that > People have worried with every single new technology that it will enfeeble the masses, rather than empower them, and yet in the end, we usually find ourselves better off. If you're posting this from America, you're living in a society that is fatter than ever thanks to cars. So there's surely some nuance here, not every technology upgrade is strictly better with no downsides
- wongarsu 7mo ago
  The car seems like a great example of a technology with a lot of problematic side effects. Places that had a more measured adoption ended up a lot better than those that replaced all public transit with cars and routinely demolished neighborhoods to make space for bigger highways Cars are an essential part of modern life, but the sweetspot for car adoption isn't on either of the extremes

HarHarVeryFunny 7mo ago

> Across studies, participants with higher trust in AI and lower need for cognition and fluid intelligence showed greater surrender to System 3 So the smart get smarter and the dumb get dumber? Well, not exactly, but at least for now with AI "highly jagged", and unreliable, it pays to know enough to NOT trust it, and indeed be mentally capable enough that you don't need to surrender to it, and can spot the failures. I think the potential problems come later, when AI is more capable/reliable, and even the intelligentsia perhaps stop questioning it's output, and stop exercising/developing their own reasoning skills. Maybe AI accelerates us towards some version of "Idiocracy" where human intelligence is even less relevant to evolutionary success (i.e. having/supporting lots of kids) than it is today, and gets bred out of the human species? Maybe this is the inevitable trajectory: species gets smarter when they develop language and tool creation, then peak, and get dumber after having created tools that do the thinking for them? Pre-AI, a long time ago, I used to think/joke we might go in the other direction - evolve into a pulsating brain, eyes, genitalia and vestigial limbs, as mental work took over from physical, but maybe I got that reversed!

RodgerTheGreat 7mo ago
I think everyone who believes that they can personally resist the detrimental psychological effects of exposure to LLMs by "remaining aware" or "being careful", because they have cultivated an understanding of how language models work, is falling into precisely the same fallacy as people who think they can't be conned or that marketing doesn't work on them. Don't kid yourself. If you use this junk, it's making you dumber and damaging your critical thinking skills, full-stop. This is delegation of core competency. You may feel smarter, or that you're learning faster, of that you're more productive, but to people who aren't addicted to LLMs it sounds exactly like gamblers insisting they have a foolproof system for slots, or alcoholics insisting that a few beers make them a better driver. Nobody outside the bubble is impressed with the results.
- thesumofall 7mo ago
  I fully agree that it’s close to impossible to not eventually fall into the trap of overrelying on them. However, it’s also true that I was able to do things with them that I would never have done otherwise for a lack of time or skill (all sorts of small personal apps, tools, and scripts for my hobbies). Maybe it’s a bit similar to only reading the comment section in a newspaper instead of the news? They will introduce you to new perspectives but if you stop reading the underlying news you’ll harm your own critical thinking? So it’s maybe a bit more grey than black & white?

andai 7mo ago

Damn. I came up with a hypothetical "System 3" last year! I didn't find AI very helpful in that regard though. Current status: partially solved. Problem: System 2 is supposed to be rational, but I found this to be far from the case. Massive unnecessary suffering. Solution (WIP): Ask: What is the goal? What are my assumptions? Is there anything I am missing? -- So, I repeatedly found myself getting into lots of trouble due to unquestioned assumptions. System 2 is supposed to be rational, but I found this to be far from the case. So I tried inventing an "actually rational system" that I could "operate manually", or with a little help. I called it System 3, a system where you use a Thinking Tool to help you think more effectively. Initial attempt was a "rational LLM prompt", but these mostly devolve into unhelpful nitpicking. (Maybe it's solvable, but I didn't get very far.) Then I realized, wouldn't you get better results with a bunch of questions on pen and paper? Guided writing exercises? So here are my attempts so far: reflect.py - https://gist.github.com/a-n-d-a-i/d54bc03b0ceeb06b4cd61ed173512f1f#file-reflect-py https://gist.github.com/a-n-d-a-i/d54bc03b0ceeb06b4cd61ed173... unstuck.py - https://gist.github.com/a-n-d-a-i/d54bc03b0ceeb06b4cd61ed173512f1f#file-unstuck-py https://gist.github.com/a-n-d-a-i/d54bc03b0ceeb06b4cd61ed173... -- I'm not sure what's a good way to get yourself "out of a rut" in terms of thinking about a problem. It seems like the longer you've thought about it, the less likely you are to explore beyond the confines of the "known" (i.e. your probably dodgy/incomplete assumptions). I haven't solved System 3 yet, but a few months later found myself in an even more harrowing situation which could have been avoided if I had a System 3. The solution turned out to be trivial, but I missed it for weeks... In this case, I had incorrectly named the project, and thus doomed it to limbo. Turns out naming things is just as important in real life as it is in programming! So I joked "if being pedantic didn't solve the problem, you weren't being pedantic enough." But it's not a joke! It's about clear thinking. (The negative aspect of pedantry is inappropriate communication. But the positive aspect is "seeing the situation clearly", which is obviously the part you want to keep!)

nasretdinov 7mo ago

I mean... I don't really check calculations made by a computer (e.g. by my own programs) all that often either and I think I'm completely fine :). But I guess the difference is that we kind of know how computers work and that they're generally super accurate and make mistakes incredibly rarely. The "AI" (although I disagree with "I" part) is wrong incredibly often, and I don't think people appreciate that the difference to the "traditional" approach isn't just significant, it's astronomical: LLMs make things up at least 5% of the time, whereas CPUs male mistakes maybe (10^-12)% of time or less. It's 12 orders of magnitude or so.

danielbln 7mo ago
How often do humans make mistakes? That's the better comparison.

thr0waway001 7mo ago

AI reminds of listening to any person who seems like an intellectual authority on multiple subjects on YouTube and is not afraid to wax confidently on any topic. They seem very intelligent and knowledgable until they actually talk about something you know. In other words, I try to learn from it whenever it does something I can't do but when it does something I can do or something I'm really good at it I find myself wanting to correct it cause it doesn't do it that well. It just seems like a really quick thinking and fast executing but, ultimately, mid skilled / novice person.

ahd94 7mo ago
And it starts showing impatience when its about to run out of context, more like someone who wants to get out of the office exactly at 5.
- kykat 7mo ago
  Not just when running out of context, it's always. Once it fixates on a goal, all hell breaks loose and there's nothing that it won't be sacrificed to get there. At least that's my experience with Claude Code, I am pressing the figurative breaks all the time.
  - bitwize 7mo ago
    "I'm Mr. Meeseeks, look at meeee!"
    - thr0waway001 7mo ago
      Claude, I need you to help me take two strokes off my golf game.
- xg15 7mo ago
  It was sort of funny when codex switched mid-session from "patch complete, I'll now automatically run the tests and verify results" to "patch complete, if you want to run the tests, just paste this instruction into a terminal somewhere: ..." Apparently this was caused by the context window getting full! (At least I assume that because it went back to the old behavior after I triggered a compaction)

johnnymonster 7mo ago

blocking access to a site because you don't enable javascript is diabolical

bjourne 7mo ago

"Time pressure (Study 2) and per-item incentives and feedback (Study 3) shifted baseline performance but did not eliminate this pattern: when accurate, AI buffered time-pressure costs and amplified incentive gains; when faulty, it consistently reduced accuracy regardless of situational moderators." I LOLed.

bobokaytop 7mo ago

[dead]

danilor 7mo ago

I couldn't figure if this was published to a journal? Or is it only published to a pre-print server?

tastroder 7mo ago
SSRN is a preprint server and that is the only published version.

deevelton 7mo ago

Have been curious what it could look like (and whether it might be an interesting new type of “post” people make) if readers could see the human prompts and pivots and steering of the LLM inline within the final polished AI output.

woopsn 7mo ago

In the technophile's future people aren't just getting dumber, not wanting to think or forgetting how - they aren't allowed to think. Maybe about anything. It's too big liability, costs too much to support, moreover detracts from the product. Like Sam A telling those Indian students they aren't worth the energy and water. That's what we're dealing with.

windowliker 7mo ago
That's reminiscent of Kurt Vonnegut's vision of the future in Harrison Bergeron.

vicchenai 7mo ago

[dead]

memolife23 7mo ago
[dead]

pink_eye 7mo ago

Can it design and implement a plutonium electric fuel cell with a 24,000 year half life? We have yet to witness it. Can it automate Farming and Agriculture? These are the real questions. #Born-Crusty

andrewssobral 7mo ago

[dead]

meander_water 7mo ago

I'm conflicted about this. As I was reading the paper, my AI detector senses were tingling all over the place. Large parts of the paper score very high probability of being written entirely by AI in gptzero. I'm not sure if I could trust anything written in it.

yubainu 7mo ago

[flagged]

psybrg-prtcls 7mo ago

Anyone else get the distinct impression that parts of this paper were written by AI?

nemo44x 7mo ago
Why shouldn’t it be? You can gather and form your thoughts, create a draft, and then have a LLM rewrite it for you. You can write in the style you prefer so you can focus on thoughts and then have the LLM rewrite it in the appropriate style for the audience.
- gwern 7mo ago
  One might worry that it would increase the authors' confidence even following their LLM rewrite errors and reduce accuracy overall regardless of moderators.
  - nemo44x 7mo ago
    You still need to review and edit.
    - gwern 7mo ago
      I was being sarcastic, because that point obviously also applies to the subjects in the experiment as well.

Null-Set 7mo ago

The original reseaech around thinking fast and slow (aka system 1 system 2 thinking) failed to be replicated when researchers tried.

seu 7mo ago

There's a very interesting critique of Kahneman's "Thinking fast and slow" from German psychologist Gerd Gigerenzen: https://www.researchgate.net/publication/397923694_The_Legacy_of_Daniel_Kahneman_A_Personal_View https://www.researchgate.net/publication/397923694_The_Legac... I suggest everyone interested in learning how these theories emerge, and how the social sciences work, to give it a read. Also, it kind of dismantles the whole idea of System 1 and 2, which then I guess would question the theoretical foundations of this paper too.

fhi 7mo ago
I've read the work, Gerd Gigerenzen criticises the approaches to scientific experiments conducted by Daniel Kahneman and Amos Tversky, and the lack of desire, to put it mildly, to properly define the terms they were introducing. Critique to System 1 and 2 is based mostly on using System 1 and 2 to excuse the alleged deficiencies in the experiments. I think the original article in this discussion is using "Systems 1 and 2" as intuitive and rational modes for problem-solving and interestingly enough, Gerd Gigerenzen also has a reference in this work "accuracy-effort trade-off (Payne, Bettman, and Johnson 1993): The less effort one takes, the less accurate one will be." which aligns with the broader idea of Systems 1 and 2.

alexchengyuli 7mo ago

The paper puts AI next to System 1 and 2, but those are ways you think. With AI the thinking still happens, you just can't see or control it anymore. When you googled something and got five contradictory results, that told you the question was hard. A clean AI answer doesn't give you that signal. Coherence looks the same whether the answer is right or wrong. The failure mode didn't get worse. It got quieter.

bsenftner 7mo ago

A major problem with LLM AIs is their core nature is not understood by the vast majority of everyone - developers included. They are an embodiment of literature, and if that confuses you you're probably operating on an incorrect definition of them. I like to think of them as idiot savants with exponential more savant than your typical fictional idiot savant. They pivot on every word you use, each word in your series activating areas of training knowledge, until your prompt completes and then the LLM is logically located at some biased perspective of the topic you seek (if your wording was not vague and using implied references). Few seem to realize there is no "one topic" for each topic an LLM knows, there are numerous perspectives on every topic. Those perspectives reflect the reason one person/group is using that topic, and their technical seriousness within that topic. How you word your prompts dictates which of these perspectives your ultimate answer is generated. When people say their use of AI reflects a mid level understanding of whatever they prompted, that is because the prompt is worded with the language used by "mid level understanding persons". If you want the LLM to respond with expert guidance, you have to prompt it using the same language and terms that the expert you want would use. That is how you activate their area of training to generate a response from them. This goes further when using coding AI. If your code has the coding structure of a mid level developer, that causes a strong preference for mid level developer guidance - because that is relevant to your code structure. It requires a well written prompt using PhD/Professorial terminology in computer science to operate with a mid level code base and then get advice that would improve that code above it's mid level architecture.

ben_w 7mo ago
In two words "book smart". In more words, "of course it's stupid, it's as complex as a mid-sized rodent where we taught it purely by selective breeding on getting answers right while carefully preventing any mutations which made their brains any bigger".
- cucumber3732842 7mo ago
  It's basically plotting the dots of all easily accessible written word to find your words, finding the words that are answers to your words and then charting a line through them no matter how scattered those points may be, and spitting that back out. It doesn't "know" anything nor is it reasoning even if the results are similar. You have to come into it with the same "these people are only stupid and lack the experience to answer my questions despite thinking they do, they lack the world view to even process how I arrived at the parameters of my question(s)" apprehension like you would if asking reddit about some hazardous thing that would make them all screech. AI is the margarine to that butter. It's a technology with potential to deliver great value, but there are limitations...

BANRONFANTHE 7mo ago

[dead]

tim333 7mo ago

I'm not sure if this is saying people were given a task and the option to consult an AI. When they did they were influenced by its answer. Which is kind of duh? Of course. They have some cool language like calling the AI system 3 and calling taking advice 'cognitive surrender' but I'm not sure how this differs from asking your mate Bob and taking his advice?

pugchat 7mo ago

[dead]

gregfrank 7mo ago

This framing points at something important that I think the alignment evaluation literature often misses: the distinction between what a model represents internally and what it does behaviorally. Probing can tell you what's in the representations, and linear probes can be surprisingly accurate. But in experiments I've run on DeepSeek and Qwen models, high probe accuracy for a given behavior doesn't predict whether the model actually routes through that behavior at inference time. The detection layer and the routing layer are architecturally separable, and most evaluation benchmarks are measuring the former while claiming to measure the latter.

michelejoseph 7mo ago

contradish catches when 2 users get different answers to the same question

iydr 7mo ago

This paper is irrelevant: "AI assistant (ChatGPT; GPT‑4o)" (I found no others, including 0 "claude") It's also probably bad about something else important.