6 ms·
AI advice made people less accurate but more confident – sudy
- al_borland 4mo ago[dead]
- rho138 4mo agoI appreciate that they’re bringing this study to light… however, none of the source links actually trace back to the quoted study - instead pointing to their own domain or business insider.
- toomuchtodo 4mo agohttps://osf.io/preprints/psyarxiv/5y6m4_v1 https://osf.io/preprints/psyarxiv/5y6m4_v1 Marcoccia, C., et al. AI Advice Suppresses People’s Willingness to Say “I Don’t Know”, Even When the Advice Is Wrong and Accuracy Is Incentivized. PsyArXiv, 15 July 2026, osf.io/preprints/psyarxiv/5y6m4_v1
- wisty 4mo agoAI pessimist view. Even if AI gets smarter it will still be agreeable, and people will use it to more confidently reinforce thier stupider ideas, especially in areas where they lack the knowledge to know if they are right. Even if this can be solved, technically, people won't want to use the model that says they are wrong, so they will choose the glib lies that reinforce their beliefs. Freedown of speech forces people to think they may be wrong, but freedom of association lets people avoid this. AI is going to be internet hug box echo chambers at an unbelievable scale.
- emp17344 4mo agoThe solution is to stop treating AI as superhuman semi-living entities, as many boosters are apt to do. We can acknowledge it as a useful tool while understanding it has limits and is not “alive” or “thinking” in the same way a human is.
- palmotea 4mo ago> The solution is to stop treating AI as superhuman semi-living entities, as many boosters are apt to do. It's not just fanboys doing that, but "official" educational channels. Both an internal training (more like an all-hands knowledge share) and some dumb training session by a guy from Anthropic explicitly, repeatedly recommended users anthropomorphize their chatbots on no uncertain terms. "Treat it as your super-smart coworkers", "I named mine Becky and I go to her with all my work challenges", etc.
- customguy 4mo agoChatGPT just hit me with "I smiled at this:" ...
- crooked-v 4mo agoWhat I'd like is the TNG ship computer: an extremely smart system that nonetheless limits its "creativity" to the clearly marked holodeck, where simple prompting is treated like the casual throwaway diversion it is, and actual authors are still putting in substantial effort and personal preferences and viewpoints into their work.
- lll-o-lll 4mo agoThat’s not a solution. Humans are wired to anthropomorphise; we do it with sticks and rocks. How much more so with LLMs. We’ve invented a technology that hijacks that part of humanity more effectively than anything ever. Asking people to “not be so stupid” is not going to help.
- emp17344 4mo ago
- andy99 4mo agoThis was posted earlier but didn’t get traction, and I made the following comment: Id want to know if “AI” makes a material difference vs just having access to the wrong answer. Like someone could be given search access that successfully retrieved wrong answers to questions, would that give the same results. How much do uniquely AI characteristics, like sycophancy or the conversational aspect play into this, vs people just being willing to believe what they read?
- fxwin 4mo agoYou can read the study here: https://osf.io/preprints/psyarxiv/5y6m4_v1 https://osf.io/preprints/psyarxiv/5y6m4_v1 TLDR: They studied both cases (Access to a LLM/Chat interface which gives a wrong answer when asked, and access to pregenerated (wrong) answers). Both experiments yielded similar results.
- tsimionescu 4mo agoThat is not a good summary. In all their versions of the experiment, they presented the tool as AI, and used actual LLM answers. In the first example, participants were directly interacting with a real, though small, LLM, but they had technical issues because of that - 10% of the time the LLM setup failed to present an answer at all. So they redid the experiment with pre-generated answers - the participants saw the same UI, but when they asked the question of the AI, they instead got one of 3 pre-generated answers from that AI, to avoid the technical issues.
- fxwin 4mo agoim not sure which part of my summary you take issue with then? the reason for study 1b is not relevant to answer the question asked in the comment i replied to
- tsimionescu 4mo agoIt is relevant, because of (1) framing (people thought they were getting answers from AI, not a Google search, and any associated reputation works in that way); and (2) sycophancy and other similar characteristics of the answer's text, which were present in the answers presented in 1b and wouldn't be in a Google search. Basically, the difference between 1a and 1b is not at all relevant to the question of whether the observed behavior is caused by AI or simply by faulty tools. The difference between 1a and 1b was designed specifically to be transparent to the actual test takers, and only to eliminate some confounding variable (technical issues in 1a).
- reticulates 4mo agoAdvice and information subreddits have gone to shit because of AI usage. A large number of people seem to think that when someone asks a question, what they really want is not someone with direct knowledge, but instead someone to relay the question to ChatGPT and post the result as if it is their own hard earned knowledge and insight. I have no idea about the quality of this research but in the real world (well, real-ish, as real as Reddit can be) it is stark, people aren’t just refusing to say “I don’t know” they’re actively seeking out opportunities to pretend they know things.
- darkstarsys 4mo agoExactly the same thing happened when Google appeared on the scene. (Not that this makes it any better!)
- skippyfish 4mo agoSort of, but I think this was a mix of "I want to sound smart" and "I actually want to know the answer", because summarizing Google search results still required you to read the content, form an opinion, and then rephrase it. With AI, people literally just act as token proxies and are doing it purely for internet points (upvotes, followers, etc)... I guess on some level, they're just providing a service that the "market" implicitly values. But also, in Bob Slydell's voice: "what would you say you do here?"
- wccrawford 4mo agoI think some may think they're helping like that, but I'm sure there are quite a few who enjoy the feeling of being a notable figure in the community because they answer so many questions. These people have always existed and gave bad answers, but now they can do it so much faster, and with so much less actual thinking.
- ffaccount2 4mo agoI did that when I was younger and internet forums we're still a thing. The difference is that I had to do actual research, which often took me half an hour or more, to solve the problem for the asker. I got knowledge and "the feeling of being a notable figure" as you have said. The other side for the answer. I believe my answers were good. The difference is that nowadays people can just skip the research part and copy-paste Llama. No value added.
- 1659447091 4mo agoAI assisted/induced Dunning–Kruger effect
- fwlr 4mo agoVery concerning - I’ve tried n+m, n*m, n^m, and even m^n, but I can’t seem to reproduce the popular “10x” claim.
- warkdarrior 4mo agoDid you try n*m where n=10 and m=x?
- dwohnitmok 4mo agoThis study is pretty bad. The comment (https://news.ycombinator.com/item?id=48970182 https://news.ycombinator.com/item?id=48970182) on the other link with the direct PDF explains the problem well, which is that nothing here being tested is specific to AI systems. This study gave people access to an LLM that the researchers knew would give incorrect answers to certain questions, and then quizzed people on those questions, with the option to not respond to a given question if they are unsure about the answer. This is akin to giving someone a textbook on an obscure subject that has certain factual errors, letting them know they can use that textbook in a quiz on that subject, and then quizzing that person on those facts that the textbook gets wrong. Obviously that person is both more likely to be willing to respond to the question and is more likely to get it wrong! There are a lot of things I'm very interested in that are specific to modern LLMs and how they affect learning and confidence (sycophancy, cognitive helplessness, etc.). This study tested none of those. Its experimental setup is not very different than simply substituting the LLM with a textbook with errors.
- nkrisc 4mo agoYou could make the point that it’s no different than the textbook example you gave, but people don’t generally use textbooks like that, while out in the world people do use LLMs like that all the time.
- dwohnitmok 4mo agoPeople do use textbooks like that all the time in the experimental setup tested (essentially an open book quiz). I agree there are important differences in how textbooks and LLMs are used in real life. This study didn't explore that at all. It used a setup that essentially elided the difference between the two. This is why I think it's a bad study. It didn't measure anything of the essential differences of how people use LLMs.
- nkrisc 4mo agoIt would be interesting to test those LLM-specific features and issues, but I don’t see how it’s a bad study if it does reflect how people actually use LLMs, even if they could use other sources similarly. The number of people using LLMs must dwarf the number of people using textbooks for any reason.
- scoriiu 4mo ago[flagged]
- lukeschlather 4mo agoDid they ask any substantive questions? It sounds like they asked movie trivia questions, in a way that the LLM was almost guaranteed to be wrong. So extremely low-stakes where there is no conceivable downside to being wrong.
- tsimionescu 4mo agoThat was exactly the point - there was no downside to saying you just don't know, people didn't know, but they still chose to use (what they thought was) AI and got it wrong that way.
- abernard1 4mo agoWouldn't it be wild if there were real-world penalties for being wrong? And people who follow bad advice will get bad results, and people who follow good advice will get good results? I wonder if this will impact the quality and prevalence of certain AI models in the future.
- onlyrealcuzzo 4mo agoThe TL;DR delivers: > Researchers found AI advice suppressed judgment suspension from 44% to 3%, accuracy from 27% to 9%, while confidence rose from 30% to 76%. People trusted wrong AI answers. I didn't see the article mentioned what they were actually asked, but I'm surprised confidence was only ~30%.
- m0llusk 4mo agoRight now we don't have AI. We have LLMs. LLMs compose responses and have now ability to say they don't know any answer or are unsure. This combines with the chipper manner of their communication to make for a dangerous mix.
- deleted 4mo ago[deleted]
- idopmstuff 4mo ago"The study, authored by Capraro with Chiara Marcoccia of École Normale Supérieure and Walter Quattrociocchi of Sapienza University of Rome, deliberately used questions where AI models typically fail: visual details from films, such as the colour of a team’s uniform in Bend It Like Beckham." I get why they used questions where AI models fail, but it also really reduces the value of this study. Nobody is really asking AI the color of a team's uniform in a movie, and if they do and confidently get it wrong, it just doesn't matter at all. Asking trivial questions also feels like it would affect the rate at which people are willing to confidently say things that are wrong. If you ask me some question of pointless trivia and I ask ChatGPT, I'll probably just repeat the answer because who cares. If you ask me something even mildly important and I ask ChatGPT, I'll either verify the information before I repeat it to you, or I'll qualify that I looked it up with ChatGPT and didn't verify. But some things are just so unimportant that they don't even warrant the disclaimer.
- formvoltron 4mo agois that better or worse than 3 beers?
- SubiculumCode 4mo agoInstead of monetary rewards, they should preface it with, "This is AI model is not very accurate on these types of questions" and then see how they fare. In some areas, AI is quite accurate, so their experience may lead to different expectations.
- nlawalker 4mo agoAnd that’s a net win for a lot of people in a lot of situations; that’s the part that needs acknowledgment and study, sociologically. You can’t say it didn’t warn us, it’s on the tin: “AI might be wrong.”
- imoverclocked 4mo agoI saw a post that some teachers are now asking students to ask ChatGPT to do their assignments… and then critique it where it is wrong. I feel like this might be an interesting method for places where the default response tends to be someone submitting copy-pasta: just give the ai-response and then have community discussion around what it missed.
- rsoto2 4mo agonice try clammy sammy. We're gonna continue to push for scientific based approaches to education not some lunatic's idea of an educational experiment that waste's children's education. We tried this with Bill Gates already.
- rsanek 4mo agoAnother case of study design not really supporting the headline. According to the paper, the comparison here was "no advice" or "AI advice": there was no "placebo" where you have advice / references from non-AI materials. Think e.g. Google's instant answer box, pre-AI. It's hard to say without the comparison, but I have to imagine displaying that would have similar effects. There's plenty of other issues with the study, one of the primary ones being that they chose a relatively 'dumb' AI (Step 3.5 Flash), but also, specifically hand-selected wrong answers. People that are used to using competent AIs that are mostly correct would be operating off of experience that suggests they should trust AI. In this case, by design, they shouldn't have, but it's hard to fault the user for that.
- diamondtea 4mo ago[dead]
- senshan 4mo agoThis could serve as a beacon for those who still care: Richard Feynman: "I have the advantage of having found out how hard it is to get to really know something, how careful you have to be about checking the experiments, how easy it is to make mistakes and fool yourself..." https://www.youtube.com/watch?v=tWr39Q9vBgo https://www.youtube.com/watch?v=tWr39Q9vBgo
- leecommamichael 4mo agoI read the study. The wager is: $0.10 awarded for a correct answer $0.10 deducted for a wrong answer $0.00 (no change for a refusal to answer) There was no detail on whether participants were actually going to be paid, or if they were in the negative at the end, be expected to pay their losses. I am inclined to believe the effects of the study are real, but not nearly as pronounced as the data. If there were more serious amounts of money on the table, I think common sense would prevail.
- luciana1u 4mo ago[flagged]
- Joel_Mckay 4mo agoAlso may be irrelevant for people at some jobs, and cause cognitive skill losses over time. https://www.youtube.com/watch?v=axOcn--n_lM https://www.youtube.com/watch?v=axOcn--n_lM https://www.anthropic.com/research/AI-assistance-coding-skills https://www.anthropic.com/research/AI-assistance-coding-skil... We should remember LLM do have legitimate use-cases like search, as we enter the "Trough of disillusionment" in the hype cycle. =3 https://en.wikipedia.org/wiki/Gartner_hype_cycle https://en.wikipedia.org/wiki/Gartner_hype_cycle
- greatsage_sh 4mo ago[flagged]
- fbastosbkk 4mo ago[flagged]
- ojinai 4mo ago[dead]
- zahlman 4mo ago> made people less accurate > sudy Intentional?
- TimByte 4mo agoObviously without a proper RAG pipeline or web search it is just going to hallucinate with maximum confidence. It would be more interesting to see the results with top tier models that have proper alignment to refuse to answer when token probability is low. As it is they just proved that people tend to trust well written text in a chat ui
- mcmisieck 4mo agoSo basically AI made people to be Americans ;)
- madhu_ghalame 4mo ago[dead]