5 ms·
> There's something incredibly peaceful about being in the hands of an expert you trust. [...] AI can absolutely shatter that feeling in an uncomfortable way [.
by AceJohnny2 3mo ago
> There's something incredibly peaceful about being in the hands of an expert you trust. [...] AI can absolutely shatter that feeling in an uncomfortable way [...] but I don't know if I can fully trust AI either.
This really is key. We know we can't trust the AI, but at the same time we're also more comfortable asking the AI for clarifications or confronting it. Not having a time-bound appointment or paying by the hour helps a lot. But even then, more information doesn't necessarily help!
I once brought my 11-year-old car, a Civic with 150k miles, to multiple garages. I figured I'd play the "second opinion" game to correlate what the garages recommended to decide on what needed to be done...
I got 3 completely unrelated recommendations, including one that I knew was invalid! I felt worse off than when I started!
The solution to uncertain information isn't more information, which the AI can certainly provide, it's better information, and AI cannot currently provide that.
- 010101010101 3mo ago> The solution to uncertain information isn't more information, which the AI can certainly provide, it's better information, and AI cannot currently provide that. I'd argue that AI _can_ currently provide that, but that it can't do it _reliably_, and that to non-experts it's impossible to differentiate, which makes it all the more dangerous.
- margorczynski 3mo agoIsn't that the case with human "experts"? If you had encounters with doctors, mechanics, etc. you'll know you can get a completely different diagnosis for the same problem which obviously means (in most cases) that the person you thought an expert is wrong. What is needed are studies that will take a cold look at the actual results because AI seems to be required to be perfect or it is useless. It just needs to be as good as a human for most stuff, but in the long run it will be much better. At least that what extrapolating current reality shows us.
- wwweston 3mo agoWe have systems around humans that exist to manage expertise gaps, credibility signals, and accountability. This is part of what makes humans as good as they are, along with specialized training and some measure of meritocratic selection. We license and regulate and account and litigate to make a system that responds and improves. Some of this might be applicable to LLMs, but some isn’t and much of it would be resisted. This is one reason we’re not likely to get “as good as a human” because at some level we’re not optimizing for the outcomes; we’re optimizing for speed, convenience, some participant’s economics, and underlying beliefs.
- malfist 3mo agoI've been going through PT for a hypermobility disorder related injury and I've use an AI to help me figure out "interview questions" to see if a PT knows anything about hypermobility or is willing to learn. I found it helpful to select a new PT after my first PT I trusted made things worse by prescribing stretches and no load progression from rest and recovery back to deadlifts
- draftsman 3mo agoMay I ask, is the hypermobility disorder you refer to EDS? If so, what was the injury?
- malfist 3mo agoMy doc didn't do the full set of test/exam for an hEDS diagnosis, so technically it's just generalized hypermobility spectrum disorder based on past medical history and a near "perfect" beighton score. It could be hEDS though, as a lay person reading the diagnostic criteria I either fit it, or are borderline. Injury was to my SI joint, I've historically always irritated it lifting, but I set a new PR for deadlifts and it was debilitating for 3 weeks before my other half made me stop being stubborn and see a doctor about it. I've also had my left shoulder joint surgically repaired after multiple dislocations.
- 3mo ago
- ed_elliott_asc 3mo agoThe soothing sound of ChatGPT telling us how right and clever we are…how could it possibly hallucinate, certainly not 5.5
- nonethewiser 3mo agoYou’ve really honed in on the key issue. This is exactly how keen hackers news commenters approach this.
- Aurornis 3mo agoI have multiple LLM subscriptions at any given time, plus an array of local models. When I ask a question outside of my domain of expertise I like to ask all of the LLMs I have access to. I also create separate sessions and ask the same question multiple ways. It’s revealing to see how many different and contradictory answers I get, most of which are presented confidently. The last time I ran a medical question through Claude I couldn’t even get consistent answers between sessions. It’s also scary how easily you can lead each LLM to the answer you have in mind. When I would start asking questions about different options that other LLMs had presented, each session would drift toward that explanation.
- Esophagus4 3mo agoHave you ever let the LLMs “discuss” with each other to see if that would give better answers? You might end up with the answer from the most persuasive LLM, but you might also end up with better results. Wonder if there is a paper out there on this.
- scheme271 3mo agoThe problem is how do you know whether the answer is just the most persuasive or actually the most accurate one? It's hard to figure this out without domain knowledge.
- XorNot 3mo agoWorse is that LLMs are trained to be persuasive by default. The "you're absolutely right..." stereotype is because these things are A/B tested on response quality and we know from studies people reliably rate vibes better then anything else - e.g. while the quality of hospital accomodations likely has some impact on patient outcomes, the view and decor of the room certainly did not fundamentally change the quality of the care provided but it is the largest determinant in how well people rate that care.
- Esophagus4 3mo agoI dunno, I could see it working. I do something similar with reviewing code: I have one agent write the code and another reviews it, then they go back and forth for a bit improving the code. Seems to yield better results than one agent alone. Seems like a similar principle.
- john-tells-all 3mo agoThere's a big difference between a _puzzle_ and a _mystery_. In a puzzle, the goal state is known, and as more pieces - data - appears, the goal gets closer. You know how far you are from the goal. A mystery is worse. With each additional piece of data, the goal gets farther away. Everything is more and more confusing. (Popularized by Malcom Gladwell)
- mrlongroots 3mo agoMaybe I am missing something but I just find this wrong. Everything is a puzzle: there is one "Truth" or one diagnosis. You (a smart human) should be able to converge on it by cross-examining your LLMs. By themselves, they have no interest in revealing this, no stakes, which makes them tools only useful at the hands of a capable investigator.
- Paracompact 3mo ago> You (a smart human) should be able to converge on it by cross-examining your LLMs. What makes you think this is fundamentally different from cross-examining ELIZA? There is no guarantee that the LLM will help you converge on anything. Indeed actually calling out an LLM on BS tends to eventually produce an "I don't know and can't help you further" answer (as it should).
- fc417fc802 3mo agoThe same goes for a human expert. There's no guarantee of convergence and you could eventually end up at "I don't know".
- mrlongroots 3mo ago> There is no guarantee that the LLM will help you converge on anything. Absolutely. The guarantee does not come from the LLM. The LLM is a simply an improved version of Google Search. The guarantee can only come from a systemic application of epistemic discipline and reasoning, which is very much (smart) human territory. Put it another way, I could make good decisions with/without LLMs, with some uncertain diagnostics as input. I would have to trawl through 50 papers myself, and it is possible that my decision arrives 5 years too late as a result. LLMs enable trawling and do some of the legwork in connecting the dots, but are ultimately only as capable as the orchestrating human.
- Bratmon 3mo agoTo provide a competing point of anecdata: A Gemini diagnosis saved me $3,000 in unnecessary repairs on my Civic.
- dyauspitr 3mo agoSaved me $2000 on a koi pond pump and filtration system
- fluidcruft 3mo agoYouTube has saved me at least that much in appliance repairs... and it doesn't even have an AI. It's amazing how valuable access to information can be.
- deleted 3mo ago[deleted]
- ahepp 3mo agoI would love to hear more about this
- serial_dev 3mo agoThese tools can’t reliably fix a 4px misalignment on my icon, better ask them about a medical report… but honestly, I would do the same.
- Gigachad 3mo agoTbh LLMs pulling data out of medical documents in it's training set and searchable online is likely a much easier task than fixing some weird CSS alignment issue.
- dd8601fn 3mo agoAlso most of them can’t actually see what they’re doing. It’s hard for me to get things pixel perfect while blindfolded, too.
- UltraSane 3mo ago> There's something incredibly peaceful about being in the hands of an expert you trust This is the primary business model of enterprise IT and is why companies pay so much for 4 hour disk replacement.
- nonethewiser 3mo agoYou only got 3 opinions on your car? Why not 50? You could have found a more useful signal by getting more information. I get it - getting an opinion from a mechanic is time consuming. Not true of AI though.
- jdblair 3mo agoThe best mechanic I ever had kept my ‘98 Subaru going past 200k miles. Once during a repair I asked him to do an inspection and tell me if there was anything else I should replace. He told me not to do that, and that any mechanic would always find something, but not necessarily the next thing to break. He said it better using an expression I hadn’t heard before or since, something like “don’t go looking for goats when your herd is already with you.”
- dumb1224 3mo agoExactly. Old parts of the system will be working if you leave them undisturbed. Mechanics have very good intuitions of this sort of thing. I read about before there's proper engineering / physics theory about this too, it's like a car as a machine is a linear/smooth physics system with multiple weaknesses. Overtime longtime period of running many places might weaken but it still evolves into a slightly different smooth system, until you introduce a replacement which cause a mis-match of impedance or something like that.
- tass 3mo agoMaintenance-induced failures are what it’s called with small aircraft. You’ll do something to prevent a failure (like, replace an old but functional alternator) but cause an oil leak or engine vibrations because you had to remove the propeller to complete the job.
- ryukoposting 3mo ago> I got 3 completely unrelated recommendations, including one that I knew was invalid! I felt worse off than when I started! I almost had a very similar experience with my beater Lexus. It took 2 independent shops and 3 dealers to finally figure out what was causing the ABS to go off randomly at low speeds. Turns out there's some obscure Toyota-specific tool from the late '90s that picked up a proprietary diagnostic code, and the third dealer was the only one that still had that particular piece of equipment. ...and of course, the thing that's broken has been out of production for 20 years and remanufactured ones cost more than the car is worth. I ended up just unplugging the ABS control module. Point being: once I knew what was wrong, all the seemingly contradictory information from the other 4 shops suddenly fit together. It's just such a weird thing to go wrong that no reasonable tech would ever have considered it.
- weatherlite 3mo ago> it's better information, and AI cannot currently provide that It sometimes can, if it straight out never can no one would use it. People use it , lots of them.
- dumb1224 3mo agoI tried that AI diagnosis for my 15 old Ford C MAx too, however with a diagnostic problem the issue is unless you've got the ground truth, there's simply no way to verify any tool / human with a metric that you can compare and decide on future tasks. The AI might be very good at diagnosing all minor issues, but might not lead to a successful repair, whereas human mechanics are extremely good on 80% of major issues that's not the ground truth, but will lead to successful repairs (that might not address the root but simply patch it). So it comes down to manage expectation / outcomes.
- darkwater 3mo ago> I got 3 completely unrelated recommendations, including one that I knew was invalid! I felt worse off than when I started! I would frame it differently: you now know which shops are not to be trusted. So, next time you need one, you will take a better decision.
- abirch 3mo agoThere are few things better in this world than having a car shop you can trust. I found one and pray that management doesn't change.
- throwaway2037 3mo agoYou nerd sniped me with the story about your used car. What happened in the end? I really want to know! There are some fun YouTube channels that basically do the same. Someone who is an expert auto mechanic takes a used car to various repair garages and asks them to recommend a course of action.
- namelessone 3mo agoSounds like a fun watch! What is the name of the channel?
- jbs789 3mo agoEspecially in the medical field where the placebo effect / mindset shapes outcomes.
- clates 3mo ago> The solution to uncertain information isn't more information, which the AI can certainly provide, it's better information, and AI cannot currently provide that. Aside from the LLM-ism (it isn't foo, it's bar) - this is a thought terminating cliche. You definitionally don't know if some information is better or not given that you were uncertain about the information in the first case. "I went to three mechanics and got three different answers" - your takeaway is just "Ah - I clearly need better informed mechanics." Which is on it's face absurd because if you could clearly judge the ability of the mechanics you wouldn't need their evaluation. You'd just do the evaluation yourself.
- rockostrich 3mo agoThere are 3 kinds of mechanics: Scammers who do the lowest effort diagnostic and "fix" to get you to pay a smaller amount of money to fix the problem in the short term even though it'll re-present itself a week/month/year later. Upsellers who will find other things "wrong" with your car and pressure you into paying to fix them because they sound a lot worse than they are. Good mechanics that will explain what they did to diagnose the issue and recommend different options depending on what the issue is. Funnily enough, I've found that doctors tend to also fit into these 3 archetypes.
- palata 3mo agoYes, and that's a problem. Doctors (or experts in general) hate it when people don't trust them, but the thing with experts is that people have to trust them. And in my life (and a few times just in the last few years), enough doctors have been wrong enough that I cannot just trust them anymore [1]. If it is important, I will ask them to explain to me, and sometimes I will just ask for a second opinion. I have read about doctors complaining that "with AI, patients now come with their own diagnosis and don't trust us when we say it's bullshit, and it is a problem". I can feel for them, but if they give the feeling that they don't listen to the patients and the patients don't trust them, it's not only the patients' fault, I would say. [1]: I have more than one examples of my relatives like this: A doctor says "wow that's bad go to the ER", the ER says "nope it's all good, go home", first doctor learns about that and says "WTF you GO TO THE ER, call me and I will insult them on the phone", and finally resulting in a surgery where the doctors say "they were lucky we could operate right now, because in a matter of hours they could have died from this". How in the world can I trust them after one event like this? Happened to me (in some variation) 3 times. Not based on an LLM diagnosis in the first place: based on a doctor's diagnosis.
- rockostrich 3mo agoHeh, I hear stories like that everyday from my partner who is an ICU nurse. Not as dire, but there are constant inter-department arguments about moving patients because of resource constraints and the ICU could end up completely understaffed/resource constrained if the wrong NP or charge nurse is working. I'm amazed our healthcare system works at all to be honest.