4 ms·
I had a weird LLM use instance happen at work this week, we were in a big important protocol review meeting with 35 remote people and someone asks how long IUDs
by randycupertino 11mo ago
I had a weird LLM use instance happen at work this week, we were in a big important protocol review meeting with 35 remote people and someone asks how long IUDs begin to take effect in patients. I put it in ChatGPT for my own reference and read the answer in my head but didn't say anything (I'm ops, I just row the boat and let the docs steer the ship). Anyone this bigwig Oxford/Johns Hopkins cardiologist who we pay $600k a year pipes up in the meeting and her answer is VERBATIM reading off the ChatGPT language word for word. All she did was ask it the answer and repeat what it said! Anyway it kinda made me sad that all this big fancy doctor is doing is spitting out lazy default ChatGPT answers to guide our research :( Also everyone else in the meeting was so impressed with her, "wow Dr. so and so thank you so much for this helpful update!" etc. :-/
- mtlynch 11mo ago>her answer is VERBATIM reading off the ChatGPT language word for word How could it be verbatim the same response you got? Even if you both typed the exact same prompt, you wouldn't get the exact same answer.[0, 1] [0] https://kagi.com/assistant/8f4cb048-3688-40f0-88b3-931286f8ac4b https://kagi.com/assistant/8f4cb048-3688-40f0-88b3-931286f8a... [1] https://kagi.com/assistant/4e16664b-43d6-4b84-a256-c038b1534999 https://kagi.com/assistant/4e16664b-43d6-4b84-a256-c038b1534...
- randycupertino 11mo agoWe have a work enterprise GPT account across the company.
- mtlynch 11mo agoHow does that explain what you observed? The only way I can understand that as an explanation is if your entire company can see each other's chats, and so she clicked yours and read the response you got. Is that what you're saying?
- randycupertino 11mo agoHow else would she have been able to parrot the exact same GPT response without reading it directly? You think she just thought of it word for word exactly the same off the top of her head?
- khannn 11mo agoThey're saying that the shared account is enough for OpenAI to provide the same result. Very interesting, I'd like to know more like was it a generic IUD or a specific one in the query. Also, the Doc is a cardiologist, they don't specialize in Gyno stuff and their training/schooling is enough for them to evaluate sources. Just for reference before AI it was typical for employers of doctors to pay for a service/app called UpToDate which provided vetted info for docs like google.
- xeonmc 11mo agoItemized Bill: - hitting the ChatGPT button: $1 - knowing where to hit the ChatGPT button: $600k/year
- randycupertino 11mo agoThere were several specific brands cited in the response and she read through them one by one in the same order with the supporting details, word for word. I think it just gave us the same response and she read it off the page.
- grey-area 11mo agoThe LLM may well have pulled the answer from a medical reference similar to that used by the dr. I have no idea why you think an expert in the field would use ChatGPT for a simple question, that would be negligence.
- anonymouskimmer 11mo agoA climate scientist I follow uses Perplexity AI in some of his YouTube videos. He stated one time that he uses it for the formatting, graphs, and synopses, but knows enough about what he's asking that he knows what it's outputting is correct. An "expert" might use ChatGPT for the brief synopsis. It beats trying to recall something learned about a completely different sub-discipline years ago.
- pardon_me 11mo agoThis is the root of the problem with LLMs. At best, the can attempt to recall sections of scraped information, which may happen to be the answer to a question. No different to searching the web except you instantly know the source and how much to trust it, if you search yourself. I've found LLMs tend to invent sources when queried (although that seems to be getting better), so it's slower than searching for information I already know exists. If you have to be more of an expert than the LLM to then verify the output, it requires more careful attention than going back to the original source. Useful, but it's always writing in a different way to previous models/conversations and your own writing style. LLMs can be used to suggest ideas and summarize sources, if you can verify and mediate it. They can be used for a potential sourcing of information (and the more data agreeing, the better). However, they cannot readily be used to accurately infer new information, so the best they can do here is guess. It would be useful if they could provide a confidence indicator for all scenarios.
- randycupertino 11mo agoShe read it EXACTLY as written from the ChatGPT response, verbatim. If it was her own unique response there would have been some variation.
- NortySpock 11mo agoOr both the doctor and ChatGPT were quoting verbatim from a reputable source?
- anonymouskimmer 11mo agoThe one thing a cardiologist should be able to do better than a random person is verify the plausibility of a ChatGPT answer on reproductive medicine. So I guess/hope you're paying for that verification, not just the answer itself.