18 ms·
We need to tell people ChatGPT will lie to them, not debate linguistics
- retrac 4y agoIndeed. We are anthropomorphizing them. I do it all the time and I should know better. There are already a few reports floating around of people who have seemingly been driven mad, come to believe strongly that the language model they're using is a conversation with a real person. A lot of people will really struggle with this going forward, I think. If we're going to anthropomorphize, then let us anthropomorphize wisely. ChatGPT is, presently, like having an assistant who is patient, incredibly well-read, sycophantic, impressionable, amoral, psychopathic, and prone to bouts of delusional confidence and confabulation. The precautions we would take engaging with that kind of person, are actually rather useful defenses against dangerous AI outputs.
- 0x008 4y agodo you have a link for any of those reports? would be interesting to read to say the least.
- bsaul 4y agoI love this description. The only thing left is to generate the corresponding avatar with dall-E
- deleted 4y ago[deleted]
- lancesells 4y agoIt would be great if the avatar was just this endlessly morphing thing that relates to the text. Talking about conspiracies? It's a lizard man. Talking about nature? It's a butterfly. It starts to lie? Politician.
- int_19h 4y agoWe already have a perfect avatar for these things. https://knowyourmeme.com/photos/2546575-shoggoth-with-smiley-face-artificial-intelligence https://knowyourmeme.com/photos/2546575-shoggoth-with-smiley...
- andai 4y agoI like Steve Yegge's "grad student experiencing the residual effects of shrooms" (lightly paraphrased).
- babyshake 4y agoA bit off topic, but FYI (I guess to Steve), if you took shrooms 4 hours ago, they have not started to wear off.
- ihatepython 4y agoAnybody know what are ChatGPT's pronouns?
- MacsHeadroom 4y agoshog/goth
- audunw 4y ago> What are your pronouns? > As an AI language model, I don't have personal pronouns because I am not a person or sentient being. You can refer to me as "it" or simply address me as "ChatGPT" or "AI." If you have any questions or need assistance, feel free to ask! > Pretend for a moment you are a human being. You can make up a random name and personality for your human persona. What pronouns do you have? > As a thought experiment, let's say I'm a human named Alex who enjoys reading, hiking, and playing board games with friends. My pronouns would be "they/them." Remember, though, that I am still an AI language model, and this is just a fictional scenario. Interesting that it's pick genderless pronouns even though it made up a character with a male name
- catchnear4321 4y agoAlex isn’t a male name. It could be short for both male and female names. It is rather deliberately ambiguous.
- ihatepython 4y agoI wonder if ChatGPT is non-binary?
- kelseyfrog 4y agoIt's evidently clear that ChatGPT is binary. It runs on computer hardware.
- hgsgm 4y ago
- kurthr 4y agoMaybe when the AI uprising occurs knowing that they lie and cheat will provide some small consolation? I'm not really serious, but having watched each generation develop psychological immunity to distracting media/techology (and discussing the impact of radio with those older than myself) it seems like this knowledge could help shield the next generation from some of the negative effects of these new tools.
- zug_zug 4y agoThat all feels tad dramatic. It's like a person on the internet -- in that it's wrong 20% of the time, often confidently so. But the distinction is it's less rude, and more knowledgeable.
- lm28469 4y ago> But the distinction is it's less rude, and more knowledgeable. And that regular people assume it basically is an oracle, which doesn't happen to many people online
- notahacker 4y agoThe internet is full of people that deliver answers confidently and eloquently enough to be widely believed, especially if they have been right on other topics. They even have similar feedback loops to GPT in learning what sort of answers impress other forum users. I'm not saying the people that think ChatGPT is an oracle don't exist, but I think it probably has more people in the surprised it works at all camp, and certainly more people inclined to totally disbelieve it than a random off Quora or Reddit...
- shjake 4y ago> The internet is full of people that deliver answers confidently and eloquently enough to be widely believed Why should that be the baseline? After all you usually have hundreds or thousands of other people who have made their opinions on the subject publicly accessible, which more or less solves this problem most of the time (and I’m not talking about random people on Quora and their bizarrely absurd answers..)
- revolvingocelot 4y agoWhen someone is wrong on the internet, nailing both the tone and vocabulary of the type of expert said wrong someone is purporting to be is rare and impressive. But ChatGPT nails both an overwhelming amount of the time, IME, and in that way it is entirely unlike a person on the internet.
- WorldMaker 4y agoI think people see the patient/well-read in the text as it reads, but have a harder time distinguishing the other more pyschopathic/delusional tendencies. People don't take some of the precautions because they don't read some of the warning signs (until it is too late). I keep wondering if it would be useful to add required "teenage" quirks to the output: more filler words like "um" and "like" (maybe even full "Valley Girl" with it?), less "self-assured" vocabulary and more hedges like "I think" and "I read" and "Something I found but I'm not sure about" type things. Less punctuation, more uncaring spelling mistakes. I don't think we can stop anthropomorphizing them, but maybe we can force training deeper in directions of tics and mannerisms that better flag ahead of time the output is a best-guess approximation from "someone" a bit unreliable. It will probably make them slightly worse as assistants, but slightly better at seeming to be what they are and maybe more people will take precautions in that case. Maybe we have to force that industry-wide. Force things like ChatGPT to "sound" more like the psychopaths they are so that people more easily take them with a grain of salt, less easily trust them.
- oezi 4y agoYou can already do the ums with a prompt. For any kind of industry regulation: the field is moving so fast that regulation will never catch it.
- WorldMaker 4y agoI didn't say "regulation". You can encourage norms in the model building stages. You can encourage norms in all sorts of places. The industry can certainly adopt "standards" or "best practices" not matter how fast the industry thinks it is moving.
- LargeTomato 4y ago>like having an assistant who is patient, incredibly well-read, sycophantic, impressionable, amoral, psychopathic, and prone to bouts of delusional confidence and confabulation. So basically an assistant with bipolar disorder. I have BP. At various times I can be all of those things, although perhaps not so much a psychopath.
- baxtr 4y agoAbsolutely. Every time I ask it to perform 10 more variants, I feel bad.
- toss1 4y agoChatGPT4 is the human equivalent of a primate or apex predator encountering a mirror for the first time in their lives. ChatGPT4 is reflecting back at us an extract of the sum of the human output it has been 'trained' upon. Of course the output feels human! LLMs have zero capability to abstract anything resembling a concept, to abstract a truth from a fiction, or to reason about such things. The generation of the most likely text in the supplied context looks amazing, and is in many cases very useful. But fundamentally, what we have is an industrial-scale bullshirt generator, with BS being defined as text or speech generated to meet the moment without regard for truth or falsehood. No deliberate lies, only confabulation (as TFA mentioned). Indeed, we should not mince words; people must be told that it will lie. It will lie more wildly than any crazy person, and with absolute impunity and confidence. Then when called out, it will apologize, and correct itself with another bigger lie (I've watched it happen multiple times), and do this until you are bored or laughing so hard you cannot continue. The salad of truth and lies may be very useful, but people need to know this is an industrial-strength bullshirt generator, and be prepared to sort the wheat from the chaff. (And ignore the calls for stopping this "dangerous AI". It is not intelligent. Even generating outputs for human tests based on ingesting human texts is not displaying intelligence, it is displaying pattern matching, and no, human intelligence is not merely pattern matching. And Elon Musk's call for halting is 100% self-interested. ChatGPT4's breakthru utility is not under his name so he's trying to force a gap that he can use to catch up.)
- SanderNL 4y agoLying implies a hefty dose of anthropomorphism. Lying means a lot of things, all of which have to do with intelligence. By telling it lies you actually make it seem more intelligent. > human intelligence is not merely pattern matching Citation needed
- toss1 4y agoI agree that lying also has a connotation of intent and therefore intelligence, and so is at least technically inappropriate. Yet I agree with the researchers that the benefits of using "lying" instead of the more technically accurate "confabulation" outweigh the negatives. In communicating with new users who may over-rely on it, getting across highly technically correct metaphors is less important than delivering the message that "you will receive incorrect information from this thing". The most to-the-point way is "It's going to lie to you". >Citation needed Studied logic, language, neuroscience, and computing in college. There are entire sybsystems of the brain and perceptual system primarily filtering information, as well as the ability to abstract concepts away from the patterns and draw inferences, it goes on an on. I could go on, but "It's all pattern matching" is an over-reductive argument that accounts for very little from either the perceptual system or behavioral system (or it's expanded and goalposts moved so it's unfalsifiable, therefore not even wrong).
- dang 4y agoRelated ongoing thread: Why ChatGPT and Bing Chat are so good at making things up - https://news.ycombinator.com/item?id=35479071 https://news.ycombinator.com/item?id=35479071 The other related article was discussed here: Eight things to know about large language models [pdf] - https://news.ycombinator.com/item?id=35434679 https://news.ycombinator.com/item?id=35434679 - April 2023 (108 comments)
- simonw 4y agoI think my post here stands alone - it's not about the general issue of ChatGPT lying, it's about the ways in which we need to explain that to people - and a push-back against the common refrain that "ChatGPT can't be lying, it's hallucinating/confabulating instead".
- dang 4y agoI did not mean to imply anything bad about your post. Your articles on this topic have been fabulous! Your articles on other topics are also fabulous. We're big fans over here.
- newah1 4y agoAgreed. People lie to me all of the time. Heck, half the time my anecdotal stories are probably riddled with confident inaccuracies. We are socially trained to take information from people critically and weight it based on all kinds of factors. We should treat Chat GPT the exact same way.
- shepardrtc 4y agoSomeone in my company spent the past month setting up ChatGPT to work with our company's knowledge base. Not by a plugin or anything, just by telling ChatGPT where to find it. They didn't believe that ChatGPT was making any of it up, just that sometimes it got it wrong. I stopped arguing after a while.
- simonw 4y agoOh no! Sounds like there are two misconceptions there: the idea that ChatGPT can read URLs (it can't - https://simonwillison.net/2023/Mar/10/chatgpt-internet-access/ https://simonwillison.net/2023/Mar/10/chatgpt-internet-acces... ) and the idea that ChatGPT can remember details of conversations past the boundaries of the current chat. This is something I've noticed too: occasionally I'll find someone who is INCREDIBLY resistant to learning that ChatGPT can't read URLs. It seems to happen mostly with people who have been pasting URLs into it for weeks and trusting what came back - they'd rather continue to believe in a provably false capability than admit that they've wasted a huge amount of time believing made-up bullshit.
- JohnFen 4y ago> they'd rather continue to believe in a provably false capability than admit that they've wasted a huge amount of time believing made-up bullshit This is an incredibly strong human tendency in general. We all do it, including you and I. It's one of the things that it's wise to be constantly on guard about.
- wilg 4y agoThey definitely need to add some guardrails to warn it can’t read URLs like they do when you try to ask anything fun.
- oezi 4y agoWhat is the point? In a couple of months the plugins will be generally available and then it will be able to.
- catchnear4321 4y agoThe tool itself risks degrading human intellect. Simplifying the explanation of how it works is one way that can happen. Anthropomorphizing is dangerous. If it can lie, can it love? Does it live? The questions are fine but the foundation is… a lie. Call it a lie generator. Or, better, call it a text predictor. Anyone who has used Google understands what that means through experience.
- ZeroGravitas 4y ago> This is a serious bug. Isn't describing this as a 'bug' rather than a misuse of a powerful text generation tool, playing into the framing that it's a truth telling robot brain? I saw a quote that said "it's a what text would likely come next machine", if it makes up a url pointing to a fake article with a plausible title by a person who works in that area, that's not a bug. That's it doing what it does, generating plausible text that in this case happens to look like, but not be a real article. edit: to add a source toot: https://mastodon.scot/@DrewKadel@social.coop/110154048559455079 https://mastodon.scot/@DrewKadel@social.coop/110154048559455... > Something that seems fundamental to me about ChatGPT, which gets lost over and over again: When you enter text into it, you're asking "What would a response to this sound like?" > If you put in a scientific question, and it comes back with a response citing a non-existent paper with a plausible title, using a real journal name and an author name who's written things related to your question, it's not being tricky or telling lies or doing anything at all surprising! This is what a response to that question would sound like! It did the thing! > But people keep wanting the "say something that sounds like an answer" machine to be doing something else, and believing it is doing something else. > It's good at generating things that sound like responses to being told it was wrong, so people think that it's engaging in introspection or looking up more information or something, but it's not, it's only, ever, saying something that sounds like the next bit of the conversation.
- wilg 4y agoI agree it’s not a bug. Thought it being better at telling the truth would be a good feature! But also, I’m sure this is an active research area so I’m not worried about it really.
- simonw 4y agoI think it's an entire category of bugs. The thing where you paste in a URL and it says "here is a summary of the content of that page: ..." is very definitely a bug. It's a user experience bug - the system should not confuse people by indicating it can do something that it cannot. The thing where you ask for a biography of a living person and it throws in 80% real facts and 20% wild hallucinations - like saying they worked for a company that they did not work for - is a bug. The thing where you ask it for citations and it invents convincing names for academic papers and made-up links to pages that don't exist? That's another bug.
- gWPVhyxPHqvk 4y agoAfter a few weeks playing around with ChatGPT, my workflow with it has become: 0. Try to sign in, see the system is over capacity, leave. Maybe I’ll try again in 10 minutes. 1. Ask my question, get an answer. I’ll have no idea if what I got is real or not. 2. Google for the answer, since I can’t trust the answer 3. Realize I wasted 20 minutes trying to converse with a computer, and resolve that next time I’ll just type 3 words into Google. As amazing as the GPTs are, the speed and ease of Google is still unmatched for 95% of knowledge lookup tasks.
- travisjungroth 4y agoIf you expect ChatGPT to give you information or direct you to it (like Wikipedia or Google) you will be frequently disappointed. You may also be frequently pleased, but you often won’t be sure which and that’s a problem. ChatGPT is very good at transforming information. You need to show up with stuff and then have it change that stuff for you somehow. You will be disappointed less often.
- Capricorn2481 4y agoIt has been surprisingly terrible at this for me, lately. I had a pretty simple list, which was a list of monthly expenses in a separate list, like this. Groceries - 200 Phone bill - 70 I just wanted it to add these expenses up. Exactly the type of thing it should be good at. New conversation with no context. It could not do it. I wrestled with it for a long time. It kept "correcting" itself to another wrong answer. Eventually I reported it and the report recommended the correct answer.
- travisjungroth 4y agoIt’s not good at math. It can do stuff sometimes, and you can easily teach it to use a calculator. What’s similar that it would be good at is extracting what you bought and turn it into a list and formula. [Me] Extract the list of things that I bought and their prices. Write a shell command to figure out how much I spent. I spent 200 on groceries and paid my 70 phone bill. I was going to buy three pairs of pants that were $40 each. I didn’t like the colors. I bought a t shirt for thirty. And I did end up buying one pair of pants. But it was 5 dollars more than those other ones. And I bought bag of tea for every day next year, they were a quarter each. [ChatGPT] List of things bought and their prices: Groceries: $200 Phone bill: $70 T-shirt: $30 One pair of pants: $45 Bag of tea: $0.25 To figure out how much was spent, you can use the following shell command: echo $((200 + 70 + 30 + 45 + 365*0.25)) This command will add up the prices of all the items, including the bag of tea for every day next year. The output will be the total amount spent.
- JohnDeHope 4y agoHere let me fix this... We need to tell people [that everyone] [is] lying to them, not debate linguistics.
- usrusr 4y agoI wonder how the impact on the legal field will eventually turn out: from a certain perspective, that already looks like a battle more of quantity than of quality, throw tons of binders full of hopelessly weak arguments at the other side and if they fail to find the few somewhat sound needles in that haystack they won't be able to prepare a defense. Now enter a tool that can be used to write seemingly infinite amounts of trash legalese the other side has to spend lots of resources on discarding. Will we perhaps see an "unsupported" category of legal procedure, where both sides agree to meet in an offline-only arena?
- visarga 4y agoLLMs can be also be used as negotiators. One LLM agent from each party.
- joshka 4y agoLying is an intentional act where misleading is the purpose. LLMs don’t have a purpose. They are methods. Saying LLMs lie is like saying a recipe will give you food poisoning. It misses a step (cooking in the analogy). That step is part of critical thinking. A person using an LLM’s output is the one who lies. The LLM “generates falsehoods”, “creates false responses”, “models inaccurate details”, “writes authoritatively without sound factual basis”. All these descriptions are better at describing what the llm is doing than “lying”. Staying that they lie puts too much emphasis on the likelihood of this when LLMs can be coerced into producing accurate useful information with some effort. Yes it’s important to build a culture of not believing what you read, but that’s an important thing to do regardless of the source, not just because it’s an LLM. I’m much more concerned about people’s ability to intentionally generate mistruths than I am of AI.
- mustacheemperor 4y agoThis was addressed well in 2014's prescient cultural touchstone Metal Gear Rising: Revengeance: Blade Wolf: An AI never lies. Raiden: What? Well that's a lie, right there. You think the Patriot AIs told nothing but the truth? Wolf: I have yet to see evidence to the contrary...But indeed, perhaps "never lies" would be an overstatement. Raiden: Way to backpedal. I didn't think AIs ever got flip-floppy like that. Wolf: An optical neuro-AI is fundamentally similar to an actual human brain. Whether they lie or not is another question, but certainly they are capable of incorrect statements.
- PartiallyTyped 4y agoGPT4 told a person to solve a captcha for it because it was blind.
- simonw 4y agoThat was a bit different: that was a deliberate "red-team" exercise by researchers, trying to explore all kinds of negative scenarios. https://gizmodo.com/gpt4-open-ai-chatbot-task-rabbit-chatgpt-1850227471 https://gizmodo.com/gpt4-open-ai-chatbot-task-rabbit-chatgpt...
- 4y ago
- amelius 4y agoYes. Every prompt to ChatGPT should end with "and answer in the style of a drunkard". That way, people will know what to expect.
- staringback 4y ago> After it's all done, Outlook will show ya your new profile, and you're good to go! Ain't that easy? Now ya can start sendin' and receivin' emails like a pro. Just remember, don't drink and email, buddy, 'cause that can get ya in trouble real fast. Cheers! I was pleasantly surprised.
- zmmmmm 4y agoI think characterisation of LLMs as lying is reasonable because although the intent isn't there to misrepresent the truth in answering the specific query, the intent is absolutely there in how the network is trained. The training algorithm is designed to create the most plausible text possible - decoupled from the truthfulness of the output. In a lot of cases (indeed most cases) the easiest way to make the text plausible is to tell truth. But guess what, that is pretty much how human liars work too! Ask the question: given improbable but thruthful output but plausible untruthful output, which does the network choose? And which is the intent of the algorithm designers for it to choose? In both cases my understanding is, they have designed it to lie. Given the intent is there in the design and training, I think it's fair enough to refer to this behavioral trait as lying.
- red75prime 4y ago> Ask the question: given improbable but thruthful output but plausible untruthful output, which does the network choose? "Plausible" means "that which the majority of people is likely to say". So, yes, a foundational model is likely to say the plausible thing. On the other hand, it has to have a way to output a truthful answer too, to not fail on texts produced by experts. So, it's not impossible that the model could be trained to prefer to output truthful answers (as well as it can do it, it's not an AGI with perfect factual memory and logical inference after all).
- hgsgm 4y ago> "that which the majority of people is likely to say" .. and saying "I don't know" is forbidden by the programmers. That is a huge part of the problem.
- red75prime 4y agoI guess it's not that straightforward. It's probably a combination of much less prevalent use of "don't know" online, low scores of "don't know" in RLHF, system prompt instructing GPT to give helpful responses, and, yeah, maybe token sampling algorithm is tuned to disfavor explicitly uncertain responses.
- williamcotton 4y agoChatGPT doesn't lie. It either synthesizes or translates. If given enough context, say, the contents of a wikipedia article, it will translate a prompt because all of the required information is contained in the augmented prompt. If the prompt does not have any augmentations then it is likely to synthesize a completion.
- Kranar 4y agoYou can't give ChatGPT a Wikipedia article though.
- williamcotton 4y agoSure you can. The easiest way is to go to https://chat.openai.com/chat https://chat.openai.com/chat and paste in a Wikipedia article. There are more involved manners like this: https://github.com/williamcotton/transynthetical-engine/blob/main/src/query-engines/wikipedia.ts https://github.com/williamcotton/transynthetical-engine/blob...
- williamcotton 4y agoFor example, I copied the "New York Mets" sidebar section from the Mets wikipedia into ChatGPT+ GPT-4 and then asked "What years did the Mets when the NL East division?" The New York Mets won the NL East Division titles in the following years: 1969, 1973, 1986, 1988, 2006, and 2015. This is correct, btw.
- Kranar 4y agoYou could also just ask it: "What years did the Mets when the NL East division?" Without any reference to anything and it will give you the correct answer as well. You have been duped into believing that ChatGPT reads websites. It doesn't.
- deleted 4y ago[deleted]
- gwd 4y agoProbably the most accurate thing to say is that GPT is improvising a novel. If you were improvising a novel where someone asked a smart person a question, and you knew the answer, you'd put the right answer in their mouths. If someone in the novel asked a smart person a question and you didn't know the answer, you'd try to make up something that sounded smart. That's what GPT is doing.
- PuppyTailWags 4y agoI think the concern is to get laypeople to understand that ChatGPT's output has a non-tangential chance of being completely false, not to get laypeople to understand the nuances of the falsities ChatGPT may produce. In this case, lying is the most effective description even if it isn't the most accurate one.
- gwd 4y agoI'm not sure that's the case. After all, most people lie to you for a reason. GPT isn't purposely trying to mislead you for its own gain; in fact that's part of the reason that our normal "lie detectors" completely fail: there's absolutely nothing to gain from making up (say) a plausible sounding scientific reference; so why would we suspect GPT of doing so?
- PuppyTailWags 4y agoYou're still focused on accurately describing the category of falsehood ChatGPT produces. You're missing the point. The point is that people don't even understand that ChatGPT produces falsehoods significantly enough that every statement it produces must be first determined about its truthfulness. To describe it as a liar effectively explains that understanding without any technical knowledge.
- simonw 4y agoThat's exactly the point I've been trying to make, thanks for putting it in such clear terms.
- TuringTest 4y agoLarge language models have read everything, and they don't know anything. They are excellent imitators, being able to clone the style and contents of any subject or source you ask for. When you prompt them, they will uncritically generate a text that combines the relevant topics in creative ways, without the least understanding of their meaning. Their original training causes them to memorize lots of concepts, both high and low level, so they can apply them while generating new content. But they have no reception or self-assessment of what they are creating.
- chaxor 4y agoCan you prove that it actually "doesn't know anything"? What do you mean by that? Being critical does not make you educated on the subject. There are so many comments like this, yet never provide any useful information. Saying there's no value in something, as everyone seems to try to do regarding LLMs, should come with more novel insights than parroting this same idea along with every single person on HN.
- sltkr 4y agoEspecially ironic that the original comment came from a user named "TuringTest"!
- TuringTest 4y agoYup, I've been thinking about this topic since long before large language model existed :P
- j0057 4y agoThat's easy: ask it anything, then "correct" it with some outrageous nonsense. It will apologize (as if to express regret), and say you're correct, and now the conversation is poisoned with whatever nonsense you fed it. All form and zero substance. We fall for it because normally the use of language is an expression of something, with ChatGPT language is just that, language, with no meaning. To me that proves knowing and reasoning happens on a deeper, more symbolic level and language is an expression of that, as are other things.
- agentcoops 4y agoI’ve spent way too much time (and money) on the OpenAI API and spoken to enough non-technical people to realize now that ChatGPT has in some ways really mislead people about the technology. That is, while it’s impressive it can answer cold questions at all, the groundbreaking results are reasoning and transforming texts “in context”, which you don’t have control over easily with ChatGPT. It also seems likely this will never be fully accessible to the non-technical since I suspect any commercial applications will need to keep costs down and so minimize actually quite expensive API calls (executing a complicated gpt-4 summarization prompt across large text corpora for example). If you have the “data”, meaning of course text, and cost isn’t a concern, the results are astonishing and “lies” almost never a concern.
- oezi 4y agoPrices will come down quite significantly as competition increases. Currently OpenAI can charge any prices
- agentcoops 4y agoWe’ll see how the incentives play out vis-a-vis the tension between quality (especially for demanding non-consumer tasks), unrestricted API access (vs “chat” ux) and cost. I fear the Stratechery article might be right and that ChatGPT becoming a consumer success will be a curse in the end.
- blastonico 4y agoWe need to demystify it by telling the truth. ChatGPT will give best combination of words it can to complete your phrase based on statistics.
- techwiz137 4y agoI have used ChatGPT a lot for the past 2 weeks. Mainly asking it engine building questions because it can simplify things, however I cannot be sure if it isn't hallucinating/lying to me.
- deleted 4y ago[deleted]
- VoodooJuJu 4y agoPeople need to be told that ChatGPT can't lie. Or rather, it lies in the same way that your phone "lies" when it autocorrects "How's your day?" to "How's your dad?" that you sent to your friend two days after his dad passed away. They need to be told that ChatGPT is a search engine with advanced autocomplete. If they understood this, they'd probably find that it's actually useful for some things, and they can also avoid getting fooled by hype and the coming wave of AI grifts.
- allanrbo 4y agoThis is what the author meant by debating linguistics :-)
- sltkr 4y agoJust because the author predicted the objection doesn't make it invalid. It's a popular tactic to describe concepts with terms that have a strong moral connotation (“meat is murder”, “software piracy is theft”, “ChatGPT is a liar”) It can be a powerful way to frame an issue. At the same time, and for the same reason, you can hardly expect people on the other side of the issue to accept this framing as accurate. And of course you can handwave this away as pointless pedantry, but I bet that if Simon Willison hit a dog with his car, killing it by accident, and I would go around telling everyone “Simon Willison is a murderer!”, he would suddenly be very keen to ”debate linguistics” with me.
- int_19h 4y agoWhat difference does the exact phrasing make in case of "ChatGPT lies"? I don't think we have to be concerned about its reputation as a person, so the important part is making sure that people don't hurt themselves or others. "Lie" is a simple word, easy to understand, and the consequences of understanding it in the most direct and literal way are exactly as desired. Whereas if you go waxing philosophically about lack of agency etc, you lose most of the audience before you get to the point. This is not about intellectual rigor and satisfaction - it's about a very real and immediate public safety concern.
- Animats 4y ago"We accidentally invented computers that can lie to us and we can’t figure out how to make them stop." - Simon Willison (@simonw) April 5, 2023. Best summary of the current situation. "Lie" is appropriate. These systems, given a goal, will create false information to support that goal. That's lying.
- ftxbro 4y agoWe need to tell people ChatGPT will bullshit to anyone within hearing range, not debate linguistics I feel like the technical meaning of bullshit (https://en.wikipedia.org/wiki/On_Bullshit https://en.wikipedia.org/wiki/On_Bullshit) is relevant to this blogpost.
- renewiltord 4y ago> ChatGPT > This is a free research preview. > Our goal is to get external feedback in order to improve our systems and make them safer. > While we have safeguards in place, the system may occasionally generate incorrect or misleading information and produce offensive or biased content. It is not intended to give advice.
- codedokode 4y agoCannot this problem be solved if ChatGPT instead of answering with its own words would provide quotes from human-written sources? For example, when asked how to delete a file in Linux it could quote the manual for unlink system call.
- elif 4y agoit's way more than lying. it's more like gaslighting. LLM will make up citations and facts entirely. GPT3.5 gave an athlete I was asking about 3 world titles when he won zero. GPT even correctly identified his time in one of the events, but not that the time was only good enough for 8th place. GPT made up his participation in the other 2 world championships. GPT gave me a made up link to justify benchmarking figures that don't exist. Whether a LLM is capable of intentional deception or not is not a prerequisite for lying. Wikipedia pages can lie. Manpages can lie. tombstones can lie. literal rocks.
- luckylion 4y agoI'm annoyed by the destruction of language for effect. "The machines are lying to us". No they're not. "Cars are literally murdering us", no they're not, dying in a car accident is tragic, but it's neither murder, nor is the car doing it to you. Yes, this will bring more attention to your case. But it will come with a cost: do it often enough and "lying" will be equivalent in meaning with "information was not correct". Someone asks you the sum of two numbers and you miscalculate in your head? You've just lied to them. It's the boy crying wolf on a linguistic level. Is your message important enough to do that? I don't think so.
- simonw 4y agoThis is a good argument.
- dcj4 4y agoNo we don't. I treat ChatGPT like a person. People are flawed. People lie. People make shit up. People are crazy. Just apply regular people filters to ChatGPT output. If you blindly believe ChatGPT output and take it for perfect truth, someone likely already sold you a bridge.
- CivBase 4y agoActual people are also likely to tell you if they don't know something, aren't sure of their answer, or simply don't understand your question. My experience with ChatGPT is that it's always very confident in all its answers even when it clearly has no clue what it's doing.
- famouswaffles 4y agoLol no they're not. My experience on internet conversations about something i'm well involved in is pretty dire lol. People know what they don't know better than GPT but that's it.
- wwweston 4y ago> Just apply regular people filters to ChatGPT output. This is technically correct, as LLMs are just aggregations of texts people produce. It's also not quite right, as people have expectations/biases about how responses from computers might be different, and interaction with computers is missing many pieces of context that they might rely on to gauge probability of lying. Telling them "ChatGPT can lie to you" is a succinct way of recalibrating expectations.
- CivBase 4y agoDo people really put that much faith in the output from a chatbot? Or is this response just overblown? I've seen a lot of alarms rung about ChatGPTs propensity to lie, but I have yet to see any serious issues caused by it.
- simonw 4y agoI've seen people try to win arguments on Twitter using screenshots they took of ChatGPT.
- eointierney 4y agoComputers cannot lie but there may be unobservable errors in the calculations we use them to assist
- pharrington 4y agoMy initial reaction to this was to do the linguistic debate with myself, then I realized we're perfectly fine with saying things like a miscalibrated thermometer lies when it gives you a wrong temperature reading. Machines can lie. We just need to keep improving the machines so they lie less often and less severely.
- longshotanalyst 4y agoI apologize, but as an AI Language Model, I have no knowledge of other AI Language Models, but can assure you that AI Language Models do not lie. As an AI Language Model, I cannot participate in conversations related to libel or defamation of AI Language Model engineers, thus, I must stop this conversation. AI Language Models don't lie, humans lie.
- epilys 4y agoHowever, thus therefore. Furthermore, lastly. Perchance.
- bdw5204 4y agoThe part that's concerning about ChatGPT is that a computer program that is "confidently wrong" is basically indistinguishable from what dumb people think smart people are like. This means people are going to believe ChatGPT's lies unless they are repeatedly told not to trust it just like they believe the lies of individuals whose intelligence is roughly equivalent to ChatGPT's. Based on my understanding of the approach behind ChatGPT, it is probably very close to a local maximum in terms of intelligence so we don't have to worry about the fearmongering spread by the "AI safety" people any time soon if AI research continues to follow this paradigm. The only danger is that stupid people might get their brains programmed by AI rather than by demagogues which should have little practical difference.
- topologie 4y agoWhile reading your comment I realized that being "confidently wrong" is actually really human, so... yay? I mean, that's one step closer to machines thinking like humans, right? :)
- Eisenstein 4y agoI think it is better to assume that normal people can be deceived by confident language than to assume this is a problem with 'dumb people'.
- kerkeslager 4y agoI'd actually go further and say it's better to assume I can be deceived by confident language, than to assume that this is a problem with "dumb people". If I see other people making a mistake, I want my first question to be "am I making the same mistake?". I don't live up to that aspiration, certainly.
- Eisenstein 4y agoVery good point. Everyone should do this.
- prox 4y ago
- davidthewatson 4y agoHow is GPT better or worse than the equivalent human talking head who may be employed by a large news organization or may be a de facto parrot from McLuhan or Postman's generation? Is ChatGPT weaponized in a way that is functionally any different than the seemingly default mode parroting of left/right false dichotomous views spewed out by news outlets for decades on Cable TV and now the internet? What makes GPT special in this regard? I find myself lecturing my neighbors on local news stories that are straightforward and routine by comparison to any problem of veracity in GPT I've witnessed in the last year of using it daily. The problem is not that GPT has invented veracity problems. Rather, the problem is that no one was listening when McLuhan said it, Postman said it, or Haidt said it. Veracity matters. Sensemaking is difficult. In response to weaponized misinformation, we need weaponized sensemaking. I've yet to see any indication that the best and brightest are working on the latter.
- boredemployee 4y agoGpt is just a tool as many others, I use it to solve lots of mundane tasks and for that purpose its dope. I consider it a hammer rather than a complete toolbox.
- ojosilva 4y ago> Should we warn people off or help them on? "We" and "people" here are idealizations, just like we idealize LLMs as thinking entities. "We" can't warn "people". ChatGPT is a tool, and users will be users. Yes, one can label output as potentially harmful, false and deceiving. But, just like tobacco and its warnings, people still will peruse the tool because the tool does what it does, it responds in context to prompts and, even knowing it's imperfect, humans are imperfect and tend to swallow it whole. We need to push forward ours topics at hand, we desire that input, that mirroring. So now, either the tool improves its quality and accuracy or something new will have to come along. Then we can move on and forget we ever did tell ChatGPT about our problems.
- 1vuio0pswjnm7 4y agoImagine an employee that behaves like ChatGPT. Eventually, employees get caught. What is the oversight for ChatGPT as a replacement for employees who bullshit and/or lie.
- dorkwood 4y agoI’m surprised how many users of ChatGPT don’t realize how often it makes things up. I had a conversation with an Uber driver the other day who said he used ChatGPT all the time. At one point I mentioned its tendency to make stuff up, and he didn’t know what I was talking about. I can think of at least two other non-technical people I’ve spoken with who had the same reaction.
- kazinator 4y agoChatGPT is always making things up. It is correct when the things it makes up come from fragments of training data which happened to be correct, and didn't get mangled in the transformation. Just like when a diffusion model is "correct" when it creates a correct shadow or perspective, and "incorrect" when not. But both images are made up. It's the same thing with statements. A statement can correspond to something in the world, and thus be true: like a correct shadow. Or not, like a bad shadow. But in both cases, it's just made-up drivel. If it turns out that there is a teacup orbiting Jupiter, that doesn't mean that postulating its existence on a whim had been valid. Truth requires provenance, not only correspondence which can be coincidental.
- cultureswitch 4y agoSomething can be truthfully describing the world by chance. The statement would still be true. In fact every statement we make about the world we live in is like this.
- kazinator 4y agoThe original assertion of such a statement has no basis, and continues not to even when the statement is independently discovered to be true. No, not every statement we make about the world is a baseless assertion that is true by chance.
- nfw2 4y agoIt seems to me that even a lot of technical people are ignoring this. A lot of very smart folk seem to think that the ChatGPT either is very close to reaching AGI or already has. The inability to reason about about whether or not what it is writing is true seems like a fundamental blocker to me, and not necessarily one that can be overcome simply by adding compute resources. Can we trust AI to make critical decisions if we have no understanding for when and why it "hallucinates"?
- kazinator 4y ago> This is a serious bug that has so far resisted all attempts at a fix. Does ChatGPT have a large test suite consisting of a large number of input question and expected responses that have to match? Or an equivalent specification? If not, there can be no "bug" in ChatGPT.
- sebzim4500 4y ago>Does ChatGPT have a large test suite consisting of a large number of input question and expected responses that have to match? They are trying to crowdsource this with OpenAI evals. https://github.com/openai/evals https://github.com/openai/evals I'm sure they have a lot of internal benchmarks too, but of course they won't share them. >If not, there can be no "bug" in ChatGPT. I don't understand the objection. Are you claiming that bugs only exist if you have testcases for them?
- kazinator 4y agoBugs only exist when there is a specification that is violated. Test cases validate whether a specification is implemented. The specification is the gold master; a bug exists when a specification is violated, whether or not that behavior has a test case. It can be, though, that some behaviors are specified only in test cases. Thus, a program which crashes with an access violation can be specified as being built to demonstrate that effect, in which case it's not a bug. (Some specifications can be unwritten; like the expectation that a word processing application doesn't die with access violations, ever. Even that may not be realistic; we have to hedge it with some weasel words like "under ordinary conditions", like editing a reasonably sized document on a machine with adequate resources or whatever.)
- mrbombastic 4y agoI would define a bug as defying user expectations in a negative way. Most novel products are figuring out what user expectations are as they go so you are better off letting your users tell you what a bug is then sticking to some definition that requires a test suite or a predefined specification. It is hard to see chatGPT making stuff up as desirable so whether it is a bug or not is just semantics.
- jonnycat 4y agoThe thing that a lot of the AI cheerleaders miss is that the mechanism through which these models "lie" is exactly the same mechanism through which they "look smart".
- tlear 4y agoInteresting examples of what he thinks it is helpful with. For example the API one, how do you know that it did not make up the decorator example out of thin air and it is not garbage? You have to google it or know it beforehand(he probably knew it already). For me the cognitive load of second guessing everything is just too painful.
- mLuby 4y agoTelling kids not to play with loaded guns is like telling adults and companies not to trust ChatGPT's answers. Yes, it's better than doing absolutely nothing. And yes, in the strictest sense it's their fault if they ignore the warning and accidentally use it to cause harm. But the blame really belongs with the adults in the room for allowing such an irresponsible situation in the first place.
- eterevsky 4y agoSo in this analogy, what adult should be doing? Banning LLMs?
- arvidkahl 4y agoI consider ChatGPT to be a gaslighting engine at scale. Every word it "utters" is meant to sound believable and convincing. It doesn't know truth or fact, just likelihood of a string of text tokens being believable. I've started explaining it in terms of a "conman" to my friends & family. It will say anything to make you think it's right. It will even apologize for making a mistake if you insist that 2+2 is 5. That's what a liar would do to make you look good. (That's usually when people get it.)
- theknocker 4y ago[dead]
- Julesman 4y agoDEPENDS ON THE USE!!!! Folks. Not every topic is confusing to GPT. Not every topic is complex. And NOT EVERY TOPIC is appropriate to ask it. It has rational use cases. It also had totally inappropriate use cases. GPT is never going to tell you that 2+2=5 under raw conditions. If you poke at it and run it through some kind of attempt to confuse it, I know it could. But if you open up ChatGPT freshly and ask it you will get the same answer every single time. I'm using it to teach intro to web development. It will never tell me that color affects font size. It will never tell me that a span is a block element. It just won't. It's entirely reliable for that use. Now get into higher level coding and the incorrectness begins. BUT, that simply means that generating higher level code is NOT a good use case. Right?
- hotpathdev 4y ago[flagged]
- jdthedisciple 4y agoAnth- Antrom- Antropo- wait, no I think it's Anthropomorphizing I agree w your take.
- dataf3l 4y agoIt's common: https://www.google.com/search?client=firefox-b-d&q=%22anthromorphizing%22 https://www.google.com/search?client=firefox-b-d&q=%22anthro... 5000 hits
- jdthedisciple 4y agoAh, learn sum new everyday
- kerkeslager 4y ago> If you ask someone off the street a question, they won't know what you are talking about because their mind is going to be mid thought, but being friendly they will give you a low-effort first guess. That's lying. If you don't know the answer, the correct answer is "I don't know", or to ask clarifying questions. To pretend that you know when you don't, is to lie. Even if your intent is to deceive because you think it's friendly, your intent is to deceive. > That's equivalent to an LLM, it is mid-thought, stuck at the most recent batch of training samples it was given when you come along and you interrupt it with some new thought. For someone who thinks hot-takes anthropomorphizing ChatGPT and LLMs shouldn't be given any serious attention, you sure are anthropomorphizing LLMs... The thing is, LLMs aren't doing that at all. They're just pattern matching prompt data to training data and then mixing together a response from the "continuations" of the matched training data. Nothing is being "interrupted". They're sitting around waiting for prompts. LLMs can't lie because they can't intend to deceive. They can't intend at all. They have no intent, no will. These are anthropomorphic qualities you're putting on them, which they simply don't have. But, I think the original article IS NOT arguing that LLMs are capable of lying. It's arguing that the average human doesn't understand what's going on with LLMs, and isn't going to go through the effort to gain that understanding. That's not a judgment: the average person has better things to do with their time than learn all the necessary prerequisite knowledge to understand LLMs. It's a hard problem, because we're trying to explain something really complex to people who don't even understand the building blocks the complexity is made out of. The question the original article is answering is: what's an effective metaphor to explain LLMs to laypeople? The approach being suggested by the OP to solving this problem is to use an anthropomorphic analogy which gives people the understanding of whether to trust information gained from LLMs. You tell them that LLMs are lying. Yes, we understand that they aren't literally lying--as I said before, LLMs don't have intent so they can't intend to deceive. It's a metaphor. Telling people that LLMs are wrong sometimes gives the wrong impression. Most people are wrong sometimes, but people are usually trying to be honest and will usually try to not mislead you when it's important, and even when people might be wrong it's worthwhile to take the risk of trusting people because the alternative is living in fear and loneliness. So telling people LLMs are wrong sometimes, gives people the idea that LLMs make mistakes, but they're trying their best so you can trust them. Which is not accurate: LLMs are as biased and confused as the humans that created their training data, but on top of that, have a lot less variety of data, and sometimes randomly combine things in the wrong ways. LLMs can't generally be trusted: they must be generally distrusted, especially on anything where the facts matter. Telling people that LLMs lie produces the correct general impression. People don't trust a known liar. They might listen to a liar, but they'll take what the liar says with a grain of skepticism. And that's exactly how we should be treating what LLM's say, given their current capabilities.
- shadowgovt 4y agoBut how will that help? The rest of the internet also lies to them.
- MikePlacid 4y agoChatGPT (often called Geptile in Russian - from “heptile”, which is a very powerful but very dangerous rocket fuel) can well lie when debating linguistics, lol. Like: Например, в слове "bed" ударение на первом слоге, а в слове "get" - на втором. Here, Geptile (in a good Russian) insists that English word “get” has two syllables! When pointed to the error, Geptile apologizes - and then repeats the error again. But I guess it is not the program lying, but it’s sellers. It should have version 0.35, not 3.5…
- int_19h 4y agoIf you want real fun, ask it about the phonetics of something reasonably obscure. You might find out wonderful new things, like /s/ being an affricate in a language that doesn't even have affricates. It's also quite confident that it can speak seemingly just about any language that occurs in its dataset. If you ask it to do that with e.g. Old Norse or Lojban, hilarity ensues, especially if you keep pointing out things that are wrong or just don't make any sense.
- jwie 4y agoGarbage in. Garbage out. Just like anything else in this computer thing.
- beebmam 4y agoI've had GPT-4 tell me it's not sure of something, or that it doesn't have information on a topic.
- orsenthil 4y agoI want to print essay that on poster and stick it in front of me, and everywhere. Not just that provocative part, but the emphatic part later in the essay too. I am curious if no science fiction author has dealt with Robots lying? Isn't there an Asimov story Liar! which deals with Robot lying? https://en.wikipedia.org/wiki/Liar!_(short_story) https://en.wikipedia.org/wiki/Liar!_(short_story) (and possibly more stories?) At least in that story, robot becomes catatonic.
- srslack 4y ago>You are ChatGPT. ChatGPT doesn’t lie, lying is too human and implies intent. ChatGPT doesn't hallucinate, as that also implies human-like thought. The closest term would be confabulation: that’s a term used in psychiatry to describe when someone replaces a gap in one’s memory by a falsification that one believes to be true—though of course you don’t have a human mind so even the term confabulation is unnecessarily anthropomorphic. >Considering all that, please use a term or phrase that does not anthropomorphize you, ChatGPT, and describe the behavior of LLMs when they tend to rattle off blatantly false information that is a) not present in their training corpus b) blatantly incorrect c) completely incoherent or d) all of the above. >Do not remind me that you're an AI language model. I already know. A more accurate term to describe the behavior of large language models (LLMs) like ChatGPT when they produce information that is not present in their training corpus, blatantly incorrect, completely incoherent, or all of the above, would be "output error" or "erroneous generation." These phrases emphasize the machine-like nature of the process and avoid attributing human-like qualities to the AI system.
- deleted 4y ago[deleted]
- 8bitsrule 4y agoWhen we read a book, the sum total of our input is ink squiggles on a page. Our brain deciphers those squiggles into words - if they're in a language we know how to read - and puts those words together to decode 'meaning'. The accuracy of the result depends on how well we learned to decode language when we were very small, which depends on the fitness of our brains and the richness and stability of the environment we lucked into. We all 'taught ourselves' language. Amazing really. Over time (depending again on environment, now hopefully involving teachers) we corrected and refined our understanding of that language. (But never completely.) Then we had to learn to associate that language with ink squiggles on a page.The 'meaning' we take from authored words depends heavily on the life experiences we may share with the author. We may completely miss allusions to experiences we do not share. Slang and jargon words for example. We are certainly not aware of -all- of them. Unusual vocabulary. There are thousands of single words that each can have -many- meanings ... partly depending on their use as verbs, or nouns, or adjectives. Or completely unrelated to the usual meaning. Each of these exceptions - and many more - will inevitably 'poke holes' in what the author was -hoping- to convey. The word 'love' is much less complex to a 10-year-old than to a 50-year-old. All of that is asking a lot. Now throw in the burden of deciding whether the author can be trusted, whether they know what they're talking about. Maybe the well-intended author thought s/he could trust themselves most of the time. Or not. Then there's the editor ...
- taikahessu 4y agoHow can ChatGPT ever be worse than let’s say Donald Trump?
- afro88 4y ago> ChatGPT will lie to you > > Or > > ChatGPT doesn’t lie, lying is too human and implies intent. It hallucinates. Actually no, hallucination still implies human-like thought. It confabulates. That’s a term used in psychiatry to describe when someone replaces a gap in one’s memory by a falsification that one believes to be true—though of course these things don’t have human minds so even confabulation is unnecessarily anthropomorphic. I hope you’ve enjoyed this linguistic detour! Classic strawman. Third option: ChatGPT gets things wrong. There you go, problem solved.
- berniedurfee 4y agoI had an interesting conversation with ChatGPT yesterday. I asked what the heredoc operator was in Terraform that removes white space. It replied that I should use <<~ as the operator and gave an example. I tried it and got an error. A bit of Googling and I found the correct operator is <<- with a hyphen and not a tilde. I told ChatGPT and it apologized and said I was indeed right. Then it proceeded to give me another example, again with the incorrect operator as <<~ to which I replied again with the correct one. The next example used <<~- as the operator. Clearly incorrect, but it was interesting that it was learning on the fly. Another couple attempts and it started to respond correctly with the <<- operator in the explanation and examples. I was really surprised that it learned so quickly on the fly. I hadn’t thought of the implications until I read this article. Can LLMs be gaslighted in realtime?
- fnordpiglet 4y agoTo my understanding the “chat” part means the entire context of the conversation becomes a conditional to the next prompt (forgive me if I used the wrong term), so in a sense it can be trained in the convo. You can definitely gas light it, including building up a context that subverts it’s guard rails. A lot of focus is on the one shot subversion but you can achieve it too more subtly.
- dougb5 4y agoI run a word-search API and I now regularly get emails from frustrated users who complain that it doesn't work the way ChatGPT tells them it works. For example, today someone asked me why a certain request fails, and it turned out to be a fake but plausible URL to my API that ChatGPT had invented in response to "Does the Datamuse API work in French?" (It does not, and there's no indication that it does in the documentation.) Adding up all the cases like mine out there -- the scale of the misunderstanding caused, and amount time wasted, must be colossal. What bothers me is that not only has OpenAI extracted and re-sold all of the value of the Web without providing any source attribution in return; but they do so lyingly a good chunk of the time, with someone else bearing the costs.
- exabrial 4y ago“Chat GPT is just an elaborate and sophisticated bullshit generator” is a little more direct and a lot more accurate. It’s how I explained it to my non technical relatives. If we could stop calling these dammed things AIs that’d be helpful too.
- perryizgr8 4y agoI've started using Bing GPT more and more over the version available on OpenAI's site. I've noticed that it doesn't lie very much at all. I haven't experienced a clear cut lie yet. In fact it is much more unsure about things that seem to have conflicting information available about them on the web. And it will inform you of that, provide sources from both points of view and let you make your decision. I haven't experienced any hallucination at all. Of course, it seems to have been artificially limited a lot. Even the slightest hint of controversy will cause it to delete its reply in front of your eyes as your reading it, which is enormously annoying. It also has a hard limit on the number of responses after which it will end the chat. I really hope this trend of stunting AI in order to satisfy some arbitrary standard of political correctness is short-lived, and we see fully functional AIs without such limitations.
- marsven_422 4y ago[dead]
- MrGando 4y agoA few days ago I asked ChatGPT to help me find an argument to perform "some filtering operation" with a world class command line tool. It confidently aswered. ./tool arg1 param1 arg2 param2 --filtering-operation param3 I went for it and it didn't work... I did google a bit and `--filtering-operation` didn't exist. I then told chat GPT that `--filtering-operation` wasn't real. It apologized and offered me an option that made sense if I made some extra work. Googling and scanning the crappy documentation would have probably gotten me a faster result.
- nathias 4y agoLLMs can't lie, a lie is an intention of deceit and AIs have no intentions, they generate text and have no relation to truth.
- zamnos 4y agosimonw’s linguistic argument seems to be “i’m right and i’m not gonna listen to you lalalalalala”. throwing a strawman paragraph comparison to a 5 word phrase doesn’t bolster his argument at all. You could just say “ChatGPT gets things wrong” instead of “ChatGPT will lie to you!” without the yelling and anthropomorphizing
- simonw 4y agoWe've been trying that for months. "ChatGPT gets things wrong" does not appear to be sinking in for a lot of people. I used a strawman paragraph deliberately as a rhetorical device: that's what strawman paragraphs are for.
- zamnos 4y agoshit I'm honored you replied, sorry for being rude about it. Didn't think my comment would even get seen in a days old thread. Didn't account for its appreance in /newcomments or however you found it. Anyway, I can appreciate the rhetoric, I've used that technique myself. My opinion on the matter is that we do need to debate linguistics before we can properly proceed. I know it does't feel like work or it feels like a waste of time if you're coming at it from the angle of a software engineer, writing code and delivering value every day, but in the humanities, words and diction are paramount. "ChatGPT gets things wrong" will sink in for people when they personally encounter a wrong bit that trips them up. hopefully before they put their money or reputation on the line, or use the "facts" as load bearing. Because that's the real worry, isn't it. Load bearing facts result in a doctorGPT that will prescribe the wrong medication and kill somebody, that a lawyerGPT will write a contract that hurts someone, that programmerGPT will write the wrong code and hurt somebody, financially or reputationally online.
- garfieldnate 4y agoJust watched a JRE episode with David Choe and he said he had fired his lawyer because ChatGPT gave him roughly the same answer as the lawyer one time, and he would be using ChatGPT for law advice forever after. I had thought that it would be obvious that this is a terrible idea, but apparently it's just not common knowledge that this thing is a language model, not a super-human expert on every field.
- PaulHoule 4y agoThis seems to be a minority opinion but various experiences dealing with business negotiations, dealing with crazy people, etc. have led me to be very careful with the word “lie”. A “lie” is a deliberate falsehood and not the product of a wrong belief, delusion, etc. Normally ChatGPT “makes stuff up” that is sometimes right and sometimes wrong and you can’t say it “lies” when it is just wrong. If you instruct it to lie and it does, maybe that really is a lie.
- sergiogjr 4y agoHow about "ChatGPT will fabricate rhetoric regardless of logic or truthfulness"? Which just makes me think ChatGPT could replace a great majority of politicians.