4 ms·

Lying implies a hefty dose of anthropomorphism. Lying means a lot of things, all of which have to do with intelligence. By telling it lies you actually make it

Lying implies a hefty dose of anthropomorphism. Lying means a lot of things, all of which have to do with intelligence. By telling it lies you actually make it seem more intelligent. > human intelligence is not merely pattern matching Citation needed

toss1 4y ago
I agree that lying also has a connotation of intent and therefore intelligence, and so is at least technically inappropriate. Yet I agree with the researchers that the benefits of using "lying" instead of the more technically accurate "confabulation" outweigh the negatives. In communicating with new users who may over-rely on it, getting across highly technically correct metaphors is less important than delivering the message that "you will receive incorrect information from this thing". The most to-the-point way is "It's going to lie to you". >Citation needed Studied logic, language, neuroscience, and computing in college. There are entire sybsystems of the brain and perceptual system primarily filtering information, as well as the ability to abstract concepts away from the patterns and draw inferences, it goes on an on. I could go on, but "It's all pattern matching" is an over-reductive argument that accounts for very little from either the perceptual system or behavioral system (or it's expanded and goalposts moved so it's unfalsifiable, therefore not even wrong).
- SanderNL 4y ago
  What’s wrong with: “it is often wrong”? I - or GPT - can think of various variations that do not have links with “lying”. I think it is a bad idea. Let people use it and see for themselves. It has unique failure modes, not unlike every other tool we have. You will only learn by interacting with it. Some people are in the habit of “protecting the people” but I am not quite sure that is necessary.
  - toss1 4y ago
    >>What’s wrong with: “it is often wrong”? Good q; the problem is the passive voice The passive voice connotes far less urgency and importance to any warning. Even "It is often dangerously wrong.", is still a more passive phrasing than "It will lie to you, with a highly confidant tone." I also cringe a bit at using active verbs that imply agency and intent in a statistical model; it makes an inaccurate implication about the technology. But the same applies whether we call it "lying" or "confabulating". I also do not see it as 'paternalistically protecting the user', but simply as 'truth in advertising': - accurately representing the strengths and weaknesses of the product. I absolutely agree that everyone should try it themselves - there are MANY useful use-cases. But they should simply be accurately forewarned, in no uncertain terms, about it's behavior.
    - SanderNL 4y ago
      May I ask why? How is this not “paternalistic”? Why protect the people, from what? Is this worse than what they are already exposed to on a daily basis? Does facebook say it fucks you up and makes you an addict? Why this urge to enforce “truth in advertising”? I am not saying you shouldn’t do it, it’s just that I do not get why and the article doesn’t mention it. It is somehow a given that The People need to be protected from this lying machine because they will revert to cannibalism if not properly informed.
      - toss1 4y ago
        >>How is this not “paternalistic”? I see three categories of handling this issue, perhaps call them "Paternalistic", "Responsible", and "Anti-Social". - Paternalistic would be like: "We assess [thing] to be dangerous in a number of ways and so we forbid you to use [thing], except by using our high priesthood representatives as intermediaries". - Responsible would be like: We assess [thing] to be dangerous in a number of ways, so we clearly inform you that [thing] is dangerous, specify the risks, and how to avoid or mitigate them. You are then free to use [thing] as you see fit. (Note: if using [thing] badly causes public as well as private risks, it is responsible to require training, tests, and licensing to use [thing] in public). - Anti-Social would be like: We know [thing] is dangerous, but we DGAF what happens, you should figure it out all by yourself, and if you have a problem, piss-off. Examples: giving a CRT monitor to a novice hardware hacker who has only ever seen LED monitors and not telling her in detail about the high-voltage capacitor hazards inside or how to deal with them. Giving a gun to someone without the basic rules (always treat it as if it's loaded, never point it at anything you do not intend to destroy, trigger finger discipline, etc.). A mere "Hey, watch it with that thing" is inadequate, and you'd rightly bear responsibility when they electrocute themselves, shoot themselves in the foot, etc. A decent warning around LLMs could be: "Be warned: while this system can produce amazing and useful results it will frequently lie to you, and with great confidence. Be sure to check all results independently. Do NOT USE IT for any life-critical or potentially life-changing decisions. E.g., Do NOT use it as a sole source for medical diagnoses, as it may give a misdiagnosis that would kill you [0]. Do NOT use it as a sole source for therapy, as it may counsel you to suicide [1]. These are real harms, do NOT use this as the sole source of any answers." - - - Then, let them loose on it. They know about the footguns, and have been warned Anything less borders on anti-social. >> because they will revert to cannibalism if not properly informed It is not that they'll revert to cannibalism if not properly informed, it is that there are potential and serious harms that are easily avoided if they are told. The civilized thing to do when there is a bridge out ahead leading the road into raging floodwaters, is to tell your fellow travelers before they get there, not merely expect that every one of them will notice in time and handle it well. What is a given among civilized people is that if we know something has hazards, we give warnings and information to our fellow travelers. That's not paternalistic. Paternalistic is to not give warnings, but to forbid use. And it is uncivilized and anti-social to fail to take the trouble to give full and straight information. Informed consent is a good thing. Handing somebody what is effectively a booby-trap without telling them is not. [0] https://inflecthealth.medium.com/im-an-er-doctor-here-s-what-i-found-when-i-asked-chatgpt-to-diagnose-my-patients-7829c375a9da https://inflecthealth.medium.com/im-an-er-doctor-here-s-what... [1] https://interestingengineering.com/culture/belgian-woman-blames-chatgpt-like-chatbot-eliza-for-her-husbands-suicide https://interestingengineering.com/culture/belgian-woman-bla...

toss1 4y ago

I agree that lying also has a connotation of intent and therefore intelligence, and so is at least technically inappropriate. Yet I agree with the researchers that the benefits of using "lying" instead of the more technically accurate "confabulation" outweigh the negatives. In communicating with new users who may over-rely on it, getting across highly technically correct metaphors is less important than delivering the message that "you will receive incorrect information from this thing". The most to-the-point way is "It's going to lie to you". >Citation needed Studied logic, language, neuroscience, and computing in college. There are entire sybsystems of the brain and perceptual system primarily filtering information, as well as the ability to abstract concepts away from the patterns and draw inferences, it goes on an on. I could go on, but "It's all pattern matching" is an over-reductive argument that accounts for very little from either the perceptual system or behavioral system (or it's expanded and goalposts moved so it's unfalsifiable, therefore not even wrong).

SanderNL 4y ago
What’s wrong with: “it is often wrong”? I - or GPT - can think of various variations that do not have links with “lying”. I think it is a bad idea. Let people use it and see for themselves. It has unique failure modes, not unlike every other tool we have. You will only learn by interacting with it. Some people are in the habit of “protecting the people” but I am not quite sure that is necessary.
- toss1 4y ago
  >>What’s wrong with: “it is often wrong”? Good q; the problem is the passive voice The passive voice connotes far less urgency and importance to any warning. Even "It is often dangerously wrong.", is still a more passive phrasing than "It will lie to you, with a highly confidant tone." I also cringe a bit at using active verbs that imply agency and intent in a statistical model; it makes an inaccurate implication about the technology. But the same applies whether we call it "lying" or "confabulating". I also do not see it as 'paternalistically protecting the user', but simply as 'truth in advertising': - accurately representing the strengths and weaknesses of the product. I absolutely agree that everyone should try it themselves - there are MANY useful use-cases. But they should simply be accurately forewarned, in no uncertain terms, about it's behavior.
  - SanderNL 4y ago
    May I ask why? How is this not “paternalistic”? Why protect the people, from what? Is this worse than what they are already exposed to on a daily basis? Does facebook say it fucks you up and makes you an addict? Why this urge to enforce “truth in advertising”? I am not saying you shouldn’t do it, it’s just that I do not get why and the article doesn’t mention it. It is somehow a given that The People need to be protected from this lying machine because they will revert to cannibalism if not properly informed.
    - toss1 4y ago
      >>How is this not “paternalistic”? I see three categories of handling this issue, perhaps call them "Paternalistic", "Responsible", and "Anti-Social". - Paternalistic would be like: "We assess [thing] to be dangerous in a number of ways and so we forbid you to use [thing], except by using our high priesthood representatives as intermediaries". - Responsible would be like: We assess [thing] to be dangerous in a number of ways, so we clearly inform you that [thing] is dangerous, specify the risks, and how to avoid or mitigate them. You are then free to use [thing] as you see fit. (Note: if using [thing] badly causes public as well as private risks, it is responsible to require training, tests, and licensing to use [thing] in public). - Anti-Social would be like: We know [thing] is dangerous, but we DGAF what happens, you should figure it out all by yourself, and if you have a problem, piss-off. Examples: giving a CRT monitor to a novice hardware hacker who has only ever seen LED monitors and not telling her in detail about the high-voltage capacitor hazards inside or how to deal with them. Giving a gun to someone without the basic rules (always treat it as if it's loaded, never point it at anything you do not intend to destroy, trigger finger discipline, etc.). A mere "Hey, watch it with that thing" is inadequate, and you'd rightly bear responsibility when they electrocute themselves, shoot themselves in the foot, etc. A decent warning around LLMs could be: "Be warned: while this system can produce amazing and useful results it will frequently lie to you, and with great confidence. Be sure to check all results independently. Do NOT USE IT for any life-critical or potentially life-changing decisions. E.g., Do NOT use it as a sole source for medical diagnoses, as it may give a misdiagnosis that would kill you [0]. Do NOT use it as a sole source for therapy, as it may counsel you to suicide [1]. These are real harms, do NOT use this as the sole source of any answers." - - - Then, let them loose on it. They know about the footguns, and have been warned Anything less borders on anti-social. >> because they will revert to cannibalism if not properly informed It is not that they'll revert to cannibalism if not properly informed, it is that there are potential and serious harms that are easily avoided if they are told. The civilized thing to do when there is a bridge out ahead leading the road into raging floodwaters, is to tell your fellow travelers before they get there, not merely expect that every one of them will notice in time and handle it well. What is a given among civilized people is that if we know something has hazards, we give warnings and information to our fellow travelers. That's not paternalistic. Paternalistic is to not give warnings, but to forbid use. And it is uncivilized and anti-social to fail to take the trouble to give full and straight information. Informed consent is a good thing. Handing somebody what is effectively a booby-trap without telling them is not. [0] https://inflecthealth.medium.com/im-an-er-doctor-here-s-what-i-found-when-i-asked-chatgpt-to-diagnose-my-patients-7829c375a9da https://inflecthealth.medium.com/im-an-er-doctor-here-s-what... [1] https://interestingengineering.com/culture/belgian-woman-blames-chatgpt-like-chatbot-eliza-for-her-husbands-suicide https://interestingengineering.com/culture/belgian-woman-bla...