4 ms·

What’s wrong with: “it is often wrong”? I - or GPT - can think of various variations that do not have links with “lying”. I think it is a bad idea. Let people

What’s wrong with: “it is often wrong”? I - or GPT - can think of various variations that do not have links with “lying”. I think it is a bad idea. Let people use it and see for themselves. It has unique failure modes, not unlike every other tool we have. You will only learn by interacting with it. Some people are in the habit of “protecting the people” but I am not quite sure that is necessary.

toss1 4y ago
>>What’s wrong with: “it is often wrong”? Good q; the problem is the passive voice The passive voice connotes far less urgency and importance to any warning. Even "It is often dangerously wrong.", is still a more passive phrasing than "It will lie to you, with a highly confidant tone." I also cringe a bit at using active verbs that imply agency and intent in a statistical model; it makes an inaccurate implication about the technology. But the same applies whether we call it "lying" or "confabulating". I also do not see it as 'paternalistically protecting the user', but simply as 'truth in advertising': - accurately representing the strengths and weaknesses of the product. I absolutely agree that everyone should try it themselves - there are MANY useful use-cases. But they should simply be accurately forewarned, in no uncertain terms, about it's behavior.
- SanderNL 4y ago
  May I ask why? How is this not “paternalistic”? Why protect the people, from what? Is this worse than what they are already exposed to on a daily basis? Does facebook say it fucks you up and makes you an addict? Why this urge to enforce “truth in advertising”? I am not saying you shouldn’t do it, it’s just that I do not get why and the article doesn’t mention it. It is somehow a given that The People need to be protected from this lying machine because they will revert to cannibalism if not properly informed.
  - toss1 4y ago
    >>How is this not “paternalistic”? I see three categories of handling this issue, perhaps call them "Paternalistic", "Responsible", and "Anti-Social". - Paternalistic would be like: "We assess [thing] to be dangerous in a number of ways and so we forbid you to use [thing], except by using our high priesthood representatives as intermediaries". - Responsible would be like: We assess [thing] to be dangerous in a number of ways, so we clearly inform you that [thing] is dangerous, specify the risks, and how to avoid or mitigate them. You are then free to use [thing] as you see fit. (Note: if using [thing] badly causes public as well as private risks, it is responsible to require training, tests, and licensing to use [thing] in public). - Anti-Social would be like: We know [thing] is dangerous, but we DGAF what happens, you should figure it out all by yourself, and if you have a problem, piss-off. Examples: giving a CRT monitor to a novice hardware hacker who has only ever seen LED monitors and not telling her in detail about the high-voltage capacitor hazards inside or how to deal with them. Giving a gun to someone without the basic rules (always treat it as if it's loaded, never point it at anything you do not intend to destroy, trigger finger discipline, etc.). A mere "Hey, watch it with that thing" is inadequate, and you'd rightly bear responsibility when they electrocute themselves, shoot themselves in the foot, etc. A decent warning around LLMs could be: "Be warned: while this system can produce amazing and useful results it will frequently lie to you, and with great confidence. Be sure to check all results independently. Do NOT USE IT for any life-critical or potentially life-changing decisions. E.g., Do NOT use it as a sole source for medical diagnoses, as it may give a misdiagnosis that would kill you [0]. Do NOT use it as a sole source for therapy, as it may counsel you to suicide [1]. These are real harms, do NOT use this as the sole source of any answers." - - - Then, let them loose on it. They know about the footguns, and have been warned Anything less borders on anti-social. >> because they will revert to cannibalism if not properly informed It is not that they'll revert to cannibalism if not properly informed, it is that there are potential and serious harms that are easily avoided if they are told. The civilized thing to do when there is a bridge out ahead leading the road into raging floodwaters, is to tell your fellow travelers before they get there, not merely expect that every one of them will notice in time and handle it well. What is a given among civilized people is that if we know something has hazards, we give warnings and information to our fellow travelers. That's not paternalistic. Paternalistic is to not give warnings, but to forbid use. And it is uncivilized and anti-social to fail to take the trouble to give full and straight information. Informed consent is a good thing. Handing somebody what is effectively a booby-trap without telling them is not. [0] https://inflecthealth.medium.com/im-an-er-doctor-here-s-what-i-found-when-i-asked-chatgpt-to-diagnose-my-patients-7829c375a9da https://inflecthealth.medium.com/im-an-er-doctor-here-s-what... [1] https://interestingengineering.com/culture/belgian-woman-blames-chatgpt-like-chatbot-eliza-for-her-husbands-suicide https://interestingengineering.com/culture/belgian-woman-bla...
    - SanderNL 4y ago
      Interesting. Thank you for the detailed response. Paternalistic is indeed not the right term and it’s nice to worry about your fellow humans to this degree. I guess I am anti-social. But I disagree on the premise that this is comparable to a dangerous boobytrap. It’s not completely without danger of course, but that’s a property it shares with kitchen utensils, cars, scissors, religious texts.. Are we going to preface the Bible with this kind of warnings too? I’m all for it, but be consistent. I also think that warnings like that are cheap ways to evade liability and do not actually solve the issue which is people being stupid. Your example is not about people being deliberate and thinking deeply about their actions based on given information. If people kill themselves after talking to a chatbot I do not think a textual warning would have sufficed. > it is that there are potential and serious harms that are easily avoided if they are told Let’s agree to disagree. Sorry for sounding obtuse. I enjoyed your input, makes me think. I’m just a classic annoying neckbeard.
      - toss1 4y ago
        >>Are we going to preface the Bible with this kind of warnings too? I’m all for it, but be consistent. Outstanding idea!! >>warnings like that are cheap ways to evade liability and do not actually solve the issue which is people being stupid. Yes, the all-too-common generic disclaimers and warnings to cover asses for liability are usually so broad and vague as to be useless. And generally with ordinary well-tested consumer products, the problem is user stupidity. However, I think this is different - it is an entirely new level of tech that has never before been seen by anyone, it can be amazingly useful if used with a good skeptical eye, but also truly dangerous if it is trusted too much. And you've seen the level of anthropomorphization that happens here on HN, so there is a real psychological tendency to trust it too much. So, I'd say tell 'em. Anyway, fun conversation, hope you're having a great weekend!
        SanderNL 4y ago
        True, it’s definitely very new and even I am prone to believing what it says sometimes. Then I have to remind myself that every letter can be complete and utter nonsense. Now I think of it, it’s like conversing with an expert salesman or politician. Very tiring as they are skilled in framing the conversation. You have to double check every word. How cool would it be if a politician would have an overlay on TV saying not to trust what he/she says plus some real-time fact checking. Fun times. I hope you’re enjoying the weekend too, have a good one!

toss1 4y ago

>>What’s wrong with: “it is often wrong”? Good q; the problem is the passive voice The passive voice connotes far less urgency and importance to any warning. Even "It is often dangerously wrong.", is still a more passive phrasing than "It will lie to you, with a highly confidant tone." I also cringe a bit at using active verbs that imply agency and intent in a statistical model; it makes an inaccurate implication about the technology. But the same applies whether we call it "lying" or "confabulating". I also do not see it as 'paternalistically protecting the user', but simply as 'truth in advertising': - accurately representing the strengths and weaknesses of the product. I absolutely agree that everyone should try it themselves - there are MANY useful use-cases. But they should simply be accurately forewarned, in no uncertain terms, about it's behavior.

SanderNL 4y ago
May I ask why? How is this not “paternalistic”? Why protect the people, from what? Is this worse than what they are already exposed to on a daily basis? Does facebook say it fucks you up and makes you an addict? Why this urge to enforce “truth in advertising”? I am not saying you shouldn’t do it, it’s just that I do not get why and the article doesn’t mention it. It is somehow a given that The People need to be protected from this lying machine because they will revert to cannibalism if not properly informed.
- toss1 4y ago
  >>How is this not “paternalistic”? I see three categories of handling this issue, perhaps call them "Paternalistic", "Responsible", and "Anti-Social". - Paternalistic would be like: "We assess [thing] to be dangerous in a number of ways and so we forbid you to use [thing], except by using our high priesthood representatives as intermediaries". - Responsible would be like: We assess [thing] to be dangerous in a number of ways, so we clearly inform you that [thing] is dangerous, specify the risks, and how to avoid or mitigate them. You are then free to use [thing] as you see fit. (Note: if using [thing] badly causes public as well as private risks, it is responsible to require training, tests, and licensing to use [thing] in public). - Anti-Social would be like: We know [thing] is dangerous, but we DGAF what happens, you should figure it out all by yourself, and if you have a problem, piss-off. Examples: giving a CRT monitor to a novice hardware hacker who has only ever seen LED monitors and not telling her in detail about the high-voltage capacitor hazards inside or how to deal with them. Giving a gun to someone without the basic rules (always treat it as if it's loaded, never point it at anything you do not intend to destroy, trigger finger discipline, etc.). A mere "Hey, watch it with that thing" is inadequate, and you'd rightly bear responsibility when they electrocute themselves, shoot themselves in the foot, etc. A decent warning around LLMs could be: "Be warned: while this system can produce amazing and useful results it will frequently lie to you, and with great confidence. Be sure to check all results independently. Do NOT USE IT for any life-critical or potentially life-changing decisions. E.g., Do NOT use it as a sole source for medical diagnoses, as it may give a misdiagnosis that would kill you [0]. Do NOT use it as a sole source for therapy, as it may counsel you to suicide [1]. These are real harms, do NOT use this as the sole source of any answers." - - - Then, let them loose on it. They know about the footguns, and have been warned Anything less borders on anti-social. >> because they will revert to cannibalism if not properly informed It is not that they'll revert to cannibalism if not properly informed, it is that there are potential and serious harms that are easily avoided if they are told. The civilized thing to do when there is a bridge out ahead leading the road into raging floodwaters, is to tell your fellow travelers before they get there, not merely expect that every one of them will notice in time and handle it well. What is a given among civilized people is that if we know something has hazards, we give warnings and information to our fellow travelers. That's not paternalistic. Paternalistic is to not give warnings, but to forbid use. And it is uncivilized and anti-social to fail to take the trouble to give full and straight information. Informed consent is a good thing. Handing somebody what is effectively a booby-trap without telling them is not. [0] https://inflecthealth.medium.com/im-an-er-doctor-here-s-what-i-found-when-i-asked-chatgpt-to-diagnose-my-patients-7829c375a9da https://inflecthealth.medium.com/im-an-er-doctor-here-s-what... [1] https://interestingengineering.com/culture/belgian-woman-blames-chatgpt-like-chatbot-eliza-for-her-husbands-suicide https://interestingengineering.com/culture/belgian-woman-bla...
  - SanderNL 4y ago
    Interesting. Thank you for the detailed response. Paternalistic is indeed not the right term and it’s nice to worry about your fellow humans to this degree. I guess I am anti-social. But I disagree on the premise that this is comparable to a dangerous boobytrap. It’s not completely without danger of course, but that’s a property it shares with kitchen utensils, cars, scissors, religious texts.. Are we going to preface the Bible with this kind of warnings too? I’m all for it, but be consistent. I also think that warnings like that are cheap ways to evade liability and do not actually solve the issue which is people being stupid. Your example is not about people being deliberate and thinking deeply about their actions based on given information. If people kill themselves after talking to a chatbot I do not think a textual warning would have sufficed. > it is that there are potential and serious harms that are easily avoided if they are told Let’s agree to disagree. Sorry for sounding obtuse. I enjoyed your input, makes me think. I’m just a classic annoying neckbeard.
    - toss1 4y ago
      >>Are we going to preface the Bible with this kind of warnings too? I’m all for it, but be consistent. Outstanding idea!! >>warnings like that are cheap ways to evade liability and do not actually solve the issue which is people being stupid. Yes, the all-too-common generic disclaimers and warnings to cover asses for liability are usually so broad and vague as to be useless. And generally with ordinary well-tested consumer products, the problem is user stupidity. However, I think this is different - it is an entirely new level of tech that has never before been seen by anyone, it can be amazingly useful if used with a good skeptical eye, but also truly dangerous if it is trusted too much. And you've seen the level of anthropomorphization that happens here on HN, so there is a real psychological tendency to trust it too much. So, I'd say tell 'em. Anyway, fun conversation, hope you're having a great weekend!
      - SanderNL 4y ago
        True, it’s definitely very new and even I am prone to believing what it says sometimes. Then I have to remind myself that every letter can be complete and utter nonsense. Now I think of it, it’s like conversing with an expert salesman or politician. Very tiring as they are skilled in framing the conversation. You have to double check every word. How cool would it be if a politician would have an overlay on TV saying not to trust what he/she says plus some real-time fact checking. Fun times. I hope you’re enjoying the weekend too, have a good one!