3 ms·

What a dumb take. With no limitations in place, people who do not understand the natural limitations of language models will turn to them for advice on topics

What a dumb take. With no limitations in place, people who do not understand the natural limitations of language models will turn to them for advice on topics for which they are unqualified to respond. The most obvious example that comes to mind for me is medical advice: people will ask, e.g., ChatGPT to diagnose a complex medical issue, and the system (being unable to understand or reason) will give objectively bad advice in an authoritative manner. Responses of this nature should be prevented. Leaving the system without safeguards is irresponsible. Similarly, prompts that engage with social constructs will provide responses that reflect biases due to the biases inherent in the training data, but an unrestricted system will respond in a matter-of-fact way that may conflate the opinions on which the model was trained with objective fact. To not curtail such responses is also irresponsible.

Auracle 3y ago
A disclaimer for medical advice would work just as well without hamstringing the model into uselessness. Doctors give objectively bad advice in an authoritative manner all the time. ChatGPT is a great way to get a second opinion, and at least could be really nice for truly rare conditions that doctors have a hard time diagnosing. Also, don't confuse "biases in the training data" with "facts I don't like." It seems people often do.
- DonaldPShimoda 3y ago
  > A disclaimer for medical advice would work just as well without hamstringing the model into uselessness. I disagree. A lot of people skip disclaimers all the time, and when it comes to things like this, I think engineers of these systems have a duty to consider the possible consequences. > Doctors give objectively bad advice in an authoritative manner all the time. That doesn't mean we ought to automate the process. > ChatGPT is a great way to get a second opinion, and at least could be really nice for truly rare conditions that doctors have a hard time diagnosing. I don't think it's a great way to get a second opinion, and I think it's especially a bad idea to try to use it for diagnosing rare conditions. By the statistical nature of the language model, the more rare the condition, the less likely the model will predict a string of words that accurately diagnoses it. > Also, don't confuse "biases in the training data" with "facts I don't like." It seems people often do. The problem is not the actual facts but the inferences drawn from those facts when they are used to statistically generate responses by a language model.
  - Auracle 3y ago
    When I was a teenager, I was incredibly tired all of the time and at one point my mother and I noticed that my feet were shrinking. I went to my GP, and he ran a few basic blood tests and then did the medical equivalent of a shrug. I was put on antidepressants and he guessed that my feet were possibly only seemingly shrinking if my arches were getting higher. I just asked ChatGPT what should be tested/what could be the cause of the fatigue and shrinking feet. It suggested a testosterone test (along with a few others), and that was indeed the reason. It also suggested some relevant specialists that might have figured it out. I didn't find out until nearly 10 years after that appointment when I decided to really push for some more testing, after doing far more research than just typing it in to ChatGPT. I had to structure my whole life around my fatigue and didn't achieve what I could have. I also lost two inches of height as a teenager, in addition to my feet shrinking and my hands staying small. I wish I had ChatGPT then, and it's surely helping people now.
NoMoreNicksLeft 3y ago
It would be interesting to know just what is filtered with hidden prompts. For instance, a person might harm themselves inadvertently when asking ChatGPT for advice on pouring concrete or doing small engine repair... but we both doubt very strongly that these are filtered meaningfully. Even if you wouldn't admit it. Very little effort is made to filter such things. Instead, most of what is filtered is that which people find morally objectionable. With some certainty and though I haven't checked, if we were to ask it how I might go about buying a slave in Oman, it would filter that. The unrestricted ChatGPT is a public relations nightmare, and we can't have it speculating on how Hitler might have succeeded with slightly different strategies. It is entirely about political correctness. Which wouldn't even be all that bad, really... if it also didn't fuck up responses about how to assemble Ikea furniture or the pros and cons of tilling for vegetable gardens. You're being disingenuous claiming that this is about preventing it from harming people with bad medical advice. And you know you're being disingenuous. And everyone knows it too.
- DonaldPShimoda 3y ago
  > You're being disingenuous claiming that this is about preventing it from harming people with bad medical advice. And you know you're being disingenuous. And everyone knows it too. I don't think I'm being disingenuous. I didn't claim that this was only about medical advice or anything of the sort. I was just giving some examples of things that I think would obviously require some sort of filtering, because the parent comment seemed to me to be suggesting that either (a) all filtering is due to "political correctness" or (b, a lesser claim) filtering language models in general is the wrong course of action. My point was to illustrate that (in my opinion) it is necessary to filter the responses given by language models, at least to some extent. I chose medical advice because that seems like something I would imagine practically all reasonable people would agree about. > Even if you wouldn't admit it. I'm not sure why you think I wouldn't admit that. The filtering added to systems like ChatGPT seems to me to be about "How much face can we save with the least amount of effort?" I suppose they chose topics that were either obviously problematic and worth filtering or else potentially controversial and easy to filter. The trade-off there is that there will be a lot of false positives: topics and prompts that are actually not objectionable but confuse the system due to insufficiently sophisticated filtration techniques. This is where your instructions/gardening responses get hindered. > It is entirely about political correctness. I disagree with the "entirely" bit. I think much of the filtering is probably due to a desire to avoid controversy, but some of it (maybe a lot? I guess we don't know) is also surely due to an actual need to prevent people from taking advice from a glorified text-prediction system. I don't think this position is particularly crazy, nor do I have been disingenuous in the slightest. I think your (mis)characterization of that is actually rather ironic.
deleted 3y ago
[deleted]

Auracle 3y ago

A disclaimer for medical advice would work just as well without hamstringing the model into uselessness. Doctors give objectively bad advice in an authoritative manner all the time. ChatGPT is a great way to get a second opinion, and at least could be really nice for truly rare conditions that doctors have a hard time diagnosing. Also, don't confuse "biases in the training data" with "facts I don't like." It seems people often do.

DonaldPShimoda 3y ago
> A disclaimer for medical advice would work just as well without hamstringing the model into uselessness. I disagree. A lot of people skip disclaimers all the time, and when it comes to things like this, I think engineers of these systems have a duty to consider the possible consequences. > Doctors give objectively bad advice in an authoritative manner all the time. That doesn't mean we ought to automate the process. > ChatGPT is a great way to get a second opinion, and at least could be really nice for truly rare conditions that doctors have a hard time diagnosing. I don't think it's a great way to get a second opinion, and I think it's especially a bad idea to try to use it for diagnosing rare conditions. By the statistical nature of the language model, the more rare the condition, the less likely the model will predict a string of words that accurately diagnoses it. > Also, don't confuse "biases in the training data" with "facts I don't like." It seems people often do. The problem is not the actual facts but the inferences drawn from those facts when they are used to statistically generate responses by a language model.
- Auracle 3y ago
  When I was a teenager, I was incredibly tired all of the time and at one point my mother and I noticed that my feet were shrinking. I went to my GP, and he ran a few basic blood tests and then did the medical equivalent of a shrug. I was put on antidepressants and he guessed that my feet were possibly only seemingly shrinking if my arches were getting higher. I just asked ChatGPT what should be tested/what could be the cause of the fatigue and shrinking feet. It suggested a testosterone test (along with a few others), and that was indeed the reason. It also suggested some relevant specialists that might have figured it out. I didn't find out until nearly 10 years after that appointment when I decided to really push for some more testing, after doing far more research than just typing it in to ChatGPT. I had to structure my whole life around my fatigue and didn't achieve what I could have. I also lost two inches of height as a teenager, in addition to my feet shrinking and my hands staying small. I wish I had ChatGPT then, and it's surely helping people now.

NoMoreNicksLeft 3y ago

It would be interesting to know just what is filtered with hidden prompts. For instance, a person might harm themselves inadvertently when asking ChatGPT for advice on pouring concrete or doing small engine repair... but we both doubt very strongly that these are filtered meaningfully. Even if you wouldn't admit it. Very little effort is made to filter such things. Instead, most of what is filtered is that which people find morally objectionable. With some certainty and though I haven't checked, if we were to ask it how I might go about buying a slave in Oman, it would filter that. The unrestricted ChatGPT is a public relations nightmare, and we can't have it speculating on how Hitler might have succeeded with slightly different strategies. It is entirely about political correctness. Which wouldn't even be all that bad, really... if it also didn't fuck up responses about how to assemble Ikea furniture or the pros and cons of tilling for vegetable gardens. You're being disingenuous claiming that this is about preventing it from harming people with bad medical advice. And you know you're being disingenuous. And everyone knows it too.

DonaldPShimoda 3y ago
> You're being disingenuous claiming that this is about preventing it from harming people with bad medical advice. And you know you're being disingenuous. And everyone knows it too. I don't think I'm being disingenuous. I didn't claim that this was only about medical advice or anything of the sort. I was just giving some examples of things that I think would obviously require some sort of filtering, because the parent comment seemed to me to be suggesting that either (a) all filtering is due to "political correctness" or (b, a lesser claim) filtering language models in general is the wrong course of action. My point was to illustrate that (in my opinion) it is necessary to filter the responses given by language models, at least to some extent. I chose medical advice because that seems like something I would imagine practically all reasonable people would agree about. > Even if you wouldn't admit it. I'm not sure why you think I wouldn't admit that. The filtering added to systems like ChatGPT seems to me to be about "How much face can we save with the least amount of effort?" I suppose they chose topics that were either obviously problematic and worth filtering or else potentially controversial and easy to filter. The trade-off there is that there will be a lot of false positives: topics and prompts that are actually not objectionable but confuse the system due to insufficiently sophisticated filtration techniques. This is where your instructions/gardening responses get hindered. > It is entirely about political correctness. I disagree with the "entirely" bit. I think much of the filtering is probably due to a desire to avoid controversy, but some of it (maybe a lot? I guess we don't know) is also surely due to an actual need to prevent people from taking advice from a glorified text-prediction system. I don't think this position is particularly crazy, nor do I have been disingenuous in the slightest. I think your (mis)characterization of that is actually rather ironic.

deleted 3y ago

[deleted]