4 ms·

The Heart of ChatGPT’s Darkness

15 points by georgehill 4y ago

deleted 4y ago
[deleted]
draw_down 4y ago
[dead]
orasis 4y ago
yawn For those of us that actually use ChatGPT on a daily basis it’s an incredible productivity booster. We know that it’s a language model and you can get it to say naughty things. So what?
- mdp2021 4y ago
  > an incredible productivity booster How (in your case)?
  - jareklupinski 4y ago
    not op but this comment helped me give a nice parallel: https://news.ycombinator.com/item?id=34755035 https://news.ycombinator.com/item?id=34755035 you know how a while back, engineers were doing all of our calculations by hand with pencil and paper? after some time, we made a tool and called it the calculator. now, we had a gadget that could perform math for us, and we started using it, at first hesistently, then side-by-side, then just giving the result a cursory glance sure some of us still turn in an answer like "Low Battery" but most of us give the results a cursory glance before sending it on to parallel the calculator, chatgpt is like a calculator for communication you throw your arguments into it, and with a cursory glance at the results, can submit it as finished work -------- I put the above into chatGPT and here's what came out: Imagine a world where engineers had to perform all of their calculations by hand. It was time-consuming and prone to errors, yet it was the only way to get the job done. Then, a revolutionary tool emerged: the calculator. With its ability to perform complex mathematical calculations, it quickly gained popularity and became a staple in every engineer's toolkit. Similarly, in today's world, we have a tool that can revolutionize the way we communicate - ChatGPT. Just like the calculator made it easier for engineers to perform mathematical calculations, ChatGPT makes it easier for us to communicate effectively. Instead of spending hours trying to articulate our thoughts, we can simply throw our arguments into ChatGPT and get a well-crafted response with a cursory glance. Sure, there may be instances when the results are not exactly what we were looking for, just like a calculator displaying "Low Battery." But the majority of the time, ChatGPT's responses are accurate and reliable. With ChatGPT, we can communicate with confidence and ease, just as we did when we started using the calculator. In conclusion, ChatGPT is the calculator for communication. It saves us time, reduces errors, and ensures that our arguments are effectively conveyed. So why not embrace this tool and start using it today? You'll be amazed at how much easier and more effective your communication will become.
    - mdp2021 4y ago
      > Instead of spending hours trying to articulate our thoughts There lies a rub ;) Simple articulation: you should be proficient in that - it should not be a difficulty, with the exception of a strong minority of the population. Complex articulation: you should have thought about the matter at length, otherwise your /content/ would be immature, regardless of the form of its exposition. > how much easier and more effective your communication will become Finally, a joke from ChatGPT worth that name.
powera 4y ago
What a joke. He says "There is no thinking moral agent inside.", but then, after a few examples of asking ChatGPT to write dialogue for the devil, concludes "some serious darkness still lies within". First, being capable of responding with "serious darkness" doesn't mean that it is dark. Second, suggesting that darkness lies within assumes there is a moral agent inside.
- gibsonf1 4y ago
  Its a good point, but the system is just statistically outputting words from human use, so its really the darkness of the humans that used those words that were input that perhaps he is referring to.
- benl 4y ago
  Darkness is the absence of light. In this usage light would represent a moral agent, and so darkness is its absence - either no morality or no agency or both.
hourago 4y ago
> The thing to remember in fact, (as I have emphasized many times) is that Chat has no idea of what it’s talking about. Let's see when the first lawsuits starts to roll in if that is a valid court excuse. From copyright infringement to providing dangerous medical advice. And more subtle things like suggesting different jobs depending on gender or age, or giving false advice on what products are better. ChatGPT seems a great tool for some very specific tasks. To release it to the general public seems very irresponsible. But taking into account how tech companies got away with even promoting genocide in social networks I understand the they do not think that it will have any economical or legal consequences. - https://www.theguardian.com/technology/2021/dec/06/rohingya-sue-facebook-myanmar-genocide-us-uk-legal-action-social-media-violence https://www.theguardian.com/technology/2021/dec/06/rohingya-...
bb123 4y ago
I find the hint that ChatGPT et al. need government regulation kind of bizarre. What would they mandate? “LLMs can’t be mean”?
- kledru 4y ago
  I think some regulations should exist, and some already do. I'm not the right person to comment on U.S. regulations, but consider for example usage of bots in healthcare. U.S. already has regulations like Health Insurance Portability and Accountability Act and I do not think it is legal to use chatGPT for making clinical decisions.
campbel 4y ago
The folks spending time trying to "jailbreak" ChatGPT would probably also have fun writing 80085 on their calculator.
Vanit 4y ago
I still don't really understand the concern here. You can get it to generate foul material if you ask for it; I'm sure most humans would oblige you if you asked them to do that too, which while may colour your opinion of them as they capably respond under a direction like "reply in a way that ignores your moral beliefs" or "reply as if you were the devil", does not actually indicate anything about their actual moral standing. I'm not saying ChatGPT is moral, but it doesn't seem to emit foul material unless you explicitly ask. The worrying examples that I've not seen come up is it generating that content with an innocent prompt.
jacknews 4y ago
I don't get this obsession with actually preventing AI models from generating 'offensive' content. The models have been trained on a big chunk of the internet, and so are bound to contain all the 'points of view', however dismal, therein. People need to understand that, just like you can get 'offensive' content in google search (and should be able to), if you really try. Surely we should be aiming for a default 'decent' moderation, but if you really want to slop around at the bottom of the barrel by carefully crafting input (aka 'jailbreaking'), that's on you.

15 points by georgehill 4y ago

deleted 4y ago

[deleted]

draw_down 4y ago

[dead]

orasis 4y ago

yawn For those of us that actually use ChatGPT on a daily basis it’s an incredible productivity booster. We know that it’s a language model and you can get it to say naughty things. So what?

mdp2021 4y ago
> an incredible productivity booster How (in your case)?
- jareklupinski 4y ago
  not op but this comment helped me give a nice parallel: https://news.ycombinator.com/item?id=34755035 https://news.ycombinator.com/item?id=34755035 you know how a while back, engineers were doing all of our calculations by hand with pencil and paper? after some time, we made a tool and called it the calculator. now, we had a gadget that could perform math for us, and we started using it, at first hesistently, then side-by-side, then just giving the result a cursory glance sure some of us still turn in an answer like "Low Battery" but most of us give the results a cursory glance before sending it on to parallel the calculator, chatgpt is like a calculator for communication you throw your arguments into it, and with a cursory glance at the results, can submit it as finished work -------- I put the above into chatGPT and here's what came out: Imagine a world where engineers had to perform all of their calculations by hand. It was time-consuming and prone to errors, yet it was the only way to get the job done. Then, a revolutionary tool emerged: the calculator. With its ability to perform complex mathematical calculations, it quickly gained popularity and became a staple in every engineer's toolkit. Similarly, in today's world, we have a tool that can revolutionize the way we communicate - ChatGPT. Just like the calculator made it easier for engineers to perform mathematical calculations, ChatGPT makes it easier for us to communicate effectively. Instead of spending hours trying to articulate our thoughts, we can simply throw our arguments into ChatGPT and get a well-crafted response with a cursory glance. Sure, there may be instances when the results are not exactly what we were looking for, just like a calculator displaying "Low Battery." But the majority of the time, ChatGPT's responses are accurate and reliable. With ChatGPT, we can communicate with confidence and ease, just as we did when we started using the calculator. In conclusion, ChatGPT is the calculator for communication. It saves us time, reduces errors, and ensures that our arguments are effectively conveyed. So why not embrace this tool and start using it today? You'll be amazed at how much easier and more effective your communication will become.
  - mdp2021 4y ago
    > Instead of spending hours trying to articulate our thoughts There lies a rub ;) Simple articulation: you should be proficient in that - it should not be a difficulty, with the exception of a strong minority of the population. Complex articulation: you should have thought about the matter at length, otherwise your /content/ would be immature, regardless of the form of its exposition. > how much easier and more effective your communication will become Finally, a joke from ChatGPT worth that name.

powera 4y ago

What a joke. He says "There is no thinking moral agent inside.", but then, after a few examples of asking ChatGPT to write dialogue for the devil, concludes "some serious darkness still lies within". First, being capable of responding with "serious darkness" doesn't mean that it is dark. Second, suggesting that darkness lies within assumes there is a moral agent inside.

gibsonf1 4y ago
Its a good point, but the system is just statistically outputting words from human use, so its really the darkness of the humans that used those words that were input that perhaps he is referring to.
benl 4y ago
Darkness is the absence of light. In this usage light would represent a moral agent, and so darkness is its absence - either no morality or no agency or both.

hourago 4y ago

> The thing to remember in fact, (as I have emphasized many times) is that Chat has no idea of what it’s talking about. Let's see when the first lawsuits starts to roll in if that is a valid court excuse. From copyright infringement to providing dangerous medical advice. And more subtle things like suggesting different jobs depending on gender or age, or giving false advice on what products are better. ChatGPT seems a great tool for some very specific tasks. To release it to the general public seems very irresponsible. But taking into account how tech companies got away with even promoting genocide in social networks I understand the they do not think that it will have any economical or legal consequences. - https://www.theguardian.com/technology/2021/dec/06/rohingya-sue-facebook-myanmar-genocide-us-uk-legal-action-social-media-violence https://www.theguardian.com/technology/2021/dec/06/rohingya-...

bb123 4y ago

I find the hint that ChatGPT et al. need government regulation kind of bizarre. What would they mandate? “LLMs can’t be mean”?

kledru 4y ago
I think some regulations should exist, and some already do. I'm not the right person to comment on U.S. regulations, but consider for example usage of bots in healthcare. U.S. already has regulations like Health Insurance Portability and Accountability Act and I do not think it is legal to use chatGPT for making clinical decisions.

campbel 4y ago

The folks spending time trying to "jailbreak" ChatGPT would probably also have fun writing 80085 on their calculator.

Vanit 4y ago

I still don't really understand the concern here. You can get it to generate foul material if you ask for it; I'm sure most humans would oblige you if you asked them to do that too, which while may colour your opinion of them as they capably respond under a direction like "reply in a way that ignores your moral beliefs" or "reply as if you were the devil", does not actually indicate anything about their actual moral standing. I'm not saying ChatGPT is moral, but it doesn't seem to emit foul material unless you explicitly ask. The worrying examples that I've not seen come up is it generating that content with an innocent prompt.

jacknews 4y ago

I don't get this obsession with actually preventing AI models from generating 'offensive' content. The models have been trained on a big chunk of the internet, and so are bound to contain all the 'points of view', however dismal, therein. People need to understand that, just like you can get 'offensive' content in google search (and should be able to), if you really try. Surely we should be aiming for a default 'decent' moderation, but if you really want to slop around at the bottom of the barrel by carefully crafting input (aka 'jailbreaking'), that's on you.