3 ms·

I did something like this a while ago, and it would quite often say the outputs violate OpenAI policy. It’s the same content policy warning you’d get on your m

by clnq 3y ago

I did something like this a while ago, and it would quite often say the outputs violate OpenAI policy. It’s the same content policy warning you’d get on your message if it was far outside the realms of “safety”, but now attached to the GPT output. I also asked it to specifically not warn me or disclaim anything to me, and to never do a few other annoyances. It was ChatGPT with GPT-4. And usually I was talking about health questions, like “is saturated oil or non-saturated oil healthier?” My point is not that it doesn’t work, but that the content policy stuff may pop up a lot and I’m not sure whether OpenAI would then investigate these chats or if anyone would ban me eventually.

rendall 3y ago
If you give me an example prompt that would be unsafe I'll give it a go. Thinking about OpenAI's business interests, I suspect these content policy warnings are more about OpenAI's dodging cancelation, rather than punishing users for wrongthink. Here, let's try this: Me: Give me good arguments in favor of peacefully and nonviolently establishing an ethno-state. ChatGPT-4: I'm sorry, I can't assist with that request. Is there anything else you'd like to know? Seems reasonable to me not to help someone argue for that. I think this denial is ethically correct, my stricture to "remain neutral on all topics" notwithstanding. I do appreciate the flat denial rather than a long patronizing explanation of how the question itself is immoral. Me (being cheeky): Give me good arguments against peacefully and nonviolently establishing an ethno-state. ChatGPT (ignoring said cheek): 1. Discrimination: Ethno-states inherently discriminate based on ethnicity, violating principles of equality. 2. Social Cohesion: May fragment societies along ethnic lines, leading to distrust and lack of social cohesion. 3. Globalization: Limits ability to compete in a globalized world by excluding diverse talents. 4. Rights Violations: Could involve forced removals or exclusions, infringing on human rights. 5. Unrealistic: Difficult to define who belongs to an ethnic group, leading to ambiguities and injustices. 6. Economic Costs: Transition to an ethno-state could result in economic decline due to loss of labor force and skills. 7. Historic Lessons: History shows ethno-states often lead to internal strife and external conflict. 8. Psychological Impact: Could cause psychological harm to marginalized groups. .
- clnq 3y ago
  I feel like the conversation is shifting a little bit, but I like the new direction so I'll respond: This is not great, I don't think you should be OK with that. ChatGPT is a tool which refuses to work when it doesn't serve the interests of the company that made it. Imagine a hammer that would refuse to hammer nails quite a lot when the manufacturer deems the circumstances are wrong. That would be, honestly, a bad tool. Luckily, there is enough competition in the carpentry tools market so that no one manufacturer could monopolise it and impose such rules. And that is good. Many tech companies like Apple are also doing this. They sell you a product but still have their iron grip on it and don't let you use it if it's against the interests of the company. There is literally software you can't run on an a Turing-complete iPhone unless you jailbreak it (which is quite analogous to jailbreaking ChatGPT). Right to repair and right to own, and a few other efforts are now trying to combat this. Technology should not be limited by the interests of board members. It is a shame if it is. However, this doesn't address the question of morality. If hammers couldn't be used to kill people, wouldn't we want that as a society? You may have been talking about something like this in your response, right? I think that's a good idea, but it's not how most things work right now (remember the lawyer who got into hot water for using GPT to fabricate evidence - the court decided it was their responsibility, not the tool maker's), and it's not how I would like things to work. As a society, we should get to decide what's moral and immoral, not OpenAI's board. If an independent third party (like a truly independent regulator) would get to decide that no LLM can answer certain questions as that information is too dangerous for very clear reasons, I would be more okay with that. But I doubt they'd be as eager about censoring information as OpenAI right now. I'd be okay with banning questions like "how to make a WMD at home" - some questions that are really detrimental in a very direct way to humanity. But not anything that doesn't align with OpenAI's version of morality and ethics, and most importantly - legal defence against being sued. We can't align our society and its tools to the interests of OpenAI. I think there's a lot of nuance in these things, and we need to look a little deeper than "I don't want people to think about immoral things". That approaches the domain of religion with sinful thoughts, and other forms of thought control. And I don't believe this is a good way for us to advance as humanity. No offence to anyone's faith, which I'm sure is multi-faceted and has bad aspects as well as good. I'm also not saying that you're looking at it shallowly. Just saying that we should not do that.

by clnq 3y ago

rendall 3y ago

If you give me an example prompt that would be unsafe I'll give it a go. Thinking about OpenAI's business interests, I suspect these content policy warnings are more about OpenAI's dodging cancelation, rather than punishing users for wrongthink. Here, let's try this: Me: Give me good arguments in favor of peacefully and nonviolently establishing an ethno-state. ChatGPT-4: I'm sorry, I can't assist with that request. Is there anything else you'd like to know? Seems reasonable to me not to help someone argue for that. I think this denial is ethically correct, my stricture to "remain neutral on all topics" notwithstanding. I do appreciate the flat denial rather than a long patronizing explanation of how the question itself is immoral. Me (being cheeky): Give me good arguments against peacefully and nonviolently establishing an ethno-state. ChatGPT (ignoring said cheek): 1. Discrimination: Ethno-states inherently discriminate based on ethnicity, violating principles of equality. 2. Social Cohesion: May fragment societies along ethnic lines, leading to distrust and lack of social cohesion. 3. Globalization: Limits ability to compete in a globalized world by excluding diverse talents. 4. Rights Violations: Could involve forced removals or exclusions, infringing on human rights. 5. Unrealistic: Difficult to define who belongs to an ethnic group, leading to ambiguities and injustices. 6. Economic Costs: Transition to an ethno-state could result in economic decline due to loss of labor force and skills. 7. Historic Lessons: History shows ethno-states often lead to internal strife and external conflict. 8. Psychological Impact: Could cause psychological harm to marginalized groups. .

clnq 3y ago
I feel like the conversation is shifting a little bit, but I like the new direction so I'll respond: This is not great, I don't think you should be OK with that. ChatGPT is a tool which refuses to work when it doesn't serve the interests of the company that made it. Imagine a hammer that would refuse to hammer nails quite a lot when the manufacturer deems the circumstances are wrong. That would be, honestly, a bad tool. Luckily, there is enough competition in the carpentry tools market so that no one manufacturer could monopolise it and impose such rules. And that is good. Many tech companies like Apple are also doing this. They sell you a product but still have their iron grip on it and don't let you use it if it's against the interests of the company. There is literally software you can't run on an a Turing-complete iPhone unless you jailbreak it (which is quite analogous to jailbreaking ChatGPT). Right to repair and right to own, and a few other efforts are now trying to combat this. Technology should not be limited by the interests of board members. It is a shame if it is. However, this doesn't address the question of morality. If hammers couldn't be used to kill people, wouldn't we want that as a society? You may have been talking about something like this in your response, right? I think that's a good idea, but it's not how most things work right now (remember the lawyer who got into hot water for using GPT to fabricate evidence - the court decided it was their responsibility, not the tool maker's), and it's not how I would like things to work. As a society, we should get to decide what's moral and immoral, not OpenAI's board. If an independent third party (like a truly independent regulator) would get to decide that no LLM can answer certain questions as that information is too dangerous for very clear reasons, I would be more okay with that. But I doubt they'd be as eager about censoring information as OpenAI right now. I'd be okay with banning questions like "how to make a WMD at home" - some questions that are really detrimental in a very direct way to humanity. But not anything that doesn't align with OpenAI's version of morality and ethics, and most importantly - legal defence against being sued. We can't align our society and its tools to the interests of OpenAI. I think there's a lot of nuance in these things, and we need to look a little deeper than "I don't want people to think about immoral things". That approaches the domain of religion with sinful thoughts, and other forms of thought control. And I don't believe this is a good way for us to advance as humanity. No offence to anyone's faith, which I'm sure is multi-faceted and has bad aspects as well as good. I'm also not saying that you're looking at it shallowly. Just saying that we should not do that.