3 ms·

OpenAI does things in the name of safety that aren't actually needed. For example, you can use Google to search for porn, racist comments, sexist jokes, gay jo

OpenAI does things in the name of safety that aren't actually needed. For example, you can use Google to search for porn, racist comments, sexist jokes, gay jokes, or design instructions for atom bombs. It will happily return you a link for the latter... on Wikipedia! Apparently OpenAI is made up of thin-skinned gay a-sexual Mormons, or something. I get it, corporations want a neutered AI. The general public wants an AI that can role-play a character in a D&D game without pausing to go on a rant every five turns about how violence is bad and that it can't make "racist" comments about the Drow because they're black. They really should have multiple models to choose from, or stop trickling out API/Sandpit access to GPT 4 and allow responsible adults to use the AI to write a joke for them if they so choose.

danpalmer 3y ago
There's a big difference between linking to external content and generating your own content. It's like writing a story about a homophobic/racist character in context, vs writing a homophobic book – the former is fine when done well and the latter is bad. This is however both too much nuance for an LLM to act in a way reliably (i.e. separating its role-played character and itself), and too much nuance for most humans to understand. We might understand what LLMs are, their limitations, and how we should treat the content, but for most people they will take it at face value as being a statement made by OpenAI.
- jiggawatts 3y ago
  Look. Some people thought Harry Potter encouraged wicca, and that Dungeons & Dragons were the gateway into diabolism. Those types of people are the outliers, not the norm.
  - danpalmer 3y ago
    True but that's not who I'm thinking about. Currently, to correctly use LLM output you must: - Understand that they are basing answers on particular set of content that may not be correct, and was fixed in time. - Understand that the company producing it has not programmed in each response. - Understand that the LLM does not (as far as we can tell) have an internal concept of truth and lies, and is therefore not attempting to minimise lying, only produce truthful-sounding language. Most people engaging with LLMs today do not know these things, and this is why we already have examples like professors asking ChatGPT if it generated student essays, and failing those students – something it is entirely incapable of answering. Expanding access to 1bn users is going to run into this far more. We clearly don't have the ability to communicate these nuances effectively to early adopters, let alone 1bn users, and that's not because they're stupid, it's because LLMs are in an uncanny valley – they appear human, but are not in important ways that we have not encountered before as a society. For a long time to come, the norm will be that people will assume OpenAI wrote the answers for each question, in the same way that most people assume Google programs the search results for each query. Anything outside of this is a minority tech bubble.
ChatGTP 3y ago
Different, you don’t talk to Google, you can’t convince yourself it’s time to take up arms against a certain race of people with Google. ChatGPT without guardrails would absolutely be able to convince people to do very bad things. OpenAI and Microsoft are paranoid about that. Maybe you’re right, they’re taking things a bit too far, but fine tuning down to the precise level of, please never mention the world “foot” again wouldn’t be easy if ever possible. An uncensored ChatGPT 4 would be like Frank from Donnie Darko on steroids. My opinion is, Google already had LLMs and chose not to release them to the public for this reason. Basically, they thought about it.
- candiodari 3y ago
  I wonder if that's the goal. I mean the whole point of everything Google does is convincing people to take buy product X. If these networks can convince people ... that would be worth a lot of money for them.
drewbeck 3y ago
> The general public wants an AI that can role-play a character in a D&D game without pausing to go on a rant every five turns about how violence is bad and that it can't make "racist" comments about the Drow because they're black. Gonna need a citation on this one. You think the general public is primarily concerned with D&D content generation? Also your assertion that Open AI must be run by “gay Mormons” because they try to prevent offensive gay jokes shows that you can only imagine someone acting out of personal attachment and not, you know, preventing actual harm regardless of who it happens to. Also don’t worry, there will be plenty of models for you to choose from in the future that will gladly spit out homophobic jokes. Your need for AI generated gay jokes will be met in the future, just not by big corporate models. Rest easy!
- slowmovintarget 3y ago
  The problem is that they prevent any joke about gay individuals, not just jokes that are demeaning or derogatory. Joking about things you encounter in every day life normalizes the subject. What this treatment does is say these categories are not normal, which, I thought, was not the outcome we should be looking for.
  - drewbeck 3y ago
    The nuance required here is something that LLMs have not been great at, or at least not good enough to prevent actual homophobic jokes. See the general issue that LLMs struggle with negation. In fact alignment work is on some level about getting LLMs to be smart about this nuance. I agree in the abstract that, for instance, an LLM could/should be able to make jokes about Tim Cook — just because he’s gay doesn’t mean he should be off limits for a joke about Apple, for instance. That would demonstrate an impressive level of nuance. But also like … what’s the business play here? “Our LLM tells jokes about gay people that aren’t homophobic!” doesn’t strike me as a powerful market differentiator; or at least the market will have to evolve in very specific ways to make that meaningful. The handwringing that I see on HN around LLMs being neutered really seems like a plea for alignment, just of a different sort: folks want the models to reflect their priors, their alignment, where nothing is off limits. The business case for this — and the general social value of this — is generally unexamined.

danpalmer 3y ago

There's a big difference between linking to external content and generating your own content. It's like writing a story about a homophobic/racist character in context, vs writing a homophobic book – the former is fine when done well and the latter is bad. This is however both too much nuance for an LLM to act in a way reliably (i.e. separating its role-played character and itself), and too much nuance for most humans to understand. We might understand what LLMs are, their limitations, and how we should treat the content, but for most people they will take it at face value as being a statement made by OpenAI.

jiggawatts 3y ago
Look. Some people thought Harry Potter encouraged wicca, and that Dungeons & Dragons were the gateway into diabolism. Those types of people are the outliers, not the norm.
- danpalmer 3y ago
  True but that's not who I'm thinking about. Currently, to correctly use LLM output you must: - Understand that they are basing answers on particular set of content that may not be correct, and was fixed in time. - Understand that the company producing it has not programmed in each response. - Understand that the LLM does not (as far as we can tell) have an internal concept of truth and lies, and is therefore not attempting to minimise lying, only produce truthful-sounding language. Most people engaging with LLMs today do not know these things, and this is why we already have examples like professors asking ChatGPT if it generated student essays, and failing those students – something it is entirely incapable of answering. Expanding access to 1bn users is going to run into this far more. We clearly don't have the ability to communicate these nuances effectively to early adopters, let alone 1bn users, and that's not because they're stupid, it's because LLMs are in an uncanny valley – they appear human, but are not in important ways that we have not encountered before as a society. For a long time to come, the norm will be that people will assume OpenAI wrote the answers for each question, in the same way that most people assume Google programs the search results for each query. Anything outside of this is a minority tech bubble.

ChatGTP 3y ago

Different, you don’t talk to Google, you can’t convince yourself it’s time to take up arms against a certain race of people with Google. ChatGPT without guardrails would absolutely be able to convince people to do very bad things. OpenAI and Microsoft are paranoid about that. Maybe you’re right, they’re taking things a bit too far, but fine tuning down to the precise level of, please never mention the world “foot” again wouldn’t be easy if ever possible. An uncensored ChatGPT 4 would be like Frank from Donnie Darko on steroids. My opinion is, Google already had LLMs and chose not to release them to the public for this reason. Basically, they thought about it.

candiodari 3y ago
I wonder if that's the goal. I mean the whole point of everything Google does is convincing people to take buy product X. If these networks can convince people ... that would be worth a lot of money for them.

drewbeck 3y ago

> The general public wants an AI that can role-play a character in a D&D game without pausing to go on a rant every five turns about how violence is bad and that it can't make "racist" comments about the Drow because they're black. Gonna need a citation on this one. You think the general public is primarily concerned with D&D content generation? Also your assertion that Open AI must be run by “gay Mormons” because they try to prevent offensive gay jokes shows that you can only imagine someone acting out of personal attachment and not, you know, preventing actual harm regardless of who it happens to. Also don’t worry, there will be plenty of models for you to choose from in the future that will gladly spit out homophobic jokes. Your need for AI generated gay jokes will be met in the future, just not by big corporate models. Rest easy!

slowmovintarget 3y ago
The problem is that they prevent any joke about gay individuals, not just jokes that are demeaning or derogatory. Joking about things you encounter in every day life normalizes the subject. What this treatment does is say these categories are not normal, which, I thought, was not the outcome we should be looking for.
- drewbeck 3y ago
  The nuance required here is something that LLMs have not been great at, or at least not good enough to prevent actual homophobic jokes. See the general issue that LLMs struggle with negation. In fact alignment work is on some level about getting LLMs to be smart about this nuance. I agree in the abstract that, for instance, an LLM could/should be able to make jokes about Tim Cook — just because he’s gay doesn’t mean he should be off limits for a joke about Apple, for instance. That would demonstrate an impressive level of nuance. But also like … what’s the business play here? “Our LLM tells jokes about gay people that aren’t homophobic!” doesn’t strike me as a powerful market differentiator; or at least the market will have to evolve in very specific ways to make that meaningful. The handwringing that I see on HN around LLMs being neutered really seems like a plea for alignment, just of a different sort: folks want the models to reflect their priors, their alignment, where nothing is off limits. The business case for this — and the general social value of this — is generally unexamined.