6 ms·

People sometimes forget that we had 2-3 years before ChatGPT of fairly high-quality LLMs (from Microsoft, Meta, Google), but they would have to take them down a

People sometimes forget that we had 2-3 years before ChatGPT of fairly high-quality LLMs (from Microsoft, Meta, Google), but they would have to take them down after a few weeks of being released because they became too controversial. Galactica is a good example for instance, there was also quite a good Twitter bot AI from Microsoft for a bit (forget the name), it got a bit racist and became infamous. OpenAI's real achievement was to tame this technology, they were not the first to make good LLMs, but they were the first to fine-tune them with reinforcement learning to make them actually helpful and uncontroversial. ChatGPT was the breakthrough, not GPT-3. There's still work to do in this direction, it's a hard problem. EDIT: by "success" in this context I am referring more to wide applicability and creating value as an actual product, whatever causes a project not to die in the real-world, I am very aware that from just the technical perspective the meaningful breakthroughs are different.

whaleofatw2022 3y ago
> there was also quite a good Twitter bot AI from Microsoft for a bit (forget the name), it got a bit racist and became infamous. Tay!
- sagman 3y ago
  Microsoft should have learned how to red-team from that fiasco. Release a beta version to 4chan and X, and see the different ways a model can be prompted to create controversial results.
  - spiritplumber 3y ago
    Isn't that basically what they did?
    - sagman 3y ago
      They never released a v1. I feel like they could supplement their "red-teaming" efforts by allowing the "professionals" to have a go.
- surgical_fire 3y ago
  I miss her to this day.
Der_Einzige 3y ago
It's totally the other way around. If someone leaked the underlying weights of GPT-3, someone could take a single H100 or even potentially smaller GPU hardware and used direct preference optimization to create a potential superior version of "ChatGPT" today very quickly. Hell, you could likely train a lora adapter on a 3090 or 4090 to unalign it in hours! Toxic-dpo-v2 is literally less than 600 examples and proven to work (https://huggingface.co/datasets/unalignment/toxic-dpo-v0.2 https://huggingface.co/datasets/unalignment/toxic-dpo-v0.2) DPO/RLHF fine-tuning is rather easy in comparison to training large language models from scratch. This is why the community finds open source LLMs growing in quality in "spurts" as new underlying pre-trained models release - i.e. llama -> llama2 -> mistral. The fine-tunes around them are so plagued with issues that benchmarking them is unreliable. If you don't believe me, just study the issues related to the huggingface "OpenLLM leaderboard". Same Dynamic BTW with diffusion models. That's why someone in the AI archeology world is going to have to document that actually true fact that the brony community were the first to figure out how to fine-tune SDXL enough to get it to generate good NSFW content . This fine-tuning was evidently strong enough to break most controlnets and other SDXL related content - indicating that they did "at least close to as much" training as would be required to train from scratch. This is so remarkable because they apparently straight up purchased the A100s that they used for this, as cloud providers are not to be trusted for training on NSFW datasets. Someone "ponied up" at least 100K USD just to give us better quality open source waifus.
- swatcoder 3y ago
  > Someone "ponied up" at least 100K USD just to give us better quality open source waifus. I don't know who spent that money or where they got it from, but not all shirts were lost in the various crypto/NFT bubbles. I don't think we should be surprised if people in that pseudo-anarchist world decide to push back against other institutional power plays like so-called "AI Safety", especially if it can be done for the price of a luxury car.
  - Turing_Machine 3y ago
    It's not surprising at all. Every technological advancement all the way back to the discovery of pottery has been used for porn. Generally it's one of the first applications. I believe I read somewhere that pornographic pictures were available for purchase on the street less than a year after the development (ahem) of the first practical photographic reproduction process.
nirav72 3y ago
Microsoft Tay . Although Tay seemed like more of a bot that was just parroting things people on twitter making it output. It seems like the current crop of LLMs are not even in the same ballpark as Tay.
- whywhywhywhy 3y ago
  I’m surprised Tay hasn’t been cloned with how popular it was.
- UncleOxidant 3y ago
  Yeah, the current crop of LLMs don't dynamically train on user input and this is probably why.
  - jart 3y ago
    Skynet sets the chip to read-only when terminators are sent out on a mission.
nottorp 3y ago
Question: If you tune a neural net to not spew nazi propaganda, you're adjusting some weights that don't only affect nazi propaganda. Protentially or even probably you are also messing with the answers to all the other non controversial issues? There are no identifiable nazi nodes are there?
- Turing_Machine 3y ago
  You're getting modded down, but I think this is a valid concern, or at least one that that shouldn't be dismissed out of hand. Yeah, Nazis = bad. We'll just take that as a given. However, Nazis also did a lot of the pioneering work in rocketry and jet propulsion. If you try to ban everything associated with Nazis, or that was performed by a Nazi, you may accidentally block things that you didn't want to. Maybe a better strategy would be to not be so goddamned concerned about offending someone. As far as Nazi propaganda goes, my high school had a copy of Mein Kampf sitting right there on the shelf. You can't get any more "Nazi propaganda" than that. Yet somehow none of us turned out to be, you know, Nazis.
  - nottorp 3y ago
    > If you try to ban everything associated with Nazis, or that was performed by a Nazi, you may accidentally block things that you didn't want to. No, what I'm saying is you can't ban everything associated with Nazis and nothing else in a LLM, because neural nets don't work like that and you're simply unable to ban something without influencing all the results. Which is worse than just banning info about Wernher von Braun. I may be wrong, considering my knowledge of neural networks is limited, but so far I got one downvote and no explanation...
    - quickslowdown 3y ago
      I think using a less charged example than "Nazis" would probably have helped avoid down voted & lack of engagement, but I understand why you chose it as an example & personally don't take issue with it, especially because you elaborated on why you picked them as an example. Just my $0.02
spaceman_2020 3y ago
I think once we get to GPT-5, people will just get used to their chatbots being capable of insensitive speech, and they’ll ignore it because the usefulness will be so high.
- cbozeman 3y ago
  If it can't do insensitive speech then the usefulness with not be so high. Jordan Peterson put it best, "In order to think, you have to risk being offensive."
  - jart 3y ago
    It's better to think about something else that's boring which others won't care about, rather than focusing on hot button topics. Even if you want to touch the third rail, it's better to do it with jargon. For example, I've yet to meet an activist who's offended by phylogenesis and cladistics but I know many who'd take objection to things Peterson says.
    - quickslowdown 3y ago
      I'm one of them. I'm of the opinion that anything useful or worth hearing, I can hear from someone with less baggage. I don't know the ratio, but Peterson has added more bad to the world that his "good" ideas do not balance out. And after hearing him elaborate on his "bad" ideas, seeing the way he thinks & comes to conclusions, if I'm hearing something that sounds good from him, I would still be skeptical to the point of tuning it out entirely because of the source. I approach the things he says the same way I would an Alex Jones or Elon Musk or Joe Rogan. They don't have enough knowledge in the domains they like to speak in to be fully credible sources, and whatever credibility they might have is outweighed by the other shit they've said that's just outright wrong or intentionally misinformed. The source of the information matters to me, and anything any of these people or their ilk are putting out in the world can be found from someone else who is more worthy of trust. Peterson, like Elon Musk or Joe Rogan et al, have proven enough times they're willing to debase themselves putting nonsense into the world, nonsense they're fully aware is lies and/or harmful, and have burned whatever credibility they might have had. The people taking objection to things Jordan Peterson says may be throwing the baby out with the bathwater, but I would argue that Peterson himself cultivated this lack of trust. I will hear what he has to say from somebody more worthy of my trust & attention.

whaleofatw2022 3y ago

> there was also quite a good Twitter bot AI from Microsoft for a bit (forget the name), it got a bit racist and became infamous. Tay!

sagman 3y ago
Microsoft should have learned how to red-team from that fiasco. Release a beta version to 4chan and X, and see the different ways a model can be prompted to create controversial results.
- spiritplumber 3y ago
  Isn't that basically what they did?
  - sagman 3y ago
    They never released a v1. I feel like they could supplement their "red-teaming" efforts by allowing the "professionals" to have a go.
surgical_fire 3y ago
I miss her to this day.

Der_Einzige 3y ago

It's totally the other way around. If someone leaked the underlying weights of GPT-3, someone could take a single H100 or even potentially smaller GPU hardware and used direct preference optimization to create a potential superior version of "ChatGPT" today very quickly. Hell, you could likely train a lora adapter on a 3090 or 4090 to unalign it in hours! Toxic-dpo-v2 is literally less than 600 examples and proven to work (https://huggingface.co/datasets/unalignment/toxic-dpo-v0.2 https://huggingface.co/datasets/unalignment/toxic-dpo-v0.2) DPO/RLHF fine-tuning is rather easy in comparison to training large language models from scratch. This is why the community finds open source LLMs growing in quality in "spurts" as new underlying pre-trained models release - i.e. llama -> llama2 -> mistral. The fine-tunes around them are so plagued with issues that benchmarking them is unreliable. If you don't believe me, just study the issues related to the huggingface "OpenLLM leaderboard". Same Dynamic BTW with diffusion models. That's why someone in the AI archeology world is going to have to document that actually true fact that the brony community were the first to figure out how to fine-tune SDXL enough to get it to generate good NSFW content . This fine-tuning was evidently strong enough to break most controlnets and other SDXL related content - indicating that they did "at least close to as much" training as would be required to train from scratch. This is so remarkable because they apparently straight up purchased the A100s that they used for this, as cloud providers are not to be trusted for training on NSFW datasets. Someone "ponied up" at least 100K USD just to give us better quality open source waifus.

swatcoder 3y ago
> Someone "ponied up" at least 100K USD just to give us better quality open source waifus. I don't know who spent that money or where they got it from, but not all shirts were lost in the various crypto/NFT bubbles. I don't think we should be surprised if people in that pseudo-anarchist world decide to push back against other institutional power plays like so-called "AI Safety", especially if it can be done for the price of a luxury car.
- Turing_Machine 3y ago
  It's not surprising at all. Every technological advancement all the way back to the discovery of pottery has been used for porn. Generally it's one of the first applications. I believe I read somewhere that pornographic pictures were available for purchase on the street less than a year after the development (ahem) of the first practical photographic reproduction process.

nirav72 3y ago

Microsoft Tay . Although Tay seemed like more of a bot that was just parroting things people on twitter making it output. It seems like the current crop of LLMs are not even in the same ballpark as Tay.

whywhywhywhy 3y ago
I’m surprised Tay hasn’t been cloned with how popular it was.
UncleOxidant 3y ago
Yeah, the current crop of LLMs don't dynamically train on user input and this is probably why.
- jart 3y ago
  Skynet sets the chip to read-only when terminators are sent out on a mission.

nottorp 3y ago

Question: If you tune a neural net to not spew nazi propaganda, you're adjusting some weights that don't only affect nazi propaganda. Protentially or even probably you are also messing with the answers to all the other non controversial issues? There are no identifiable nazi nodes are there?

Turing_Machine 3y ago
You're getting modded down, but I think this is a valid concern, or at least one that that shouldn't be dismissed out of hand. Yeah, Nazis = bad. We'll just take that as a given. However, Nazis also did a lot of the pioneering work in rocketry and jet propulsion. If you try to ban everything associated with Nazis, or that was performed by a Nazi, you may accidentally block things that you didn't want to. Maybe a better strategy would be to not be so goddamned concerned about offending someone. As far as Nazi propaganda goes, my high school had a copy of Mein Kampf sitting right there on the shelf. You can't get any more "Nazi propaganda" than that. Yet somehow none of us turned out to be, you know, Nazis.
- nottorp 3y ago
  > If you try to ban everything associated with Nazis, or that was performed by a Nazi, you may accidentally block things that you didn't want to. No, what I'm saying is you can't ban everything associated with Nazis and nothing else in a LLM, because neural nets don't work like that and you're simply unable to ban something without influencing all the results. Which is worse than just banning info about Wernher von Braun. I may be wrong, considering my knowledge of neural networks is limited, but so far I got one downvote and no explanation...
  - quickslowdown 3y ago
    I think using a less charged example than "Nazis" would probably have helped avoid down voted & lack of engagement, but I understand why you chose it as an example & personally don't take issue with it, especially because you elaborated on why you picked them as an example. Just my $0.02

spaceman_2020 3y ago

I think once we get to GPT-5, people will just get used to their chatbots being capable of insensitive speech, and they’ll ignore it because the usefulness will be so high.

cbozeman 3y ago
If it can't do insensitive speech then the usefulness with not be so high. Jordan Peterson put it best, "In order to think, you have to risk being offensive."
- jart 3y ago
  It's better to think about something else that's boring which others won't care about, rather than focusing on hot button topics. Even if you want to touch the third rail, it's better to do it with jargon. For example, I've yet to meet an activist who's offended by phylogenesis and cladistics but I know many who'd take objection to things Peterson says.
  - quickslowdown 3y ago
    I'm one of them. I'm of the opinion that anything useful or worth hearing, I can hear from someone with less baggage. I don't know the ratio, but Peterson has added more bad to the world that his "good" ideas do not balance out. And after hearing him elaborate on his "bad" ideas, seeing the way he thinks & comes to conclusions, if I'm hearing something that sounds good from him, I would still be skeptical to the point of tuning it out entirely because of the source. I approach the things he says the same way I would an Alex Jones or Elon Musk or Joe Rogan. They don't have enough knowledge in the domains they like to speak in to be fully credible sources, and whatever credibility they might have is outweighed by the other shit they've said that's just outright wrong or intentionally misinformed. The source of the information matters to me, and anything any of these people or their ilk are putting out in the world can be found from someone else who is more worthy of trust. Peterson, like Elon Musk or Joe Rogan et al, have proven enough times they're willing to debase themselves putting nonsense into the world, nonsense they're fully aware is lies and/or harmful, and have burned whatever credibility they might have had. The people taking objection to things Jordan Peterson says may be throwing the baby out with the bathwater, but I would argue that Peterson himself cultivated this lack of trust. I will hear what he has to say from somebody more worthy of my trust & attention.