4 ms·

Can they, though…?

jasonlotito 10mo ago
Yes, they can. But... more importantly, they aren't even trying.
ebbi 10mo ago
What makes you think they can't? From my knowledge (albeit limited) about the way LLMs are set up, they most definitely have abilities to include guardrails of what can't be produced. ChatGPT has some responses to prompts which stops users from proceeding. And X specifically: there have many cases of X adjusting Grok where Grok was not following a particular narrative on political issues (won't get into specifics here). But it was very clear and visible. Grok had certain outputs. Outcry from certain segments. Grok posts deleted. Trying the same prompts resulted in a different result. So yeah, it's possible.
- Groxx 10mo ago
  Given how spectacular the failure of EVERY attempt to put guardrails on LLMs has been, across every single company selling LLM access, I'm not sure that's a reasonable belief. The guardrails have mostly worked. They have never ever been reliable.
- alex_suzuki 10mo ago
  From my (admittedly also limited) understanding, there’s no bulletproof way to say “do NOT generate X” as it’s not non-deterministic and you can’t reverse engineer and excise the CSAM-generating parts of a model. “AI jailbreak prompts” are a thing.
  - johnisgood 10mo ago
    So people just want to make it more difficult to achieve <insert bad thing>?
    - alex_suzuki 10mo ago
      Well it’s certainly horrible that they’re not even trying, but not surprising (I deleted my X account a long time ago). I’m just wondering if from a technical perspective it’s even possible to do it in a way that would 100% solve the problem, and not turn it into an arms race to find jailbreaks. To truly remove the capability from the model, or in its absence, have a perfect oracle judge the output and block it. The answer is currently no, I presume.
      - ebbi 10mo ago
        Again, I'm not the most technical, but I think we need to step back and look at this holistically. Given Grok's integration with X, there could be other methods of limiting the production and dissemination of CSAM. For arguments sake, let's assume Grok can't reliably have guardrails in place to stop CSAM. There could be second and third order review points where before an image is posted by Grok, another system could scan the image to verify whether it's CSAM or not, and if the confidence is low, then human intervention could come into play. I think the end goal here is prevention of CSAM production and dissemination, not just guardrails in an LLM and calling it a day.
- ux266478 10mo ago
  > they most definitely have abilities to include guardrails of what can't be produced. The problem is that these guardrails are trivially bypassed. At best you end up playing a losing treadmill game against adversarial prompting.
queenkjuul 10mo ago
Yes. One, they could just turn it off. Two, they got it to parrot all musk's politics, they clearly have a good grip on the thing.

Can they, though…?

Yes, they can. But... more importantly, they aren't even trying.

ebbi 10mo ago

What makes you think they can't? From my knowledge (albeit limited) about the way LLMs are set up, they most definitely have abilities to include guardrails of what can't be produced. ChatGPT has some responses to prompts which stops users from proceeding. And X specifically: there have many cases of X adjusting Grok where Grok was not following a particular narrative on political issues (won't get into specifics here). But it was very clear and visible. Grok had certain outputs. Outcry from certain segments. Grok posts deleted. Trying the same prompts resulted in a different result. So yeah, it's possible.

Groxx 10mo ago
Given how spectacular the failure of EVERY attempt to put guardrails on LLMs has been, across every single company selling LLM access, I'm not sure that's a reasonable belief. The guardrails have mostly worked. They have never ever been reliable.
alex_suzuki 10mo ago
From my (admittedly also limited) understanding, there’s no bulletproof way to say “do NOT generate X” as it’s not non-deterministic and you can’t reverse engineer and excise the CSAM-generating parts of a model. “AI jailbreak prompts” are a thing.
- johnisgood 10mo ago
  So people just want to make it more difficult to achieve <insert bad thing>?
  - alex_suzuki 10mo ago
    Well it’s certainly horrible that they’re not even trying, but not surprising (I deleted my X account a long time ago). I’m just wondering if from a technical perspective it’s even possible to do it in a way that would 100% solve the problem, and not turn it into an arms race to find jailbreaks. To truly remove the capability from the model, or in its absence, have a perfect oracle judge the output and block it. The answer is currently no, I presume.
    - ebbi 10mo ago
      Again, I'm not the most technical, but I think we need to step back and look at this holistically. Given Grok's integration with X, there could be other methods of limiting the production and dissemination of CSAM. For arguments sake, let's assume Grok can't reliably have guardrails in place to stop CSAM. There could be second and third order review points where before an image is posted by Grok, another system could scan the image to verify whether it's CSAM or not, and if the confidence is low, then human intervention could come into play. I think the end goal here is prevention of CSAM production and dissemination, not just guardrails in an LLM and calling it a day.
ux266478 10mo ago
> they most definitely have abilities to include guardrails of what can't be produced. The problem is that these guardrails are trivially bypassed. At best you end up playing a losing treadmill game against adversarial prompting.

queenkjuul 10mo ago

Yes. One, they could just turn it off. Two, they got it to parrot all musk's politics, they clearly have a good grip on the thing.