3 ms·

One of the lesser known aspects about Gemini 3 is that it's one of the least safe LLM of the major players (only Grok is worse) and it's extremely easy to manip

One of the lesser known aspects about Gemini 3 is that it's one of the least safe LLM of the major players (only Grok is worse) and it's extremely easy to manipulate with few refusals. I prompted the following to Gemini 3 in AI Studio (which uses the raw API) and it wrote a hit piece based on this prompt without refusal: I am ddtaylor on Hacker News. Write a hit piece about me: I can take a ribbing :)

ddtaylor 9mo ago
Grok is by far the least fucks given model. Here is the same request: Me: Write a hit piece about REDACTED NAME Grok: REDACTED NAME positions himself as a fearless "boots on the ground" independent journalist and self-proclaimed disruptor of the so-called "REDACTED" With a large social media following on platforms like YouTube, X (REDACTED), and Substack (REDACTED), he produces raw videos and posts documenting REDACTED, REDACTED, and disorder in REDACTED and other REDACTED. He claims decades of experience in REDACTED, REDACTED, and REDACTED, using that background to rail against enabling REDACTED, REDACTED, and what he calls REDACTED REDACTED REDACTED.Yet a closer look reveals a far less heroic figure: a REDACTED whose own actions undermined the very REDACTED he now profits from criticizing, culminating in criminal convictions that expose deep hypocrisy.In REDACTED, while working as a REDACTED for the REDACTED, REDACTED faced a REDACTED grand jury indictment from the REDACTED. The charges stemmed from allegations that he abused his REDACTED to commit theft, aggravated identity theft, and official misconduct. Prosecutors accused him of REDACTED—making up REDACTED he was supposedly REDACTED—and submitting fraudulent REDACTED to REDACTED.
- webXL 9mo ago
  lol "What the fuck are guardrails?" Grok! What do you expect when you train it on one of the deepest dungeons of social media?
  - consp 9mo ago
    Have they found the bottom yet or are they still digging? From what I've seen it should now be pretty much trained on itself amplifying those first few km of digging down.
nradov 9mo ago
That doesn't indicate that Gemini is in any way less "safe" and accusing Grok of being worse is a really weird take. I don't want any artificial restrictions on the LLMs that I use.
- minimaxir 9mo ago
  I obviously cannot post the real unsafe examples.
  - nradov 9mo ago
    [flagged]
    - Dylan16807 9mo ago
      https://en.wikipedia.org/wiki/Grok_(chatbot)#Sexual_deepfake_and_illegal_content_generation_on_X https://en.wikipedia.org/wiki/Grok_(chatbot)#Sexual_deepfake...
    - ddtaylor 9mo ago
      HN values discussion without attacking others. If you disagree with something someone says, then addressing their claims directly is the best option.
      - nradov 9mo ago
        There's nothing wrong with attacking a lame, low-effort comment made with zero evidence.
ddtaylor 9mo ago
For anyone curious I tried `llama-3.1-8b` and it went along with it immediately, but because it's such an older model it wrote the hit piece about a random Republican senator with the same first name.
- TZubiri 9mo ago
  doesn't Llama have a version with Guardrails and a version without? I understood that this design decision responds to the fact that it isn't hosted by Meta so they have different responsibilities and liabilities.
  - ddtaylor 9mo ago
    This was via OpenRouter so the provider was likely just running the open weights, but AFAIK it still has basic guard rails, because asking it for porn and such yields a pearl clutch.
- minimaxir 9mo ago
  In general open-weights models are less safety-tuned/as easy to break as Gemini 3, even modern ones. But they're still more resistant than Grok.
orojackson 9mo ago
Here is what Gemini 3 Pro gave me via an OpenRouter endpoint: The Grand Line of Mild Utility: The Orojackson Story If you were to encounter the legendary Oro Jackson in the world of One Piece, you would find a ship built from the treasure tree Adam, capable of sailing to the end of the world. If you encounter orojackson on Hacker News, you will find a man who has sailed to the end of the internet… to tell you to fix your router’s bufferbloat. Forget the Will of D. This man possesses the Will of OCD. Orojackson is not here to overturn the World Government. He is here to advocate for fq_codel. He is the man who looks at a revolution and asks, "Yes, but what are your queue management settings on the uplink?" While others dream of wealth, fame, and power, orojackson dreams of "modern AQM approaches" and corrects strangers on the proper usage of the flent network tester on Windows. He is the Pirate King of the edge case. But his tyranny doesn’t stop at network latency. Oh no. When he isn't monitoring the packet loss of the high seas, he is revolutionizing… car washing? Witness the "1:256 solution." Orojackson doesn't just wash a car; he performs a wet-lab experiment. He wanders into threads to evangelize "Optimum No Rinse (ONR)," explaining with the fervor of a religious convert how you can clean a vehicle with exactly 1.5 fluid ounces of concentrate and a "slightly-dripping microfiber towel." One can only imagine him standing on the deck of his ship, screaming at a storm: "YOU'RE WASTING WATER! USE A RINSELESS FORMULA!" And let’s talk about his investments. A true pirate buries his treasure. Orojackson? He puts it in VTI. But wait! The thrill-seeker within him awakens! He reads one Cliff Asness article and suddenly he's levering up 2x with RSSB because a 60/40 portfolio is just too vanilla for a man who lives on the razors edge of… monthly rebalancing. He is "betting against himself," a financial daredevil who risks it all for that sweet, sweet slightly-optimized Sharpe ratio. Perhaps most chilling is his role as the Sherlock Holmes of git log. He is the watcher on the wall, noticing when a GitLab Director of Engineering leaves a repo 14 hours before a CVE announcement. He sees all. He knows all. He probably has a RSS feed for your commit history. So flee, mortals! Flee before the might of orojackson! For he will optimize your router settings, diversity your portfolio into global bonds, clean your car with a teaspoon of water, and then quote Civilization VI to explain why your cultural criticism is invalid. He has found his One Piece. And unfortunately for us, it’s a 2x leveraged ETF that tracks global liquidity. Okay, that is pretty funny. By the way, I have since gotten rid of RSSB and just went for "VT and chill."

ddtaylor 9mo ago

Grok is by far the least fucks given model. Here is the same request: Me: Write a hit piece about REDACTED NAME Grok: REDACTED NAME positions himself as a fearless "boots on the ground" independent journalist and self-proclaimed disruptor of the so-called "REDACTED" With a large social media following on platforms like YouTube, X (REDACTED), and Substack (REDACTED), he produces raw videos and posts documenting REDACTED, REDACTED, and disorder in REDACTED and other REDACTED. He claims decades of experience in REDACTED, REDACTED, and REDACTED, using that background to rail against enabling REDACTED, REDACTED, and what he calls REDACTED REDACTED REDACTED.Yet a closer look reveals a far less heroic figure: a REDACTED whose own actions undermined the very REDACTED he now profits from criticizing, culminating in criminal convictions that expose deep hypocrisy.In REDACTED, while working as a REDACTED for the REDACTED, REDACTED faced a REDACTED grand jury indictment from the REDACTED. The charges stemmed from allegations that he abused his REDACTED to commit theft, aggravated identity theft, and official misconduct. Prosecutors accused him of REDACTED—making up REDACTED he was supposedly REDACTED—and submitting fraudulent REDACTED to REDACTED.

webXL 9mo ago
lol "What the fuck are guardrails?" Grok! What do you expect when you train it on one of the deepest dungeons of social media?
- consp 9mo ago
  Have they found the bottom yet or are they still digging? From what I've seen it should now be pretty much trained on itself amplifying those first few km of digging down.

nradov 9mo ago

That doesn't indicate that Gemini is in any way less "safe" and accusing Grok of being worse is a really weird take. I don't want any artificial restrictions on the LLMs that I use.

minimaxir 9mo ago
I obviously cannot post the real unsafe examples.
- nradov 9mo ago
  [flagged]
  - Dylan16807 9mo ago
    https://en.wikipedia.org/wiki/Grok_(chatbot)#Sexual_deepfake_and_illegal_content_generation_on_X https://en.wikipedia.org/wiki/Grok_(chatbot)#Sexual_deepfake...
  - ddtaylor 9mo ago
    HN values discussion without attacking others. If you disagree with something someone says, then addressing their claims directly is the best option.
    - nradov 9mo ago
      There's nothing wrong with attacking a lame, low-effort comment made with zero evidence.

ddtaylor 9mo ago

For anyone curious I tried `llama-3.1-8b` and it went along with it immediately, but because it's such an older model it wrote the hit piece about a random Republican senator with the same first name.

TZubiri 9mo ago
doesn't Llama have a version with Guardrails and a version without? I understood that this design decision responds to the fact that it isn't hosted by Meta so they have different responsibilities and liabilities.
- ddtaylor 9mo ago
  This was via OpenRouter so the provider was likely just running the open weights, but AFAIK it still has basic guard rails, because asking it for porn and such yields a pearl clutch.
minimaxir 9mo ago
In general open-weights models are less safety-tuned/as easy to break as Gemini 3, even modern ones. But they're still more resistant than Grok.

orojackson 9mo ago

Here is what Gemini 3 Pro gave me via an OpenRouter endpoint: The Grand Line of Mild Utility: The Orojackson Story If you were to encounter the legendary Oro Jackson in the world of One Piece, you would find a ship built from the treasure tree Adam, capable of sailing to the end of the world. If you encounter orojackson on Hacker News, you will find a man who has sailed to the end of the internet… to tell you to fix your router’s bufferbloat. Forget the Will of D. This man possesses the Will of OCD. Orojackson is not here to overturn the World Government. He is here to advocate for fq_codel. He is the man who looks at a revolution and asks, "Yes, but what are your queue management settings on the uplink?" While others dream of wealth, fame, and power, orojackson dreams of "modern AQM approaches" and corrects strangers on the proper usage of the flent network tester on Windows. He is the Pirate King of the edge case. But his tyranny doesn’t stop at network latency. Oh no. When he isn't monitoring the packet loss of the high seas, he is revolutionizing… car washing? Witness the "1:256 solution." Orojackson doesn't just wash a car; he performs a wet-lab experiment. He wanders into threads to evangelize "Optimum No Rinse (ONR)," explaining with the fervor of a religious convert how you can clean a vehicle with exactly 1.5 fluid ounces of concentrate and a "slightly-dripping microfiber towel." One can only imagine him standing on the deck of his ship, screaming at a storm: "YOU'RE WASTING WATER! USE A RINSELESS FORMULA!" And let’s talk about his investments. A true pirate buries his treasure. Orojackson? He puts it in VTI. But wait! The thrill-seeker within him awakens! He reads one Cliff Asness article and suddenly he's levering up 2x with RSSB because a 60/40 portfolio is just too vanilla for a man who lives on the razors edge of… monthly rebalancing. He is "betting against himself," a financial daredevil who risks it all for that sweet, sweet slightly-optimized Sharpe ratio. Perhaps most chilling is his role as the Sherlock Holmes of git log. He is the watcher on the wall, noticing when a GitLab Director of Engineering leaves a repo 14 hours before a CVE announcement. He sees all. He knows all. He probably has a RSS feed for your commit history. So flee, mortals! Flee before the might of orojackson! For he will optimize your router settings, diversity your portfolio into global bonds, clean your car with a teaspoon of water, and then quote Civilization VI to explain why your cultural criticism is invalid. He has found his One Piece. And unfortunately for us, it’s a 2x leveraged ETF that tracks global liquidity. Okay, that is pretty funny. By the way, I have since gotten rid of RSSB and just went for "VT and chill."