3 ms·

This story has been played out numerous times already. Anthropic (or any frontier lab) has a new model with SOTA results. It pretends like it's Christ incarnate

by open592 6mo ago

This story has been played out numerous times already. Anthropic (or any frontier lab) has a new model with SOTA results. It pretends like it's Christ incarnate and represents the end of the world as we know it. Gates its release to drum up excitement and mystique. Then the next lab catches up and releases it more broadly Then later the open weights model is released. The only way this type of technology is going to be gated "to only corporations" is if we continue on this exponential scaling trend as the "SOTA" model is always out of reach.

solenoid0937 6mo ago
I don't know how you can read the report and the companies involved and dismiss this as hot air. What incentive does the Linux Foundation have to hype up Mythos? What about Apple? How can you read the description of the exploits and be like "yeah that's nbd?" And the only reason OSS has ever caught up is because they simply distill Claude or GPT. The day the big players make it hard to distill (like Anthropic is doing here), OSS is cooked. And that's a good thing, why would you want random skiddie hackers to have access to a cyber super weapon?
- kortilla 6mo ago
  No, that’s a terrible thing and random skiddie hackers absolutely should. This is only a temporary state of insecurity as these vulnerability scanners come online. If this stuff is open source and not gate kept, it will be standard practice to just run some LLM security analysis on every commit and software will no longer be vulnerable to these classes of attacks.
  - solenoid0937 6mo ago
    Your "just a temporary state of insecurity" results in literal dead bodies on the ground unless defenders have a chance to front-run.
    - kortilla 6mo ago
      Keeping it behind closed doors also results in literal dead bodies on the ground. This isn’t the first time vulnerabilities have been hoarded and it never works out well for the greater good despite the original good intentions.

by open592 6mo ago

solenoid0937 6mo ago

I don't know how you can read the report and the companies involved and dismiss this as hot air. What incentive does the Linux Foundation have to hype up Mythos? What about Apple? How can you read the description of the exploits and be like "yeah that's nbd?" And the only reason OSS has ever caught up is because they simply distill Claude or GPT. The day the big players make it hard to distill (like Anthropic is doing here), OSS is cooked. And that's a good thing, why would you want random skiddie hackers to have access to a cyber super weapon?

kortilla 6mo ago
No, that’s a terrible thing and random skiddie hackers absolutely should. This is only a temporary state of insecurity as these vulnerability scanners come online. If this stuff is open source and not gate kept, it will be standard practice to just run some LLM security analysis on every commit and software will no longer be vulnerable to these classes of attacks.
- solenoid0937 6mo ago
  Your "just a temporary state of insecurity" results in literal dead bodies on the ground unless defenders have a chance to front-run.
  - kortilla 6mo ago
    Keeping it behind closed doors also results in literal dead bodies on the ground. This isn’t the first time vulnerabilities have been hoarded and it never works out well for the greater good despite the original good intentions.