6 ms·

The AI labs look rather naive here. You can’t jump up and down screaming how amazing, powerful, and dangerous your new tech is and then act surprised and annoy

by cmiles8 4mo ago

The AI labs look rather naive here. You can’t jump up and down screaming how amazing, powerful, and dangerous your new tech is and then act surprised and annoyed when the government shows up looking to regulate it. Their new argument now seems be that this was marketing hype/fluff that backfired, in a pretty obvious and predicable way, and now they’re trying to reset the conversation.

tennfown 4mo ago
I’m way more concerned about the loons willing to throw absurd amounts of money at the clearly naive individuals.
xeonmc 4mo ago
[flagged]
cyanydeez 4mo ago
No. They got caught in a change in what it means to be "regulated". Regulation in a functional democracy: Cool, lets figure this out, write up a bill for us, do some research in congress, lets find something that makes sense. Regulation in a function fascism: Cool, wheres my bribe? My boots not shiny, lick it till I say stop. See, Anthropic wasn't licking enough boot when Biden got discharged and they thought Amazon and OpenAI and Elon were just going to let them capture a market without fealty to the boot.
- voidfunc 4mo ago
  This. Theres a lot of rude awakenings in the future for corporate executive types. They are no longer driving the train. Oh well.
- redsocksfan45 4mo ago
  [dead]
- calvinmorrison 4mo ago
  [flagged]
teaearlgraycold 4mo ago
This is 99% petty drama between the US government and Anthropic and 1% actual safety concerns.
- stvltvs 4mo ago
  But the paperclips! I'm skeptical about the existential threat of AI, but a lot of smart people have been beating that drum for so long that people are afraid.
- colonCapitalDee 4mo ago
  To be clear, this is petty drama *stirred up the US government*. It's not some sort of back and forth, the government is singling them out
  - mrandish 4mo ago
    And to add more background: The administration is targeting Anthropic because of the TOU / EULA conflict with the DoD from a couple of months ago. Anthropic restricts use of all their models for lethal combat planning and mass domestic surveillance. The DoD was, and still is, very pissed about this. While this Fable ban was issued from the Commerce Department, it's painfully obvious executive branch agencies are tightly coordinated from the White House. To be clear, I'm not saying there aren't legit security concerns around Fable's release. I think Andy Jassy did forward a concerning report about an apparent jailbreak in Fable, and he probably did so in good faith. The difference is if the same concern had arisen about a new model from OAI, Google, etc the action taken would not have been inventing an all-new, hyper-extreme punitive remedy and dropping it after 5p on a Friday under a very rare mechanism forcing Ant to comply in 90 mins or be subject to immediate arrest. And the "no non-U.S. citizens anywhere, anytime" restriction is functionally unprecedented. This is the Trump admin inventing new regulatory power that's never existed before and deploying it in a punitive way to demonstrate what can happen to those who aren't sufficiently cooperative with this administration. There are half a dozen less extreme levels of restriction, which already exist, and one of those would have been deemed sufficient had it been another company. That said, I'm certainly no Anthropic fanboy. Anthropic did play their initial Mythos self-restriction for PR value. But I think it's likely the Mythos self-restriction was a responsible action initially suggested by their AI safety team in good faith. Giving security researchers time to evaluate it and major companies time to test it against their code bases probably was reasonable and prudent. That doesn't mean it wasn't also good for PR and brand perception. I think there are senior people inside Anthropic who are genuinely concerned about AI safety. Personally, I don't have the expertise to gauge if those concerns are justified, but I believe they believe it. I also think there are senior people at Anthropic who are focused more on building the business, doing the IPO and "winning" the silicon valley game. All of these things can be simultaneously true.
bellowsgulch 4mo ago
[flagged]
- binary132 4mo ago
  that seems like possibly the most unlikely outcome
  - deleted 4mo ago
    [deleted]
dofm 4mo ago
Dog caught the car
micromacrofoot 4mo ago
I suspect they're taking this as a win either way, because they're still plastering "Fable 5 unavailable" on their product and using it as an opportunity to keep themselves in the spotlight as they head to IPO. There's really not even a ban here, they could slot in Fable under the Opus label and no one would really be able to tell. It's all part of the same show to pump up valuation.
- bloppe 4mo ago
  I bet they will do a touch of RLHF and re-naming the moment OAI releases a comparable model. Otherwise, sure, they can just bask in the drama for a bit.
mips_avatar 4mo ago
Well Anthropic would love some regulatory capture.
Bender 4mo ago
Also a good wake-up call for investors as these big players can be benched at any moment.
- reje 4mo ago
  I can’t take anyone seriously who thought otherwise. You think you can become more powerful so much so the govt questions its own power? Don’t be stupid. They will simply send in the army to first seize the assets and then nationalise. It almost seems as if very few people actually understand how the world works. If the govt thinks this is the tech to end all future tech, you think future money flows for invesment matter? Hahaha. No
drtz 4mo ago
> You can’t jump up and down screaming how amazing, powerful, and dangerous your new tech is and then act surprised and annoyed when the government shows up looking to regulate it. It's entirely possible that models could be "dangerous" to fully release to the general public without guardrails and at the same time the government majorly overreacted in this case. Releasing Mythos to selected researchers and companies at least gives those researchers a head start at addressing vulnerabilities before the model hits mainstream.
- foo-bar-baz529 4mo ago
  Then why did curl only find one new vulnerability thanks to Mythos, and a low-priority one at that? It’s clear that other models are quite capable of finding largely the same vulnerabilities, and that the main key is simply running a frontier model in a good harness to find vulnerabilities.
  - ChadNauseam 4mo ago
    > Then why did curl only find one new vulnerability thanks to Mythos Maybe there weren't that many serious vulnerabilities in curl? It's like asking why it didn't find any vulnerabilities in fn main() {println!("hello, world");}. Anyway, people who have used it seem to say that Mythos was better than other models at creating exploits. From cloudflare https://blog.cloudflare.com/cyber-frontier-models/ https://blog.cloudflare.com/cyber-frontier-models/ > When we ran other frontier models through the same harness, they found a fair number of the same underlying bugs, and in some cases they got further than we expected on the reasoning side too. Where they fell short was at the point of stitching the pieces together. A model would identify an interesting bug, write a thoughtful description of why it mattered, and then stop, leaving the actual chain unfinished and the question of exploitability open. What changed with Mythos Preview is that a model can now take those low-severity bugs (which would traditionally sit invisible in a backlog) and chain them into a single, more severe exploit.
    - Terretta 4mo ago
      > Mythos was better than other models at creating exploits. Not a fan of this phrasing, prefer "discovering exploits". It makes it clearer the problem was already there, latent. Minor vocab diff, but important to better contextualize the present situation.
ChadNauseam 4mo ago
> You can’t jump up and down screaming how amazing, powerful, and dangerous your new tech is and then act surprised and annoyed when the government shows up looking to regulate it. True, you can't. But, you can think certain regulations are helpful and certain other regulations are not. And you can be annoyed when unhelpful "regulations" are put in place. This is like if I say that pitbulls are dangerous, and then the government comes and shoots my pitbull, who I've spent a lot of effort training to not be dangerous. Then you say "well you said pitbulls were dangerous, so you can't really complain." Well, I can complain because If you took me seriously, you wouldn't have responded by shooting only my pitbull! Think of what incentives this creates for other people. Do you think that OpenAI will be candid about the possible dangers of their technology now? They might not even release it now, seeing that Anthropic releasing their model was what got it export-controlled.
- dogleash 4mo ago
  The act of shooting the pitbull makes for good dramatics, but you would get zero sympathy from me if your local government banned pitbull ownership. e.g. Ontario bans pitbulls. I don't have a problem with that.
  - sheepscreek 4mo ago
    You don't need/use pitbulls, but what if you (and many many others) wanted and needed Fable? I for one was late to the bandwagon, and when I had the use-case for it - the govt pulled the rug. So yeah, I'm a bit salty about the whole endeavour. I will also say that the security concerns are probably very real (and they have been from the day ChatGPT-3.5 came our). I guess I can be salty about it and still be wrong from their perspective. The govt likely understands the fragility of their infrastructure better than us and is likely aware what this could unleash for their systems.
  - ethbr1 4mo ago
    Because it was the basis for the analogy: breed-based dog bans are idiotic, given mixed genetics, temperament, and training. Said as the owner of a pitbull, who is the sweetest and gentlest dog I've owned. And I've had multiple labradors.
jubilanti 4mo ago
Well it is reasonable to expect the bare minimum of due process, or you should be able to from a government that claims to be so committed to the rule of law.
- halJordan 4mo ago
  National security laws often don't require such. It's easy to meet due process when the process is 0. Voters have no one to blame but themselves
Grimblewald 4mo ago
Remember strawberry? I do. Rememver gpt2 hype? I do. the hype isnt real, its marketing designed to inderectly siphon capital from the less informed. The current USA government is no stranger to a grift, so they'll get it. Not that I agree with the practumice, this bubble will hurt quite badly when it goes, but at least AI fundamentally does deliver something useful, even if it isnt infinite value as typically promised. Think dotcom bubble and the hype and promises made surrouning that. Tge hyper will pass, the bubble will pop, and life goes back to normal as ai becomes part of the mundane everyday human environment. Like websites and domains, some will be used well, some for evil, not everyone needs it, and as we continue to move towards energy being our fundamental unit of value / exchange, if we cant make these models way more efdicient then their use case will be rather limited in scope.

by cmiles8 4mo ago

tennfown 4mo ago

I’m way more concerned about the loons willing to throw absurd amounts of money at the clearly naive individuals.

xeonmc 4mo ago

[flagged]

cyanydeez 4mo ago

No. They got caught in a change in what it means to be "regulated". Regulation in a functional democracy: Cool, lets figure this out, write up a bill for us, do some research in congress, lets find something that makes sense. Regulation in a function fascism: Cool, wheres my bribe? My boots not shiny, lick it till I say stop. See, Anthropic wasn't licking enough boot when Biden got discharged and they thought Amazon and OpenAI and Elon were just going to let them capture a market without fealty to the boot.

voidfunc 4mo ago
This. Theres a lot of rude awakenings in the future for corporate executive types. They are no longer driving the train. Oh well.
redsocksfan45 4mo ago
[dead]
calvinmorrison 4mo ago
[flagged]

teaearlgraycold 4mo ago

This is 99% petty drama between the US government and Anthropic and 1% actual safety concerns.

stvltvs 4mo ago
But the paperclips! I'm skeptical about the existential threat of AI, but a lot of smart people have been beating that drum for so long that people are afraid.
colonCapitalDee 4mo ago
To be clear, this is petty drama *stirred up the US government*. It's not some sort of back and forth, the government is singling them out
- mrandish 4mo ago
  And to add more background: The administration is targeting Anthropic because of the TOU / EULA conflict with the DoD from a couple of months ago. Anthropic restricts use of all their models for lethal combat planning and mass domestic surveillance. The DoD was, and still is, very pissed about this. While this Fable ban was issued from the Commerce Department, it's painfully obvious executive branch agencies are tightly coordinated from the White House. To be clear, I'm not saying there aren't legit security concerns around Fable's release. I think Andy Jassy did forward a concerning report about an apparent jailbreak in Fable, and he probably did so in good faith. The difference is if the same concern had arisen about a new model from OAI, Google, etc the action taken would not have been inventing an all-new, hyper-extreme punitive remedy and dropping it after 5p on a Friday under a very rare mechanism forcing Ant to comply in 90 mins or be subject to immediate arrest. And the "no non-U.S. citizens anywhere, anytime" restriction is functionally unprecedented. This is the Trump admin inventing new regulatory power that's never existed before and deploying it in a punitive way to demonstrate what can happen to those who aren't sufficiently cooperative with this administration. There are half a dozen less extreme levels of restriction, which already exist, and one of those would have been deemed sufficient had it been another company. That said, I'm certainly no Anthropic fanboy. Anthropic did play their initial Mythos self-restriction for PR value. But I think it's likely the Mythos self-restriction was a responsible action initially suggested by their AI safety team in good faith. Giving security researchers time to evaluate it and major companies time to test it against their code bases probably was reasonable and prudent. That doesn't mean it wasn't also good for PR and brand perception. I think there are senior people inside Anthropic who are genuinely concerned about AI safety. Personally, I don't have the expertise to gauge if those concerns are justified, but I believe they believe it. I also think there are senior people at Anthropic who are focused more on building the business, doing the IPO and "winning" the silicon valley game. All of these things can be simultaneously true.

bellowsgulch 4mo ago

[flagged]

binary132 4mo ago
that seems like possibly the most unlikely outcome
- deleted 4mo ago
  [deleted]

dofm 4mo ago

Dog caught the car

micromacrofoot 4mo ago

I suspect they're taking this as a win either way, because they're still plastering "Fable 5 unavailable" on their product and using it as an opportunity to keep themselves in the spotlight as they head to IPO. There's really not even a ban here, they could slot in Fable under the Opus label and no one would really be able to tell. It's all part of the same show to pump up valuation.

bloppe 4mo ago
I bet they will do a touch of RLHF and re-naming the moment OAI releases a comparable model. Otherwise, sure, they can just bask in the drama for a bit.

mips_avatar 4mo ago

Well Anthropic would love some regulatory capture.

Bender 4mo ago

Also a good wake-up call for investors as these big players can be benched at any moment.

reje 4mo ago
I can’t take anyone seriously who thought otherwise. You think you can become more powerful so much so the govt questions its own power? Don’t be stupid. They will simply send in the army to first seize the assets and then nationalise. It almost seems as if very few people actually understand how the world works. If the govt thinks this is the tech to end all future tech, you think future money flows for invesment matter? Hahaha. No

drtz 4mo ago

> You can’t jump up and down screaming how amazing, powerful, and dangerous your new tech is and then act surprised and annoyed when the government shows up looking to regulate it. It's entirely possible that models could be "dangerous" to fully release to the general public without guardrails and at the same time the government majorly overreacted in this case. Releasing Mythos to selected researchers and companies at least gives those researchers a head start at addressing vulnerabilities before the model hits mainstream.

foo-bar-baz529 4mo ago
Then why did curl only find one new vulnerability thanks to Mythos, and a low-priority one at that? It’s clear that other models are quite capable of finding largely the same vulnerabilities, and that the main key is simply running a frontier model in a good harness to find vulnerabilities.
- ChadNauseam 4mo ago
  > Then why did curl only find one new vulnerability thanks to Mythos Maybe there weren't that many serious vulnerabilities in curl? It's like asking why it didn't find any vulnerabilities in fn main() {println!("hello, world");}. Anyway, people who have used it seem to say that Mythos was better than other models at creating exploits. From cloudflare https://blog.cloudflare.com/cyber-frontier-models/ https://blog.cloudflare.com/cyber-frontier-models/ > When we ran other frontier models through the same harness, they found a fair number of the same underlying bugs, and in some cases they got further than we expected on the reasoning side too. Where they fell short was at the point of stitching the pieces together. A model would identify an interesting bug, write a thoughtful description of why it mattered, and then stop, leaving the actual chain unfinished and the question of exploitability open. What changed with Mythos Preview is that a model can now take those low-severity bugs (which would traditionally sit invisible in a backlog) and chain them into a single, more severe exploit.
  - Terretta 4mo ago
    > Mythos was better than other models at creating exploits. Not a fan of this phrasing, prefer "discovering exploits". It makes it clearer the problem was already there, latent. Minor vocab diff, but important to better contextualize the present situation.

ChadNauseam 4mo ago

> You can’t jump up and down screaming how amazing, powerful, and dangerous your new tech is and then act surprised and annoyed when the government shows up looking to regulate it. True, you can't. But, you can think certain regulations are helpful and certain other regulations are not. And you can be annoyed when unhelpful "regulations" are put in place. This is like if I say that pitbulls are dangerous, and then the government comes and shoots my pitbull, who I've spent a lot of effort training to not be dangerous. Then you say "well you said pitbulls were dangerous, so you can't really complain." Well, I can complain because If you took me seriously, you wouldn't have responded by shooting only my pitbull! Think of what incentives this creates for other people. Do you think that OpenAI will be candid about the possible dangers of their technology now? They might not even release it now, seeing that Anthropic releasing their model was what got it export-controlled.

dogleash 4mo ago
The act of shooting the pitbull makes for good dramatics, but you would get zero sympathy from me if your local government banned pitbull ownership. e.g. Ontario bans pitbulls. I don't have a problem with that.
- sheepscreek 4mo ago
  You don't need/use pitbulls, but what if you (and many many others) wanted and needed Fable? I for one was late to the bandwagon, and when I had the use-case for it - the govt pulled the rug. So yeah, I'm a bit salty about the whole endeavour. I will also say that the security concerns are probably very real (and they have been from the day ChatGPT-3.5 came our). I guess I can be salty about it and still be wrong from their perspective. The govt likely understands the fragility of their infrastructure better than us and is likely aware what this could unleash for their systems.
- ethbr1 4mo ago
  Because it was the basis for the analogy: breed-based dog bans are idiotic, given mixed genetics, temperament, and training. Said as the owner of a pitbull, who is the sweetest and gentlest dog I've owned. And I've had multiple labradors.

jubilanti 4mo ago

Well it is reasonable to expect the bare minimum of due process, or you should be able to from a government that claims to be so committed to the rule of law.

halJordan 4mo ago
National security laws often don't require such. It's easy to meet due process when the process is 0. Voters have no one to blame but themselves

Grimblewald 4mo ago

Remember strawberry? I do. Rememver gpt2 hype? I do. the hype isnt real, its marketing designed to inderectly siphon capital from the less informed. The current USA government is no stranger to a grift, so they'll get it. Not that I agree with the practumice, this bubble will hurt quite badly when it goes, but at least AI fundamentally does deliver something useful, even if it isnt infinite value as typically promised. Think dotcom bubble and the hype and promises made surrouning that. Tge hyper will pass, the bubble will pop, and life goes back to normal as ai becomes part of the mundane everyday human environment. Like websites and domains, some will be used well, some for evil, not everyone needs it, and as we continue to move towards energy being our fundamental unit of value / exchange, if we cant make these models way more efdicient then their use case will be rather limited in scope.