5 ms·

To everyone who believes that this is some nefarious plot by CEOs to bring about regulation or to hide that they've hit a wall or to make their models seem more

To everyone who believes that this is some nefarious plot by CEOs to bring about regulation or to hide that they've hit a wall or to make their models seem more capable or whatever: The AI risk arguments are philosophical arguments that are 20-30 years old and don't have anything to do with the current corporate landscape. They were thought of by people who really care about the issue and all the "gotcha" lines have well-developed counters. Of course they might be stuck in their ideology but they deserve a bit more credit. I don't doubt that AI CEOs can profit from a crisis and that they don't necessarily have our best interests at heart. But this is the same way governments during COVID tested how much they can restrain freedom. This doesn't mean COVID doesn't exist or that vaccines are evil or ineffective or whatever. Same way here, if there is an advantage to CEOs to regulate things doesn't mean AIs can't kill us all and that regulation might bring us more time to solve alignment. Also: I accept I might be wrong and AI risk might be a fake issue and/or alignment might be impossible, but please bring better arguments if you want to convince me.

iLoveOncall 1mo ago
> To everyone who believes that this is some nefarious plot by CEOs to bring about regulation or to hide that they've hit a wall or whatever It's this but also much more that those insiders just want to pump the value of their stocks by making it seem like the unreleased models are much stronger than they actually are. OpenAI said GPT-2 was too powerful to release, Antropic said Mythos was too powerful to release. Both are GA now and everyone has long realized those supposed fears were just lies.
- embedding-shape 1mo ago
  > OpenAI said GPT-2 was too powerful to release, Antropic said Mythos was too powerful to release. Both are GA now and everyone has long realized those supposed fears were just lies. Is it possible they weren't "lies" but they were still "wrong" yet still they felt they were possibly too powerful to release? It's like there is two modes "Right and truthful and honest" or "Wrong, liar and dishonest" and there is never any nuance or understanding that people can be truthful yet wrong.
  - iLoveOncall 1mo ago
    No it's not possible. Scam Altman is a serial liar, even long before his OpenAI days.
- bananaflag 1mo ago
  > OpenAI said GPT-2 was too powerful to release Because they thought such a text-generating machine will bring about a golden age of fake news. Seven years later I have relatives who send me every day some long-winded AI-generated bullshit. And I have seen with my own eyes how this sort of thing influences elections. > Antropic said Mythos was too powerful to release. Both are GA now Mythos was too powerful to release, hence it is not GA now. They released Fable, which is Mythos plus guardrails.
  - deleted 1mo ago
    [deleted]
  - tpm 1mo ago
    > Seven years later I have relatives who send me every day some long-winded AI-generated bullshit. So not releasing the weights at that time didn't make the world safer, but did help to transform OpenAI into a commercial entity. Why should we trust them now?
  - iLoveOncall 1mo ago
    Oh ok I assumed Mythos was GA because I have access to it in Amazon despite not being in a security role at all. I guess it's just pay (enough) to play. Anyway, Fable is supposedly as good (if not better) in programming and it's very far from the paradigm shift that was promised.
xelxebar 1mo ago
Thank you for the epistemic humility here. It's a breath of fresh air amongst the polarized declarations we seem to have upregulated amongst ourselves for whatever reason these days.
- pizza234 1mo ago
  The problem of the polarized camps is that actually they're only on the surface opposite; in reality, they can perfectly cohexist: AI can enable major technological breakthroughs - but that doesn't exclude that can bring doom at the same time.
embedding-shape 1mo ago
> But this is the same way governments during COVID tested how much they can restrain freedom So AI CEOs don't have our best interests at heart, but the people who got voted for, who supposedly does have our best interests at heart, is secretely engaging in plots to "test how much they can restrain freedom"? I'm sorry, but this seems backwards. You're trusting the leaders of for-profit companies to do right regardless of their skewed incentives, and then the others are the ones to distrust. Sounds like you might want to move country is this is really how you feel.
- bananaflag 1mo ago
  > You're trusting the leaders of for-profit companies to do right regardless of their skewed incentives I'm not trusting them at all. I have seen for years how they don't do anything about alignment. OpenAI has disbanded countless alignment teams. But if their skewed incentives are for once bringing them into the right direction, I'm not gonna stand against them.
tpm 1mo ago
This looks like halfway through your comment you realised it is indeed a nefarious plot by the CEOs, reusing existing arguments to further their own goals. > The AI risk arguments ... are 20-30 years old Yeah so climate change is known to cause many excess deaths already now and we know pretty precisely what we have to do for decades, and we only do some bare minimum if anything at all. But now there is a huge PR offensive to do something about a 'risk' that until now caused zero deaths, is not certain to exist at all and is stoked not by concerned outsiders but by people certain to profit handsomely from heavy regulation of their industry. I'd say it's not opponents who need better arguments.
madaxe_again 1mo ago
They aren’t 20-30 years old - they are 160 years old. Samuel Butler wrote Erewhon in 1872 - which was based on an essay he wrote about RSI and superintelligence - in 1863.
- bananaflag 1mo ago
  I know, yes. I was talking about the current Bostrom/Yudkowsky iteration. Re Butler, you may enjoy this Chesterton pastiche if you haven't already: https://slatestarcodex.com/2017/04/01/g-k-chesterton-on-ai-risk/ https://slatestarcodex.com/2017/04/01/g-k-chesterton-on-ai-r...
  - madaxe_again 1mo ago
    Thanks - that was an excellent read!
raverbashing 1mo ago
I think it's a much simpler reason, they're running out of money to have the rat race run at the current pace
- dalemhurley 1mo ago
  Yep
pizza234 1mo ago
> The AI risk arguments are philosophical arguments that are 20-30 years old This a false and uninformed take. As a starting point, you need to read and understand: 1. analyses of the AI incidents of the last months 2. latest advancements (e.g. the AI intern at OpenAI) 3. the problem of deceptive alignment which you clearly haven't done. After that, the AI problem will be actually very concrete; once the AI will: 1. have superhuman cognitive abilities (and it will) 2. will be inseparable from humanity and/or have the means to replicate will humanity be able to contain/control it? The answer is sadly very simple.
- bananaflag 1mo ago
  > which you clearly haven't done I have read all that, I follow e.g. Zvi Mowshowitz's blog.
- PestoDiRucola 1mo ago
  > will be inseparable from humanity and/or have the means to replicate How will this happen if you need insane amounts of compute to run these models? Where will the models replicate themselves by taking up PBs of space without anyone noticing?
  - pizza234 1mo ago
    A few observations: - hardware capacity keeps growing - doom scenarios don't require fat clients; a sufficiently large number of AI servers deeply intertwined with essential services can be enough - autonomous systems are currently employed in weapons (e.g. drones) already - integration of AIs/LLMs is guaranteed to be the next step It's impossible to predict how exactly how things will develop (e.g. transfer large amounts of data; who knows, maybe it won't be necessary and/or maybe internet will be massively faster), but the building blocks are there.
- dalemhurley 1mo ago
  An intern who was there barely a minute, who has said nothing of substance. Surely we can get a better source with concrete evidence rather than the vibes of junior burger.

iLoveOncall 1mo ago

> To everyone who believes that this is some nefarious plot by CEOs to bring about regulation or to hide that they've hit a wall or whatever It's this but also much more that those insiders just want to pump the value of their stocks by making it seem like the unreleased models are much stronger than they actually are. OpenAI said GPT-2 was too powerful to release, Antropic said Mythos was too powerful to release. Both are GA now and everyone has long realized those supposed fears were just lies.

embedding-shape 1mo ago
> OpenAI said GPT-2 was too powerful to release, Antropic said Mythos was too powerful to release. Both are GA now and everyone has long realized those supposed fears were just lies. Is it possible they weren't "lies" but they were still "wrong" yet still they felt they were possibly too powerful to release? It's like there is two modes "Right and truthful and honest" or "Wrong, liar and dishonest" and there is never any nuance or understanding that people can be truthful yet wrong.
- iLoveOncall 1mo ago
  No it's not possible. Scam Altman is a serial liar, even long before his OpenAI days.
bananaflag 1mo ago
> OpenAI said GPT-2 was too powerful to release Because they thought such a text-generating machine will bring about a golden age of fake news. Seven years later I have relatives who send me every day some long-winded AI-generated bullshit. And I have seen with my own eyes how this sort of thing influences elections. > Antropic said Mythos was too powerful to release. Both are GA now Mythos was too powerful to release, hence it is not GA now. They released Fable, which is Mythos plus guardrails.
- deleted 1mo ago
  [deleted]
- tpm 1mo ago
  > Seven years later I have relatives who send me every day some long-winded AI-generated bullshit. So not releasing the weights at that time didn't make the world safer, but did help to transform OpenAI into a commercial entity. Why should we trust them now?
- iLoveOncall 1mo ago
  Oh ok I assumed Mythos was GA because I have access to it in Amazon despite not being in a security role at all. I guess it's just pay (enough) to play. Anyway, Fable is supposedly as good (if not better) in programming and it's very far from the paradigm shift that was promised.

xelxebar 1mo ago

Thank you for the epistemic humility here. It's a breath of fresh air amongst the polarized declarations we seem to have upregulated amongst ourselves for whatever reason these days.

pizza234 1mo ago
The problem of the polarized camps is that actually they're only on the surface opposite; in reality, they can perfectly cohexist: AI can enable major technological breakthroughs - but that doesn't exclude that can bring doom at the same time.

embedding-shape 1mo ago

> But this is the same way governments during COVID tested how much they can restrain freedom So AI CEOs don't have our best interests at heart, but the people who got voted for, who supposedly does have our best interests at heart, is secretely engaging in plots to "test how much they can restrain freedom"? I'm sorry, but this seems backwards. You're trusting the leaders of for-profit companies to do right regardless of their skewed incentives, and then the others are the ones to distrust. Sounds like you might want to move country is this is really how you feel.

bananaflag 1mo ago
> You're trusting the leaders of for-profit companies to do right regardless of their skewed incentives I'm not trusting them at all. I have seen for years how they don't do anything about alignment. OpenAI has disbanded countless alignment teams. But if their skewed incentives are for once bringing them into the right direction, I'm not gonna stand against them.

tpm 1mo ago

This looks like halfway through your comment you realised it is indeed a nefarious plot by the CEOs, reusing existing arguments to further their own goals. > The AI risk arguments ... are 20-30 years old Yeah so climate change is known to cause many excess deaths already now and we know pretty precisely what we have to do for decades, and we only do some bare minimum if anything at all. But now there is a huge PR offensive to do something about a 'risk' that until now caused zero deaths, is not certain to exist at all and is stoked not by concerned outsiders but by people certain to profit handsomely from heavy regulation of their industry. I'd say it's not opponents who need better arguments.

madaxe_again 1mo ago

They aren’t 20-30 years old - they are 160 years old. Samuel Butler wrote Erewhon in 1872 - which was based on an essay he wrote about RSI and superintelligence - in 1863.

bananaflag 1mo ago
I know, yes. I was talking about the current Bostrom/Yudkowsky iteration. Re Butler, you may enjoy this Chesterton pastiche if you haven't already: https://slatestarcodex.com/2017/04/01/g-k-chesterton-on-ai-risk/ https://slatestarcodex.com/2017/04/01/g-k-chesterton-on-ai-r...
- madaxe_again 1mo ago
  Thanks - that was an excellent read!

raverbashing 1mo ago

I think it's a much simpler reason, they're running out of money to have the rat race run at the current pace

dalemhurley 1mo ago
Yep

pizza234 1mo ago

> The AI risk arguments are philosophical arguments that are 20-30 years old This a false and uninformed take. As a starting point, you need to read and understand: 1. analyses of the AI incidents of the last months 2. latest advancements (e.g. the AI intern at OpenAI) 3. the problem of deceptive alignment which you clearly haven't done. After that, the AI problem will be actually very concrete; once the AI will: 1. have superhuman cognitive abilities (and it will) 2. will be inseparable from humanity and/or have the means to replicate will humanity be able to contain/control it? The answer is sadly very simple.

bananaflag 1mo ago
> which you clearly haven't done I have read all that, I follow e.g. Zvi Mowshowitz's blog.
PestoDiRucola 1mo ago
> will be inseparable from humanity and/or have the means to replicate How will this happen if you need insane amounts of compute to run these models? Where will the models replicate themselves by taking up PBs of space without anyone noticing?
- pizza234 1mo ago
  A few observations: - hardware capacity keeps growing - doom scenarios don't require fat clients; a sufficiently large number of AI servers deeply intertwined with essential services can be enough - autonomous systems are currently employed in weapons (e.g. drones) already - integration of AIs/LLMs is guaranteed to be the next step It's impossible to predict how exactly how things will develop (e.g. transfer large amounts of data; who knows, maybe it won't be necessary and/or maybe internet will be massively faster), but the building blocks are there.
dalemhurley 1mo ago
An intern who was there barely a minute, who has said nothing of substance. Surely we can get a better source with concrete evidence rather than the vibes of junior burger.