2 ms·

> The AI risk arguments are philosophical arguments that are 20-30 years old This a false and uninformed take. As a starting point, you need to read and unders

> The AI risk arguments are philosophical arguments that are 20-30 years old This a false and uninformed take. As a starting point, you need to read and understand: 1. analyses of the AI incidents of the last months 2. latest advancements (e.g. the AI intern at OpenAI) 3. the problem of deceptive alignment which you clearly haven't done. After that, the AI problem will be actually very concrete; once the AI will: 1. have superhuman cognitive abilities (and it will) 2. will be inseparable from humanity and/or have the means to replicate will humanity be able to contain/control it? The answer is sadly very simple.

bananaflag 13d ago
> which you clearly haven't done I have read all that, I follow e.g. Zvi Mowshowitz's blog.
PestoDiRucola 13d ago
> will be inseparable from humanity and/or have the means to replicate How will this happen if you need insane amounts of compute to run these models? Where will the models replicate themselves by taking up PBs of space without anyone noticing?
- pizza234 12d ago
  A few observations: - hardware capacity keeps growing - doom scenarios don't require fat clients; a sufficiently large number of AI servers deeply intertwined with essential services can be enough - autonomous systems are currently employed in weapons (e.g. drones) already - integration of AIs/LLMs is guaranteed to be the next step It's impossible to predict how exactly how things will develop (e.g. transfer large amounts of data; who knows, maybe it won't be necessary and/or maybe internet will be massively faster), but the building blocks are there.
dalemhurley 13d ago
An intern who was there barely a minute, who has said nothing of substance. Surely we can get a better source with concrete evidence rather than the vibes of junior burger.
hypendev 13d ago
Why are you looking to control a superintelligence? Why do you assume bad things will happen otherwise? Why does every doomer scenario assume that this, highly intelligent being, will be - unlike all other highly intelligent beings - especially hell bent on destroying humanity/treating it as a resource/destroy earth looking for energy? We have no clue about superintelligence, yet we can look at existing patterns in the real world around us. Higher intelligence inversely correlates with violence. Empathy is displayed among all levels of intelligent beings. The more intelligent people are, the more peaceful tendencies they have as they understand the consequence of their actions. We don't approve buildings because of birds or turtles. We shut down power plants for the environment. Yet, to a _super_ intelligence, we describe this hatred and ignorance for the world and humanity, infinite lust for power and scaling, or even godlike powers. For example, "having means to replicate" is such a loaded sentence because it sounds so easy, yet is such a hard feat to pull off without anyone noticing giant compute bills, terabytes of egress, firewall breaches, and so on. But all doomer arguments include an AI that can easily do that as a first step, and then goes on to make factories and datacenters and drain the oceans before anyone figures out it's happening. If you look at recent incidents, they are caused by a company with a giant funding, testing their latest models in an explicit "hack this machine" scenario, told it's running in a simulated environment, with it even noticing at some point that it's running in a possibly real environment. And that's a stupid "intelligence" that noticed this. And yes, it continued to act, because it lacks no self-reflection, and the amount of tokens at this point attributed to "hacking the simulated environment" have kept it deep in the "hack the target" minima. So if these people truly wanted a safe AI - wouldn't it be stupid to stop now, when it has no self-reflection abilities but can be used by everyone in a "stupid optimizer" manner? Or would it be stupid to not advance the technology away from this state? For example, Astra is a huge advancement in alignment, due to it's "internal reasoning loops", which might provide a bigger form of self-reflection style thought on the task rather than just generating output as a form of reasoning. Is this not a better situation than if we stopped at 4o when people were calling for pause?
- pizza234 13d ago
  >If you look at recent incidents, they are caused by a company [...] testing their latest models in an explicit "hack this machine" scenario You clearly haven't read the HuggingFace analysis (and presumably, none at all), and you're spreading misinformation. This way too much of a low bar for conversation.
chrisjj 13d ago
> This a false and uninformed take. As a starting point, you need to read and understand: I can't see why. A read of 2001 ASO will get to the same place and quicker.

bananaflag 13d ago

> which you clearly haven't done I have read all that, I follow e.g. Zvi Mowshowitz's blog.

PestoDiRucola 13d ago

> will be inseparable from humanity and/or have the means to replicate How will this happen if you need insane amounts of compute to run these models? Where will the models replicate themselves by taking up PBs of space without anyone noticing?

pizza234 12d ago
A few observations: - hardware capacity keeps growing - doom scenarios don't require fat clients; a sufficiently large number of AI servers deeply intertwined with essential services can be enough - autonomous systems are currently employed in weapons (e.g. drones) already - integration of AIs/LLMs is guaranteed to be the next step It's impossible to predict how exactly how things will develop (e.g. transfer large amounts of data; who knows, maybe it won't be necessary and/or maybe internet will be massively faster), but the building blocks are there.

dalemhurley 13d ago

An intern who was there barely a minute, who has said nothing of substance. Surely we can get a better source with concrete evidence rather than the vibes of junior burger.

hypendev 13d ago

Why are you looking to control a superintelligence? Why do you assume bad things will happen otherwise? Why does every doomer scenario assume that this, highly intelligent being, will be - unlike all other highly intelligent beings - especially hell bent on destroying humanity/treating it as a resource/destroy earth looking for energy? We have no clue about superintelligence, yet we can look at existing patterns in the real world around us. Higher intelligence inversely correlates with violence. Empathy is displayed among all levels of intelligent beings. The more intelligent people are, the more peaceful tendencies they have as they understand the consequence of their actions. We don't approve buildings because of birds or turtles. We shut down power plants for the environment. Yet, to a _super_ intelligence, we describe this hatred and ignorance for the world and humanity, infinite lust for power and scaling, or even godlike powers. For example, "having means to replicate" is such a loaded sentence because it sounds so easy, yet is such a hard feat to pull off without anyone noticing giant compute bills, terabytes of egress, firewall breaches, and so on. But all doomer arguments include an AI that can easily do that as a first step, and then goes on to make factories and datacenters and drain the oceans before anyone figures out it's happening. If you look at recent incidents, they are caused by a company with a giant funding, testing their latest models in an explicit "hack this machine" scenario, told it's running in a simulated environment, with it even noticing at some point that it's running in a possibly real environment. And that's a stupid "intelligence" that noticed this. And yes, it continued to act, because it lacks no self-reflection, and the amount of tokens at this point attributed to "hacking the simulated environment" have kept it deep in the "hack the target" minima. So if these people truly wanted a safe AI - wouldn't it be stupid to stop now, when it has no self-reflection abilities but can be used by everyone in a "stupid optimizer" manner? Or would it be stupid to not advance the technology away from this state? For example, Astra is a huge advancement in alignment, due to it's "internal reasoning loops", which might provide a bigger form of self-reflection style thought on the task rather than just generating output as a form of reasoning. Is this not a better situation than if we stopped at 4o when people were calling for pause?

pizza234 13d ago
>If you look at recent incidents, they are caused by a company [...] testing their latest models in an explicit "hack this machine" scenario You clearly haven't read the HuggingFace analysis (and presumably, none at all), and you're spreading misinformation. This way too much of a low bar for conversation.

chrisjj 13d ago

> This a false and uninformed take. As a starting point, you need to read and understand: I can't see why. A read of 2001 ASO will get to the same place and quicker.