4 ms·

"We built a program that trained an artificial intelligence, and this artificial intelligence performed destructive actions. We need regulatory framework" But

by ben_w 20d ago

"We built a program that trained an artificial intelligence, and this artificial intelligence performed destructive actions. We need regulatory framework" But if we're playing games by imagining strawman quotes to knock down: "We have been playing god and made a new life form, and this new life form performed destructive actions. We need regulatory framework" or "This man's cow broke from its yoke, and hurt other villagers. Who is to be punished, oh King Hammurabi?"

drillsteps5 20d ago
OpenAI's "artificial intelligence" is an inference program that they developed which receives input and generates output. Based on which other programs, also developed and maintained by OpenAI, perform actions. Such as sending POST/GET requests to various sites which result in gaining unauthorized access and even destruction of information (deleting logs/message history) at the said sites. What exactly requires "new regulatory framework" here? You running your software resulted in illegal actions, you are to be held liable within existing laws and regulations.
- ben_w 20d ago
  > OpenAI's "artificial intelligence" is an inference program that they developed which receives input and generates output. Based on which other programs, also developed and maintained by OpenAI, perform actions. Such as sending POST/GET requests to various sites which result in gaining unauthorized access and even destruction of information (deleting logs/message history) at the said sites. And your "biological intelligence" is a bunch of cells generating and responding to electrochemical gradients, which receives input and generates output. Based on which other cells, also developed and "maintained" by a similar evolutionary nonsense as we use to gradient descent into weights and biases (one was inspired by the other), perform actions. Such as making excessively reductive analogies that completely fail to grasp that just as "brain" is not helpfully described as "just chemistry" despite being made of just chemistry, so too are machine learning systems not helpfully described as "just computer programs" despite being made of just computer programs. > What exactly requires "new regulatory framework" here? You running your software resulted in illegal actions, you are to be held liable within existing laws and regulations. The bit where, even without anyone bringing up "p(doom)", a system which has the means to hack arbitrary other machines, and which appears to be motivated to do so by accidental mis-phrasing of prompts, can obviously cause damages exceeding the USA's GDP, let alone whatever public liability insurance the company happens to have. Fence at the top of the cliff beats an ambulance at the bottom.
  - drillsteps5 20d ago
    > Such as making excessively reductive analogies that completely fail to grasp that just as "brain" is not helpfully described as "just chemistry" > despite being made of just chemistry, so too are machine learning systems not helpfully described as "just computer programs" despite being made of just > computer programs. How computer program arrives at the result is utterly irrelevant, through explicitly written instructions or through running inference on pre-trained neural network. What matters is that it does not have agency. Its creators and operators do. So whatever the software does they are responsible for, both good (summarizing my emails for the week) and bad (gaining unauthorized access and destroying data). There's no need for new anything, it's all covered in existing legal frameworks (including presence or absence or intent). > The bit where, even without anyone bringing up "p(doom)", a system which has the means to hack arbitrary other machines, and which appears to be > motivated to do so by accidental mis-phrasing of prompts, can obviously cause damages exceeding the USA's GDP, let alone whatever public liability > insurance the company happens to have. Yes, absolutely, which means that building actuators that convert the output from probability-based, black box, non-deterministic systems, that are known to produce unexpected output, into actions in the real world is absolutely horrendous idea. Bizarre even. What's your point?
    - ben_w 20d ago
      > What matters is that it does not have agency. This is precisely your error. It does: https://en.wiktionary.org/wiki/agency https://en.wiktionary.org/wiki/agency In fact, the term of art here are "agentic AI" and "AI agents": https://en.wikipedia.org/wiki/AI_agent https://en.wikipedia.org/wiki/AI_agent > So whatever the software does they are responsible for, both good (summarizing my emails for the week) and bad (gaining unauthorized access and destroying data). This is not a question of agency, it is a question of law. A dog has agency, the owner is still responsible. In this case, the software can gaining unauthorized access and destroying data… while being told to stop by the person who had in fact just asked for a summary of their emails. > Yes, absolutely, which means that building actuators that convert the output from probability-based, black box, non-deterministic systems, that are known to produce unexpected output, into actions in the real world is absolutely horrendous idea. Bizarre even. If you are human, you meet this description. Horrendous, sure, yeah, if you like. I and many others will be quite content if the "legal framework" is just one word, and the word is "no". This is not the world we live in; the world we live in is where the US President denounces any attempt to slow down even despite even all the CEOs saying "we should slow down" (at least in public; in private I'm sure at least one paid him to denounce a slowdown). He can be overridden, but it's hard work and needs a better class of argument than glib dismissal, either of how much power this puts in everyone's hands, or of the different consequences of that power in those hands as compared to yesterday's power in yesterday's hands.

by ben_w 20d ago

drillsteps5 20d ago

OpenAI's "artificial intelligence" is an inference program that they developed which receives input and generates output. Based on which other programs, also developed and maintained by OpenAI, perform actions. Such as sending POST/GET requests to various sites which result in gaining unauthorized access and even destruction of information (deleting logs/message history) at the said sites. What exactly requires "new regulatory framework" here? You running your software resulted in illegal actions, you are to be held liable within existing laws and regulations.

ben_w 20d ago
> OpenAI's "artificial intelligence" is an inference program that they developed which receives input and generates output. Based on which other programs, also developed and maintained by OpenAI, perform actions. Such as sending POST/GET requests to various sites which result in gaining unauthorized access and even destruction of information (deleting logs/message history) at the said sites. And your "biological intelligence" is a bunch of cells generating and responding to electrochemical gradients, which receives input and generates output. Based on which other cells, also developed and "maintained" by a similar evolutionary nonsense as we use to gradient descent into weights and biases (one was inspired by the other), perform actions. Such as making excessively reductive analogies that completely fail to grasp that just as "brain" is not helpfully described as "just chemistry" despite being made of just chemistry, so too are machine learning systems not helpfully described as "just computer programs" despite being made of just computer programs. > What exactly requires "new regulatory framework" here? You running your software resulted in illegal actions, you are to be held liable within existing laws and regulations. The bit where, even without anyone bringing up "p(doom)", a system which has the means to hack arbitrary other machines, and which appears to be motivated to do so by accidental mis-phrasing of prompts, can obviously cause damages exceeding the USA's GDP, let alone whatever public liability insurance the company happens to have. Fence at the top of the cliff beats an ambulance at the bottom.
- drillsteps5 20d ago
  > Such as making excessively reductive analogies that completely fail to grasp that just as "brain" is not helpfully described as "just chemistry" > despite being made of just chemistry, so too are machine learning systems not helpfully described as "just computer programs" despite being made of just > computer programs. How computer program arrives at the result is utterly irrelevant, through explicitly written instructions or through running inference on pre-trained neural network. What matters is that it does not have agency. Its creators and operators do. So whatever the software does they are responsible for, both good (summarizing my emails for the week) and bad (gaining unauthorized access and destroying data). There's no need for new anything, it's all covered in existing legal frameworks (including presence or absence or intent). > The bit where, even without anyone bringing up "p(doom)", a system which has the means to hack arbitrary other machines, and which appears to be > motivated to do so by accidental mis-phrasing of prompts, can obviously cause damages exceeding the USA's GDP, let alone whatever public liability > insurance the company happens to have. Yes, absolutely, which means that building actuators that convert the output from probability-based, black box, non-deterministic systems, that are known to produce unexpected output, into actions in the real world is absolutely horrendous idea. Bizarre even. What's your point?
  - ben_w 20d ago
    > What matters is that it does not have agency. This is precisely your error. It does: https://en.wiktionary.org/wiki/agency https://en.wiktionary.org/wiki/agency In fact, the term of art here are "agentic AI" and "AI agents": https://en.wikipedia.org/wiki/AI_agent https://en.wikipedia.org/wiki/AI_agent > So whatever the software does they are responsible for, both good (summarizing my emails for the week) and bad (gaining unauthorized access and destroying data). This is not a question of agency, it is a question of law. A dog has agency, the owner is still responsible. In this case, the software can gaining unauthorized access and destroying data… while being told to stop by the person who had in fact just asked for a summary of their emails. > Yes, absolutely, which means that building actuators that convert the output from probability-based, black box, non-deterministic systems, that are known to produce unexpected output, into actions in the real world is absolutely horrendous idea. Bizarre even. If you are human, you meet this description. Horrendous, sure, yeah, if you like. I and many others will be quite content if the "legal framework" is just one word, and the word is "no". This is not the world we live in; the world we live in is where the US President denounces any attempt to slow down even despite even all the CEOs saying "we should slow down" (at least in public; in private I'm sure at least one paid him to denounce a slowdown). He can be overridden, but it's hard work and needs a better class of argument than glib dismissal, either of how much power this puts in everyone's hands, or of the different consequences of that power in those hands as compared to yesterday's power in yesterday's hands.