6 ms·

This wasn't even GPT-4. Terrific news. It would be super cool if the people who were currently fretting about paperclips could focus a little more on bias, se

by JieJie 3y ago

This wasn't even GPT-4. Terrific news. It would be super cool if the people who were currently fretting about paperclips could focus a little more on bias, security, data, privacy, openness, and all the stuff we actually are going to have to deal with over the next five years.

UnFleshedOne 3y ago
Why don't you focus on that instead?
p-e-w 3y ago
> It would be super cool if the people who were currently fretting about paperclips [...] That's textbook trivializing of an issue that has been extensively analyzed by experts and found to have civilization-ending consequences with near certainty, unless extensive countermeasures are taken well in advance. Your comment is the equivalent of climate change denial.
- EamonnMR 3y ago
  Have any countermeasures actually been implemented besides blog posts and podcast appearances? Climate activists regularly disrupt traffic, glue themselves to things, and protest pipelines. By contrast folks who believe AI will end the world have a propensity to start AI companies.
  - ben_w 3y ago
    At the moment, we're still trying to turn the question from an easy and obvious natural language form ("give me what I meant to ask for") into a form that is amenable to being solved in the world of weights and biases.[0] So several of the AI labs are published work on this topic, but it looks like e.g. "interpretable models" or philosophy of ethics etc. So today we're doing today with AI alignment research is the equivalent of asking Ada Lovelace how to prevent the Therac-25 deaths and the best response is "Why would you even do that, just don't write the wrong commands on the punched cards". [0] Some, myself included, are optimistic that early AI can help us do that. Yudkowsky appears to be dismissive of all options, but I don't see why any near-term AI would care to insert errors into later types of AI, so we've only got out own blind spots to look out for and we had that already… …but all the AI doomers I hang out with seem to consider me a terrible optimist, so YMMV.
ben_w 3y ago
While I reckon there's at least 5 (and probably 10+) years before it's even possible for a sufficiently independent AI to start literally paperclipping us (mostly because of the speed/power needs of good image processing AI), the people who are trying to solve that don't know if we can actually do so in even double that time frame. But the paperclip scenario was only ever an illustration that the AI doesn't need to hate us to destroy us, merely to want to use our atoms for something incompatible with our own desires. For disasters to happen, there are many options which may come sooner, varying from not paying attention to what the AI is doing when it makes mistakes — the default, as we can imagine even today someone putting a Boston Dynamics Spot to work in a virology lab and it walking out with a sample because it doesn't know any better — to the malicious, for example one idiot running a more competent version of ChaosGPT: https://youtu.be/g7YJIpkk7KM https://youtu.be/g7YJIpkk7KM Those other, lesser, disasters are sometimes called "fire alarms for AI".
- jiggawatts 3y ago
  The cynic in me says that the #1 AI-related risk by far is that corporations will immediately apply it to the task of listening in on everything we say and write. This all reminds me of "A Fire Upon the Deep" by Vernor Vinge, where the Slavers enslave humans by essentially constantly monitoring them for misbehaviour. If I remember correctly, a plot element is that they use yet more slaves to keep the bulk of the slaves in line. With ChatGPT, you no longer need to do "sci fi things" like turning squishy meat brains into cyborg slavedrivers. You can simply ask the AI to do it, and it will. It will do it 24/7, without pity, or remorse. It will do vigilantly and faithfully. It will be a checkbox in Office 365 that any psychotic manager can turn on. And I guarantee they will turn it on. All of them.
  - ben_w 3y ago
    I've not yet had time to read/listen to A Fire Upon the Deep yet, only its sequel A Deepness in the Sky, but that certainly sounds like one of the many bad endings we might get. That said: > With ChatGPT, you no longer need to do "sci fi things" like turning squishy meat brains into cyborg slavedrivers. You can simply ask the AI to do it, and it will. It will do it 24/7, without pity, or remorse. It will do vigilantly and faithfully. ChatGPT has all the vigilance and faithfulness of my late mother's mid-stage Alzheimer's, so if someone tried this today it would rapidly result in everyone getting million-dollar-no-work-required contracts the company isn't allowed to break. That LLMs can be socially engineered just by asking nicely is a pleasant surprise twist in AI alignment, but unfortunately I think it illustrates how primitive is our current state of the art AI safety and alignment.
gwd 3y ago
I really don't understand this antagonism between different branches of the "AI safety" community. In the 1960's, some people were concerned about preventing a nuclear apocalypse, and some people were concerned about lead emissions and CFCs and exploding cars. But did the nuclear apocalypse people tell the exploding cars people they should stop their work on exploding cars because nuclear apocalypse was more important long-term? Did the exploding car people tell the nuclear apocalypse people to stop raving about nuclear apocalypse because there were more pressing problems affecting people right now, like exploding cars and lead poisoning and a hole in the ozone layer? I mean, maybe they did; but if they did then I think it was kind of silly. Both kinds of problems are important, then and now.
- JieJie 3y ago
  I'm sorry it sounds antagonistic, but the negativity and "it's too late and we're all going to die no matter what we do" nihilism, is having the effect of sensationalizing the issue in ways that have been incredibly counterproductive. Here's what happened the last time technology was going to destroy us all: https://www.popularmechanics.com/science/a11217/what-stephen-hawking-really-said-about-destroying-the-universe-17192502/ https://www.popularmechanics.com/science/a11217/what-stephen... "The Higgs potential has the worrisome feature that it might become metastable at energies above 100 [billion] gigaelectronvolts (GeV). This could mean that the universe could undergo catastrophic vacuum decay, with a bubble of the true vacuum expanding at the speed of light. This could happen at any time and we wouldn't see it coming." —Stephen Hawking I'm not trying to make fun of Eliezer and co., but we have to have some proper perspective. We don't do that very well these days, because end-of-the-world scenarios are good for clicks.
  - srslack 3y ago
    Hawking's statement was made after the confirmation of the Higgs, and is still valid. A metastability vacuum event could be a problem (that's putting it lightly, hah) at high enough energies. It might have already happened, elsewhere. Eliezer and co. should be made fun of. They engage with journalists and lean into statements like this: https://www.lesswrong.com/posts/FKNtgZrGYwgsz3nHT/bankless-podcast-159-we-re-all-gonna-die-with-eliezer https://www.lesswrong.com/posts/FKNtgZrGYwgsz3nHT/bankless-p... They're absolutely hysterical. Convenient for them, and for the companies, the hysteria is to their benefit. More money to fund the MIRI for Yudowsky, and regulatory capture for the companies who believe they're ahead if you can convince some dementia-ridden Sentor to pass the bill regulating some vague notion of "AI safety." There's some people in these companies who understand that they have no moat, and that you can't make GPUs illegal, as demonstrated by the most recent story about the leaked google post, and that people can outperform their models in an hour with a LoRA on a workstation laptop for a given task, that the cat is out of the bag. But, with the hysteria of the media, where interns straight out of the shithole of liberal arts believe that a post on reddit or twitter where someone was slighted at the dinner table is breaking news, we're now in this situation where we have calls for regulation for something people simply don't understand, no thanks to the useless media.

by JieJie 3y ago

UnFleshedOne 3y ago

Why don't you focus on that instead?

p-e-w 3y ago

> It would be super cool if the people who were currently fretting about paperclips [...] That's textbook trivializing of an issue that has been extensively analyzed by experts and found to have civilization-ending consequences with near certainty, unless extensive countermeasures are taken well in advance. Your comment is the equivalent of climate change denial.

EamonnMR 3y ago
Have any countermeasures actually been implemented besides blog posts and podcast appearances? Climate activists regularly disrupt traffic, glue themselves to things, and protest pipelines. By contrast folks who believe AI will end the world have a propensity to start AI companies.
- ben_w 3y ago
  At the moment, we're still trying to turn the question from an easy and obvious natural language form ("give me what I meant to ask for") into a form that is amenable to being solved in the world of weights and biases.[0] So several of the AI labs are published work on this topic, but it looks like e.g. "interpretable models" or philosophy of ethics etc. So today we're doing today with AI alignment research is the equivalent of asking Ada Lovelace how to prevent the Therac-25 deaths and the best response is "Why would you even do that, just don't write the wrong commands on the punched cards". [0] Some, myself included, are optimistic that early AI can help us do that. Yudkowsky appears to be dismissive of all options, but I don't see why any near-term AI would care to insert errors into later types of AI, so we've only got out own blind spots to look out for and we had that already… …but all the AI doomers I hang out with seem to consider me a terrible optimist, so YMMV.

ben_w 3y ago

While I reckon there's at least 5 (and probably 10+) years before it's even possible for a sufficiently independent AI to start literally paperclipping us (mostly because of the speed/power needs of good image processing AI), the people who are trying to solve that don't know if we can actually do so in even double that time frame. But the paperclip scenario was only ever an illustration that the AI doesn't need to hate us to destroy us, merely to want to use our atoms for something incompatible with our own desires. For disasters to happen, there are many options which may come sooner, varying from not paying attention to what the AI is doing when it makes mistakes — the default, as we can imagine even today someone putting a Boston Dynamics Spot to work in a virology lab and it walking out with a sample because it doesn't know any better — to the malicious, for example one idiot running a more competent version of ChaosGPT: https://youtu.be/g7YJIpkk7KM https://youtu.be/g7YJIpkk7KM Those other, lesser, disasters are sometimes called "fire alarms for AI".

jiggawatts 3y ago
The cynic in me says that the #1 AI-related risk by far is that corporations will immediately apply it to the task of listening in on everything we say and write. This all reminds me of "A Fire Upon the Deep" by Vernor Vinge, where the Slavers enslave humans by essentially constantly monitoring them for misbehaviour. If I remember correctly, a plot element is that they use yet more slaves to keep the bulk of the slaves in line. With ChatGPT, you no longer need to do "sci fi things" like turning squishy meat brains into cyborg slavedrivers. You can simply ask the AI to do it, and it will. It will do it 24/7, without pity, or remorse. It will do vigilantly and faithfully. It will be a checkbox in Office 365 that any psychotic manager can turn on. And I guarantee they will turn it on. All of them.
- ben_w 3y ago
  I've not yet had time to read/listen to A Fire Upon the Deep yet, only its sequel A Deepness in the Sky, but that certainly sounds like one of the many bad endings we might get. That said: > With ChatGPT, you no longer need to do "sci fi things" like turning squishy meat brains into cyborg slavedrivers. You can simply ask the AI to do it, and it will. It will do it 24/7, without pity, or remorse. It will do vigilantly and faithfully. ChatGPT has all the vigilance and faithfulness of my late mother's mid-stage Alzheimer's, so if someone tried this today it would rapidly result in everyone getting million-dollar-no-work-required contracts the company isn't allowed to break. That LLMs can be socially engineered just by asking nicely is a pleasant surprise twist in AI alignment, but unfortunately I think it illustrates how primitive is our current state of the art AI safety and alignment.

gwd 3y ago

I really don't understand this antagonism between different branches of the "AI safety" community. In the 1960's, some people were concerned about preventing a nuclear apocalypse, and some people were concerned about lead emissions and CFCs and exploding cars. But did the nuclear apocalypse people tell the exploding cars people they should stop their work on exploding cars because nuclear apocalypse was more important long-term? Did the exploding car people tell the nuclear apocalypse people to stop raving about nuclear apocalypse because there were more pressing problems affecting people right now, like exploding cars and lead poisoning and a hole in the ozone layer? I mean, maybe they did; but if they did then I think it was kind of silly. Both kinds of problems are important, then and now.

JieJie 3y ago
I'm sorry it sounds antagonistic, but the negativity and "it's too late and we're all going to die no matter what we do" nihilism, is having the effect of sensationalizing the issue in ways that have been incredibly counterproductive. Here's what happened the last time technology was going to destroy us all: https://www.popularmechanics.com/science/a11217/what-stephen-hawking-really-said-about-destroying-the-universe-17192502/ https://www.popularmechanics.com/science/a11217/what-stephen... "The Higgs potential has the worrisome feature that it might become metastable at energies above 100 [billion] gigaelectronvolts (GeV). This could mean that the universe could undergo catastrophic vacuum decay, with a bubble of the true vacuum expanding at the speed of light. This could happen at any time and we wouldn't see it coming." —Stephen Hawking I'm not trying to make fun of Eliezer and co., but we have to have some proper perspective. We don't do that very well these days, because end-of-the-world scenarios are good for clicks.
- srslack 3y ago
  Hawking's statement was made after the confirmation of the Higgs, and is still valid. A metastability vacuum event could be a problem (that's putting it lightly, hah) at high enough energies. It might have already happened, elsewhere. Eliezer and co. should be made fun of. They engage with journalists and lean into statements like this: https://www.lesswrong.com/posts/FKNtgZrGYwgsz3nHT/bankless-podcast-159-we-re-all-gonna-die-with-eliezer https://www.lesswrong.com/posts/FKNtgZrGYwgsz3nHT/bankless-p... They're absolutely hysterical. Convenient for them, and for the companies, the hysteria is to their benefit. More money to fund the MIRI for Yudowsky, and regulatory capture for the companies who believe they're ahead if you can convince some dementia-ridden Sentor to pass the bill regulating some vague notion of "AI safety." There's some people in these companies who understand that they have no moat, and that you can't make GPUs illegal, as demonstrated by the most recent story about the leaked google post, and that people can outperform their models in an hour with a LoRA on a workstation laptop for a given task, that the cat is out of the bag. But, with the hysteria of the media, where interns straight out of the shithole of liberal arts believe that a post on reddit or twitter where someone was slighted at the dinner table is breaking news, we're now in this situation where we have calls for regulation for something people simply don't understand, no thanks to the useless media.