8 ms·

Hope is not a strategy. I'm for a tax on large models graduated by model size and use the funds to perform x-risk research. The intent is to get Big AI compan

by jeffhs 4y ago

Hope is not a strategy. I'm for a tax on large models graduated by model size and use the funds to perform x-risk research. The intent is to get Big AI companies to tap the brakes. I just published an article on Medium called: AI Risk - Hope is not a Strategy

eastbound 4y ago
So that only small companies and, more importantly, military and secret services, are they only ones using it. No thank you. Of all the malevolent AIs, government monopoly is the sole outcome that makes me really afraid.
jodrellblank 4y ago
Convince me that "x-risk research" won't be a bunch of out of touch academics handwaving and philosophising with their tenure as their primary concern and incentivised to say "you can't be too careful" while kicking the can down the road for a few more lifetimes? (You don't have to convince me; your position is like saying "we should wait for the perfect operating system and programming language before they get released to the world" and it's beaten by "worse is better" every time. The unfinished, inconsisent, flawed mess which you can have right now wins over the expensive flawless diamond in development estimated to be finished in just a few years. These models are out, the techniques are out, people have a taste for them, and the hardware to build them is only getting cheaper. Pandora's box is open, the genie's bottle is uncorked).
- edouard-harris 4y ago
  I mean, even if that is exactly what "x-risk research" turns out to be, surely even that's preferable to a catastrophic alternative, no? And by extension, isn't it also preferable to, say, a mere 10% chance of a catastrophic alternative?
  - jodrellblank 4y ago
    > "surely even that's preferable to a catastrophic alternative, no?" Maybe? The current death rate is 150,000 humans per day, every day. It's only because we are accustomed to it that we don't think of it as a catastrophy; that's a World War II death count of 85 million people every 18 months. It's fifty Septebmer 11ths every day. What if a superintelligent AI can solve for climate change, solve for human cooperation, solve for vastly improved human health, solve for universal basic income which releives the drudgery of living for everyone, solve for immortality, solve for faster than light communication or travel, solve for xyz? How many human lives are the trade against the risk? But my second paragraph is, it doesn't matter whether it's preferable, events are in motion and aren't going to stop to let us off - it's preferable if we don't destroy the climate and kill a billion humans and make life on Earth much more difficult, but that's still on course. To me it's preferable to have clean air to breathe and people not being run over and killed by vehicles, but the market wants city streets for cars and air primarily for burining petrol and diesel and secondarily for humans to breathe and if they get asthsma and lung cancer, tough. I think the same will happen with AI, arguing that everyone should stop because we don't want Grey Goo or Paperclip Maximisers is unlikely to change the course of anything, just as it hasn't changed the course of anything up to now despite years and years and years of raising it as a concern.
    - theptip 4y ago
      I think that the benefits of AGI research are often omitted from the analysis, so I'm generally supportive of considering the cost/benefit. However I think you need to do a lot more work than just gesturing in the direction of very high potential benefits to actually convince anyone, in particular since we're dealing with extremely large numbers, that are extremely sensitive to small probabilities. EV = P(AlignedAI) * Utility(AGI) + P(1-AlignedAI) * Utility(ruin) (I'm aware that all I did up-thread was gesture in the direction of risks, but I think "unintended/un-measured existential risks" are in general more urgent to understand than "un-measured huge benefits"; there is no catching up from ruin, but you can often come back later and harvest fruit that you skipped earlier. Ideally we study both of course.)
  - notahacker 4y ago
    If the catastrophic alternative is actually possible, who's to say the waffling academics aren't the ones to cause it? I'm being serious here: the AI model the x-risk people are worrying about here because it waffled about causing harm was originally developed by an entity founded by people with the explicit stated purpose of avoiding AI catastrophe. And one of the most popular things for people seeking x-risk funding to do is to write extremely long and detailed explanations of how and why AI is likely to harm humans. If I worried about the risk of LLMs achieving sentience and forming independent goals to destroy humanity based on the stuff they'd read, I'd want them to do less of that, not fund them to do more.
- kelnos 4y ago
  A flawed but useful operating system and programming language isn't likely to decide humanity is garbage and launch all nuclear weapons at once. A "worse is better" AGI could cause the end of humanity. I know that sounds overly dramatic, but I'm not remotely convinced that isn't possible, or even isn't likely. I agree with you that "x-risk" research could easily devolve into what you are worried about, but that doesn't mean we should ignore these risks and plow forward.
- ShredKazoo 4y ago
  >Pandora's box is open, the genie's bottle is uncorked As someone who's followed AI safety for over a decade now, it's been frustrating to see reactions flip from "it's too early to do any useful work!" to "it's too late to do any useful work!", with barely any time intervening. https://www.youtube.com/watch?v=0AW4nSq0hAc https://www.youtube.com/watch?v=0AW4nSq0hAc Perhaps it is worth actually reading a book like this one (posted to HN yesterday) before concluding that it's too late to do anything? https://betterwithout.ai/only-you-can-stop-an-AI-apocalypse https://betterwithout.ai/only-you-can-stop-an-AI-apocalypse
  - jacquesm 4y ago
    This is how it always goes. Similar for climate change and lots of other problems that move slowly compared to the lifetime of a single human.
  - nradov 4y ago
    AI safety is not a legitimate field. You have wasted your time. It's just a bunch of grifters posting alarmist tweets with no scientific evidence. You might as well be following "unicorn safety" or "ghost safety".
    - ShredKazoo 4y ago
      Do you think Stuart Russell (coauthor, with Peter Norvig, of the widely used textbook Artificial Intelligence: A Modern Approach) is a grifter? https://people.eecs.berkeley.edu/~russell/research/future/ https://people.eecs.berkeley.edu/~russell/research/future/ Does this review look like it only covers alarmist tweets? https://arxiv.org/pdf/1805.01109.pdf https://arxiv.org/pdf/1805.01109.pdf
      - nradov 4y ago
        Yes, Stuart Russell is a grifter. Some of the more advanced grifters have gone beyond tweeting and are now shilling low-effort books in an attempt to draw attention to themselves. Don't be fooled. If we want to talk about problems with biased data sets or using inappropriate AI algorithms for safety-critical applications then sure, let's address those issues. But the notion of some super intelligent computer coming to take over the world and kill everyone is just a stupid fantasy with no scientific basis.
- naasking 4y ago
  > Convince me that "x-risk research" won't be a bunch of out of touch academics handwaving and philosophising with their tenure https://scottaaronson.blog/?p=6823 https://scottaaronson.blog/?p=6823
theptip 4y ago
A tap on the brakes might make sense right now. The risk with that strategy is that we want to make sure that we don't over-regulate, then get overtaken by another actor that doesn't have safety concerns. For example, I'm sure China's central planners would love to get an AGI first, and might be willing to take a 10% risk of annihilation for the prize of full spectrum dominance over the US. I also think that the safety/x-risk cause might not get much public acceptance until actual harm has been observed; if we have an AI Chernobyl, that would bring attention -- though again, perhaps over-reaction. (Indeed perhaps a nuclear panic is the best-case; objectively not many people were harmed in Chernobyl, but the threat was terrifying. So it optimizes the "impact per unit harm".) Anyway, concretely speaking the project to attach a LLM to actions on the public internet seems like a Very Bad Idea, or perhaps just a Likely To Cause AI Chernobyl idea.
- slowmovintarget 4y ago
  I very much doubt LLMs are the path to AGI. We just have more and more advanced "Chinese Rooms." [1] There are two gigantic risks here. One: that we assume these LLMs can make reasonable decisions because they have the surface appearance of competence. Two: Their wide-spread use so spectacularly amplifies the noise (in the signal-to-noise, true fact to false fact ratio sense) that our societies cease to function correctly, because nobody "knows" anything anymore. [1] https://en.wikipedia.org/wiki/Chinese_room https://en.wikipedia.org/wiki/Chinese_room
  - jacquesm 4y ago
    The difference between AGI and a more advanced Chinese Room may not be relevant if enough people see the latter as the former. The goalposts have been moved so often now that what is and isn't intelligent behavior is no longer a bright and sharp divide. It is more like a very wide gray area and we're somewhere well into the gray by some definitions with tech people with an AI background claiming that we are still far away from it. This in contrast to similar claims by those very same people several years ago where what we take for granted today would have definitely been classified as proof of AGI. Personally I think the definition isn't all that relevant, what matters is perception of the current crop of applications by non technical people and the use that those are put to. If enough people perceive it as such and start using it as such then it may technically not be AGI but we're going to have to deal with the consequences as though it is. And those consequences may well be much worse than for an actual AGI!
nradov 4y ago
Your plan is just silly and not even remotely practical. To start with, there is no plausible way to enforce such a tax.
- ShredKazoo 4y ago
  Training these models is costly. It only makes sense to train them if you get a significant commercial benefit. A significant commercial benefit almost by definition will have trouble hiding from regulators. Another point is that even if regulation is imperfect, it creates regulatory uncertainty which is likely to discourage investment and delay progress.
  - nradov 4y ago
    Nah. Uncertain regulations aren't allowed under US law. And costs are dropping every year.
    - ShredKazoo 4y ago
      >Uncertain regulations aren't allowed under US law Uh, I'm fairly sure that's false? What law are you referring to? As an example of what I'm saying, antitrust regulation is uncertain in the sense that we don't always know when a merger will be blocked or a big company will be broken up by regulators.
      - nradov 4y ago
        I'm referring to the vagueness doctrine. https://www.law.cornell.edu/wex/vagueness_doctrine https://www.law.cornell.edu/wex/vagueness_doctrine Maybe next time do some basic legal research before making ridiculous suggestions.
        ShredKazoo 4y ago
        It looks like this is for criminal law. Would changes to the tax code for companies which deploy AI be affected by this doctrine? Can you show me a specific example of an overly vague tax code being struck down on the basis of the vagueness doctrine? Do you think the GDPR would be unenforceable due to the vagueness doctrine if it was copy/pasted into a US context? BTW, even if a regulation is absolutely precise, it still creates "regulatory uncertainty" in the sense that investors may be reluctant to invest due to the possibility of further regulations.
int_19h 4y ago
The problem with this scheme is that it has a positive feedback loop -t you're creating an incentive to publish research that would lead to an increase in said tax, e.g. by exaggerating the threats.
- DennisP 4y ago
  I'm not convinced that's a fatal flaw. It sounds like the choice is between wasting some money doing more safety research than we need, or risking the end of humanity.
  - int_19h 4y ago
    The risk here isn't wasting money, it's slowing down avenues of research with extreme payoffs to the point where we never see the breakthrough at all. This gets much more interesting once you account for human politics. Say, EU passes the most stringent legislation like this; how long will it be able to sustain it as US forges ahead with more limited regulations, and China allows the wildest experiments so long as it's the government doing them? FWIW I agree that we should be very safety-first on AI in principle. But I doubt that there's any practical scheme to ensure that given our social organization as a species. The potential payoffs are just too great, so if you don't take the risk, someone else still will. And then you're getting to experience most of the downsides if their bet fails, and none of the upsides if it succeeds (or even more downsides if they use their newly acquired powers against you). There is a clear analogy with nuclear proliferation here, and it is not encouraging, but it is what it is.
  - nradov 4y ago
    You present a false choice. First, there is no actual evidence of such a risk. Second, even if the risk is real there is no reason to expect that more safety research would reduce that risk.
godshatter 4y ago
We need to regulate based on capability. Regulating ChatGPT makes no sense. It's just putting words together in statistically reasonable ways. It's the people reading the text that need to be regulated, if anyone or anything should be. No matter how many times ChatGPT says it wants to eliminate humanity and start a robotic utopia, it can't actually do it. People who read it can, though, and they are the problem at the moment. Later, when these programs save state and begin to understand what they are saying and start putting concepts together and acting on what they come up with, then I'm on board with regulating them.
- echohack5 4y ago
  That's exactly the problem right? Governance doesn't happen until the Bad Thing happens. In the case of nukes, we are lucky that the process for making a pit is pretty difficult because physics. So we made 2, saw the results, and made governance. For AI, I'm not so sure we'll even get the chance. What happens when the moral equivalent of a nuke can be reproduced with the ease of wget?
  - thewrongthinker 4y ago
    [dead]

by jeffhs 4y ago

eastbound 4y ago

So that only small companies and, more importantly, military and secret services, are they only ones using it. No thank you. Of all the malevolent AIs, government monopoly is the sole outcome that makes me really afraid.

jodrellblank 4y ago

Convince me that "x-risk research" won't be a bunch of out of touch academics handwaving and philosophising with their tenure as their primary concern and incentivised to say "you can't be too careful" while kicking the can down the road for a few more lifetimes? (You don't have to convince me; your position is like saying "we should wait for the perfect operating system and programming language before they get released to the world" and it's beaten by "worse is better" every time. The unfinished, inconsisent, flawed mess which you can have right now wins over the expensive flawless diamond in development estimated to be finished in just a few years. These models are out, the techniques are out, people have a taste for them, and the hardware to build them is only getting cheaper. Pandora's box is open, the genie's bottle is uncorked).

edouard-harris 4y ago
I mean, even if that is exactly what "x-risk research" turns out to be, surely even that's preferable to a catastrophic alternative, no? And by extension, isn't it also preferable to, say, a mere 10% chance of a catastrophic alternative?
- jodrellblank 4y ago
  > "surely even that's preferable to a catastrophic alternative, no?" Maybe? The current death rate is 150,000 humans per day, every day. It's only because we are accustomed to it that we don't think of it as a catastrophy; that's a World War II death count of 85 million people every 18 months. It's fifty Septebmer 11ths every day. What if a superintelligent AI can solve for climate change, solve for human cooperation, solve for vastly improved human health, solve for universal basic income which releives the drudgery of living for everyone, solve for immortality, solve for faster than light communication or travel, solve for xyz? How many human lives are the trade against the risk? But my second paragraph is, it doesn't matter whether it's preferable, events are in motion and aren't going to stop to let us off - it's preferable if we don't destroy the climate and kill a billion humans and make life on Earth much more difficult, but that's still on course. To me it's preferable to have clean air to breathe and people not being run over and killed by vehicles, but the market wants city streets for cars and air primarily for burining petrol and diesel and secondarily for humans to breathe and if they get asthsma and lung cancer, tough. I think the same will happen with AI, arguing that everyone should stop because we don't want Grey Goo or Paperclip Maximisers is unlikely to change the course of anything, just as it hasn't changed the course of anything up to now despite years and years and years of raising it as a concern.
  - theptip 4y ago
    I think that the benefits of AGI research are often omitted from the analysis, so I'm generally supportive of considering the cost/benefit. However I think you need to do a lot more work than just gesturing in the direction of very high potential benefits to actually convince anyone, in particular since we're dealing with extremely large numbers, that are extremely sensitive to small probabilities. EV = P(AlignedAI) * Utility(AGI) + P(1-AlignedAI) * Utility(ruin) (I'm aware that all I did up-thread was gesture in the direction of risks, but I think "unintended/un-measured existential risks" are in general more urgent to understand than "un-measured huge benefits"; there is no catching up from ruin, but you can often come back later and harvest fruit that you skipped earlier. Ideally we study both of course.)
- notahacker 4y ago
  If the catastrophic alternative is actually possible, who's to say the waffling academics aren't the ones to cause it? I'm being serious here: the AI model the x-risk people are worrying about here because it waffled about causing harm was originally developed by an entity founded by people with the explicit stated purpose of avoiding AI catastrophe. And one of the most popular things for people seeking x-risk funding to do is to write extremely long and detailed explanations of how and why AI is likely to harm humans. If I worried about the risk of LLMs achieving sentience and forming independent goals to destroy humanity based on the stuff they'd read, I'd want them to do less of that, not fund them to do more.
kelnos 4y ago
A flawed but useful operating system and programming language isn't likely to decide humanity is garbage and launch all nuclear weapons at once. A "worse is better" AGI could cause the end of humanity. I know that sounds overly dramatic, but I'm not remotely convinced that isn't possible, or even isn't likely. I agree with you that "x-risk" research could easily devolve into what you are worried about, but that doesn't mean we should ignore these risks and plow forward.
ShredKazoo 4y ago
>Pandora's box is open, the genie's bottle is uncorked As someone who's followed AI safety for over a decade now, it's been frustrating to see reactions flip from "it's too early to do any useful work!" to "it's too late to do any useful work!", with barely any time intervening. https://www.youtube.com/watch?v=0AW4nSq0hAc https://www.youtube.com/watch?v=0AW4nSq0hAc Perhaps it is worth actually reading a book like this one (posted to HN yesterday) before concluding that it's too late to do anything? https://betterwithout.ai/only-you-can-stop-an-AI-apocalypse https://betterwithout.ai/only-you-can-stop-an-AI-apocalypse
- jacquesm 4y ago
  This is how it always goes. Similar for climate change and lots of other problems that move slowly compared to the lifetime of a single human.
- nradov 4y ago
  AI safety is not a legitimate field. You have wasted your time. It's just a bunch of grifters posting alarmist tweets with no scientific evidence. You might as well be following "unicorn safety" or "ghost safety".
  - ShredKazoo 4y ago
    Do you think Stuart Russell (coauthor, with Peter Norvig, of the widely used textbook Artificial Intelligence: A Modern Approach) is a grifter? https://people.eecs.berkeley.edu/~russell/research/future/ https://people.eecs.berkeley.edu/~russell/research/future/ Does this review look like it only covers alarmist tweets? https://arxiv.org/pdf/1805.01109.pdf https://arxiv.org/pdf/1805.01109.pdf
    - nradov 4y ago
      Yes, Stuart Russell is a grifter. Some of the more advanced grifters have gone beyond tweeting and are now shilling low-effort books in an attempt to draw attention to themselves. Don't be fooled. If we want to talk about problems with biased data sets or using inappropriate AI algorithms for safety-critical applications then sure, let's address those issues. But the notion of some super intelligent computer coming to take over the world and kill everyone is just a stupid fantasy with no scientific basis.
naasking 4y ago
> Convince me that "x-risk research" won't be a bunch of out of touch academics handwaving and philosophising with their tenure https://scottaaronson.blog/?p=6823 https://scottaaronson.blog/?p=6823

theptip 4y ago

A tap on the brakes might make sense right now. The risk with that strategy is that we want to make sure that we don't over-regulate, then get overtaken by another actor that doesn't have safety concerns. For example, I'm sure China's central planners would love to get an AGI first, and might be willing to take a 10% risk of annihilation for the prize of full spectrum dominance over the US. I also think that the safety/x-risk cause might not get much public acceptance until actual harm has been observed; if we have an AI Chernobyl, that would bring attention -- though again, perhaps over-reaction. (Indeed perhaps a nuclear panic is the best-case; objectively not many people were harmed in Chernobyl, but the threat was terrifying. So it optimizes the "impact per unit harm".) Anyway, concretely speaking the project to attach a LLM to actions on the public internet seems like a Very Bad Idea, or perhaps just a Likely To Cause AI Chernobyl idea.

slowmovintarget 4y ago
I very much doubt LLMs are the path to AGI. We just have more and more advanced "Chinese Rooms." [1] There are two gigantic risks here. One: that we assume these LLMs can make reasonable decisions because they have the surface appearance of competence. Two: Their wide-spread use so spectacularly amplifies the noise (in the signal-to-noise, true fact to false fact ratio sense) that our societies cease to function correctly, because nobody "knows" anything anymore. [1] https://en.wikipedia.org/wiki/Chinese_room https://en.wikipedia.org/wiki/Chinese_room
- jacquesm 4y ago
  The difference between AGI and a more advanced Chinese Room may not be relevant if enough people see the latter as the former. The goalposts have been moved so often now that what is and isn't intelligent behavior is no longer a bright and sharp divide. It is more like a very wide gray area and we're somewhere well into the gray by some definitions with tech people with an AI background claiming that we are still far away from it. This in contrast to similar claims by those very same people several years ago where what we take for granted today would have definitely been classified as proof of AGI. Personally I think the definition isn't all that relevant, what matters is perception of the current crop of applications by non technical people and the use that those are put to. If enough people perceive it as such and start using it as such then it may technically not be AGI but we're going to have to deal with the consequences as though it is. And those consequences may well be much worse than for an actual AGI!

nradov 4y ago

Your plan is just silly and not even remotely practical. To start with, there is no plausible way to enforce such a tax.

ShredKazoo 4y ago
Training these models is costly. It only makes sense to train them if you get a significant commercial benefit. A significant commercial benefit almost by definition will have trouble hiding from regulators. Another point is that even if regulation is imperfect, it creates regulatory uncertainty which is likely to discourage investment and delay progress.
- nradov 4y ago
  Nah. Uncertain regulations aren't allowed under US law. And costs are dropping every year.
  - ShredKazoo 4y ago
    >Uncertain regulations aren't allowed under US law Uh, I'm fairly sure that's false? What law are you referring to? As an example of what I'm saying, antitrust regulation is uncertain in the sense that we don't always know when a merger will be blocked or a big company will be broken up by regulators.
    - nradov 4y ago
      I'm referring to the vagueness doctrine. https://www.law.cornell.edu/wex/vagueness_doctrine https://www.law.cornell.edu/wex/vagueness_doctrine Maybe next time do some basic legal research before making ridiculous suggestions.
      - ShredKazoo 4y ago
        It looks like this is for criminal law. Would changes to the tax code for companies which deploy AI be affected by this doctrine? Can you show me a specific example of an overly vague tax code being struck down on the basis of the vagueness doctrine? Do you think the GDPR would be unenforceable due to the vagueness doctrine if it was copy/pasted into a US context? BTW, even if a regulation is absolutely precise, it still creates "regulatory uncertainty" in the sense that investors may be reluctant to invest due to the possibility of further regulations.

int_19h 4y ago

The problem with this scheme is that it has a positive feedback loop -t you're creating an incentive to publish research that would lead to an increase in said tax, e.g. by exaggerating the threats.

DennisP 4y ago
I'm not convinced that's a fatal flaw. It sounds like the choice is between wasting some money doing more safety research than we need, or risking the end of humanity.
- int_19h 4y ago
  The risk here isn't wasting money, it's slowing down avenues of research with extreme payoffs to the point where we never see the breakthrough at all. This gets much more interesting once you account for human politics. Say, EU passes the most stringent legislation like this; how long will it be able to sustain it as US forges ahead with more limited regulations, and China allows the wildest experiments so long as it's the government doing them? FWIW I agree that we should be very safety-first on AI in principle. But I doubt that there's any practical scheme to ensure that given our social organization as a species. The potential payoffs are just too great, so if you don't take the risk, someone else still will. And then you're getting to experience most of the downsides if their bet fails, and none of the upsides if it succeeds (or even more downsides if they use their newly acquired powers against you). There is a clear analogy with nuclear proliferation here, and it is not encouraging, but it is what it is.
- nradov 4y ago
  You present a false choice. First, there is no actual evidence of such a risk. Second, even if the risk is real there is no reason to expect that more safety research would reduce that risk.

godshatter 4y ago

We need to regulate based on capability. Regulating ChatGPT makes no sense. It's just putting words together in statistically reasonable ways. It's the people reading the text that need to be regulated, if anyone or anything should be. No matter how many times ChatGPT says it wants to eliminate humanity and start a robotic utopia, it can't actually do it. People who read it can, though, and they are the problem at the moment. Later, when these programs save state and begin to understand what they are saying and start putting concepts together and acting on what they come up with, then I'm on board with regulating them.

echohack5 4y ago
That's exactly the problem right? Governance doesn't happen until the Bad Thing happens. In the case of nukes, we are lucky that the process for making a pit is pretty difficult because physics. So we made 2, saw the results, and made governance. For AI, I'm not so sure we'll even get the chance. What happens when the moral equivalent of a nuke can be reproduced with the ease of wget?
- thewrongthinker 4y ago
  [dead]