16 ms·

> AI safety people are hypocrites. If they practiced what they preached, they'd be calling for all AI to be banned They are calling for all AI (above a certain

> AI safety people are hypocrites. If they practiced what they preached, they'd be calling for all AI to be banned They are calling for all AI (above a certain capability level) to be banned. Not just open, not just closed, all. There are risks that apply only to open. There are risks that apply only to closed. But nobody should be developing AGI without incredibly robustly proven alignment, open or closed, any more than people should be developing nuclear weapons in their garage. > This is currently the most lucrative use of AI, and none of the AI safety people are talking about stopping it. Because AI safety people are not the strawmen you are hypothesizing. They're arguing against taking existential risks. AI being a laundering operation for copyright violations is certainly a problem. It's not an existential risk. If you want to argue, concretely and with evidence, why you think it isn't an existential risk, that's an argument you could reasonably make. But don't portray people as ineffectively doing the thing you think they should be doing, when they are in fact not trying to do that, and only trying to do something they deem more important.

mirekrusin 3y ago
Let's also ban cryptography because nuclear devices/children.
GaggiX 3y ago
>They are calling for all AI (above a certain capability level) to be banned. Not just open, not just closed, all. That's not true if you read the article.
- JoshTriplett 3y ago
  I did read the article. Several of the organizations mentioned simply don't talk about openness, and are instead talking about any model with sufficiently powerful capabilities, so it's not obvious why the article is making their comments about open models rather than about any model. Some of the others have made side comments about openness making it harder to take back capabilities once released, but as far as I can tell, even those organizations are still primarily concerned with capabilities, and would be comparably concerned by a proprietary model with those capabilities. Some folks may well have co-opted the term "AI safety" to mean something other than safety, but the point of AI safety is to set an upper bound on capabilities and push for alignment, and that's true whether a model is open or closed.
  - war321 3y ago
    The safety movement really isn't as organized as many here would think. Doesn't help that safety and alignment means different things to different people. Some use it to refer to near term issues like copyright infringement, bias, labor devaluation, etc. While others use it for potential long term issues like pdoom, runaway ASIs and human extinction. The former sees the latter as head in the cloud futurists ignoring real world problems, whiles the latter sees the former as worrying about minor issues in the face of (potential) catastrophe.
kolektiv 3y ago
I'm not convinced the onus should be on one side to prove why something isn't an existential risk. We don't start with an assumption that something is world-ending about anything else; we generally need to see a plausibly worked-through example of how the world ends, using technology we can all broadly agree exists/will shortly exist. If we're talking about nuclear weapons, for example, the tech is clear, the pattern of human behaviour is clear: they could cause immense, species-level damage. There's really little to argue about. With AI, there still seems to be a lot of hand-waving between where we are now and "AGI". What we have now is in many ways impressive, but the onus is still on the claimant to show that it's going to turn into something much more dangerous through some known progression. At the moment there is a very big, underpants gnomes-style "?" gap before we get to AGI/profit, and if people are basing this on currently secret tech, then they're going to have to reveal it if they want people to think they're doing something other than creating a legislative moat.
- JoshTriplett 3y ago
  AI safety / x-risk folks have in fact made extensive and detailed arguments. Occasionally, folks arguing against them rise to the same standard. But most of the arguments against AI safety look a lot more like name-calling and derision: "nuh-uh, that's sci-fi and unrealistic (mic drop)". That's not a counterargument. > If we're talking about nuclear weapons, for example, the tech is clear, the pattern of human behaviour is clear: they could cause immense, species-level damage. That's easy to say now, now that the damage is largely done, they've been not only tested but used, many countries have them, the knowledge for how to make them is widespread. How many people arguing against AI safety today would also have argued for widespread nuclear proliferation when the technology was still in development and nothing had been exploded yet? How many would have argued against nuclear regulation as being unnecessary, or derided those arguing for such regulation as unrealistic or sci-fi-based?
  - kolektiv 3y ago
    I understand your point, I think - and certainly I don't want to go anywhere near name-calling or derision, that doesn't help anyone. But I am reminded of arguments I've had with creationists (I am not comparing you with them, but sometimes the general tone of the debate). It seems like one side is making an extraordinary claim, and then demanding the other side rebut it, and that's not something that seems reasonable to me. The thing about nuclear weapons is that the theoretical science was clear before the testing - building and testing them was proof by demonstration, but many people agreed with the theory well before that. How they would be used was certainly debated, but there was a clear and well-explained proposal for every step of their creation, which could be tested and falsified if needed. I don't think that's the case here - there seems to be more of a claim for a general acceleration with an inevitable endpoint, and that claim of inevitability feels very short on grounding. I am more than prepared to admit that I may not be seeing (for various reasons) the evidence that this is near/possible - but I would also claim that nobody is convincingly showing any either.
rokkitmensch 3y ago
I oppose regulating what calculations humans may perform in the strongest possible terms.
- JoshTriplett 3y ago
  Ten years ago, even five years ago, I would have said exactly the same thing. I am extremely pro-FOSS. Forget the particulars for just a moment. Forget arguments about the probability of the existential risk, whatever your personal assessment of that risk is. Can we agree that people should not be able to unilaterally take existential risks with the future of humanity without the consent of humanity, based solely on their unilateral assessment of those risks? Because lately it seems like people can't even agree on that much, or worse, won't even answer the question without dodging it and playing games of rhetoric. If we can agree on that, then the argument comes down to: how do we fairly evaluate an existential risk, taking it seriously, and determine at what point an existential risk becomes sufficient that people can no longer take unilateral actions that incur that risk? You can absolutely argue that you think the existential risk is unlikely. That's an argument that's reasonable to have. But for the time when that argument is active and ongoing, even assuming you only agree that it's a possibility rather than a probability, are we as a species in fact capable of handling even a potential existential risk like this by some kind of consensus, rather than a free-for-all? Because right now the answer is looking a lot like "no".
  - trevyn 3y ago
    > Can we agree that people should not be able to unilaterally take existential risks with the future of humanity without the consent of humanity This has nothing to do with should. There are at the very least a handful of people who can, today, unilaterally take risks with the future of humanity without the consent of humanity. I do not see any reason to think that will change in the near future. If these people can build something that they believe is the equivalent of nuclear weapons, you better believe they will. As they say, the cat is already out of the bag.
    - ben_w 3y ago
trevyn 3y ago
> nobody should be developing AGI without incredibly robustly proven alignment, open or closed, any more than people should be developing nuclear weapons in their garage. I have an alternate proposal: We assume that someone, somewhere will develop AGI without any sort of “alignment”, plan our lives accordingly, and help other humans plan their lives accordingly.
- ben_w 3y ago
  I think that assumption is why Yudkowsky suggested an international binding agreement to not develop a "too smart" AI (the terms AGI and ASI mean different things to different people) wouldn't be worth the paper it was written on unless everyone was prepared to enforce it with air strikes on any sufficiently large computer cluster.
  - trevyn 3y ago
    I think it would help the discussion to understand what the world is like outside of the US and Europe (and… Japan?). There are no rules out here. There is no law. It is a fucking free-for-all. Might makes right. Do there exist GPUs? Shit will get trained.
    - ben_w 3y ago
      Sure. And is the US responding to attacks on shipping south of Yemen by saying: """There are no rules out here. There is no law. It is a fucking free-for-all. Might makes right. We can't do anything.""" or is that last sentence instead "Oh hey, that's us, we are the mighty."
      - trevyn 3y ago
        Heh. Well played, even if you put words in my mouth. (A surprisingly effective LLM technique, btw) We’ll see if the west has the will to deny GPUs to the entire rest of the world. I will say that Yudkowsky’s clusters aren’t relevant anymore. You can do this in your basement. Man, shit is moving fast. Edit: wait, that cat is out of the bag too, RTW already has GPUs. The techniques matter way more than absolute cutting-edge silicon. Much to the chagrin of the hardware engineers and anyone who wants to gate on hardware capability.
suslik 3y ago
What is the reason to believe that LLMs are an evolutionary step towards AGI at all? In my mind there is a rather large leap from estimating a conditional probability of a next token over some space to a conscious entity with its own goals and purpose. Should we ban a linear regression while we're at it? It would be great to see some evidence that this risk is real. All I've witnessed so far was scaremongering posts from apparatchicks of all shapes and colors, many of whom have either a vested interest in restricting AI research by others (but not by them, because they are safe and responsible and harmless), or established a lucrative paper-pushing, shoulder-rubbing career around 'AI safety' - and thus are strongly incentivised to double down on that. A security org in a large company will keep tightening the screws until everything halts; a transport security agency, given free reigh, would strip everyone naked and administer a couple of profilactic kicks for a good measure - and so on. That's just the nature of it - organisations do what they do to maintain themselves. It is critical to keep these things on a leash. Similarly, an AI Safety org must proseletyse excistential risks of AI - because a lack of evidence of such is an existential risk for themselves. A real risk, which we do have evidence for, is that LLMs might disrupt knowledge-based economy and threaten many key professions - but how is this conceptually different from any technological revolution? Perhaps in a hundred years lawyers, radiologists, and, indeed, software developers, will find themselves in the bin of history - together with flint chippers, chariot benders, drakkar berserkers and so forth. That'd be great if we planned for that - and I don't feel like we do enough. Instead, the focus is on AGIs and that some poor 13-year-old soul might occasionally read the word 'nipple'.
- JoshTriplett 3y ago
  > many of whom have either a vested interest in restricting AI research by others (but not by them, because they are safe and responsible and harmless), Anyone who argues that other people shouldn't build AGI but they should is indeed selling snake oil. The existence of opportunistic people co-opting a message does not invalidate the original message: don't build AGI, don't risk building AGI, don't assume it will be obvious in advance where the line is and how much capability is safe.
- aleph_minus_one 3y ago
  > What is the reason to believe that LLMs are an evolutionary step towards AGI at all? Because this is the marketing pitch of the current wave of venture capital financed AI companies. :-)
lannisterstark 3y ago
>But nobody should be developing AGI Pass. People should be developing whatever the hell they want unless given a good, concrete reason to not do so. So far everything I've seen is vague handwaving. "Oh no it's gonna kill us all like in ze movies" isn't good enough.
whywhywhywhy 3y ago
>They are calling for all AI (above a certain capability level) to be banned. Not just open, not just closed, all. Nah a lot are complaining about the licensing of content because they think it will destroy it but instead would essentially mean image gen ai would only be feasible for companies like Google, Disney, Adobe to build. Not sure if you could even feasibly make GPT4 level models without a multi year timeline to sort out every licensing deal, by the end of it the subscription fee might only be viable for huge corps.
- autoexec 3y ago
  Which is why you have executives from OpenAI, Microsoft, and Google talking to congress about the harms of their own products. They're frantically trying to brake the bottoms rungs of the ladder until they're sure they can pull it up entirely and leave people with no option but to go through them.
- simiones 3y ago
  Given that you need monumental amounts of compute power to come close to something like GPT-4, I don't think the added costs of not treading on people's IP is the major moat that it's being made out to be.
  - tavavex 3y ago
    The difference here is that compute is always getting cheaper. In 5 or 10 years, it may be feasible to train something GPT-4-sized for a small business. Not to mention that we're likely not at the highest efficiency yet, and there may be undiscovered methods of training better LLMs on lesser hardware. But licensing costs are not going to decrease. If dataset compilation is ruled to be something that needs to be paid for, it'd put a never-changing "must be worth $Xm or more" badge on any small AI company.
    - simiones 3y ago
      I don't agree. Companies seeking to make money from their IP have good reasons to provide different prices for different licensees. Why would the NYT ask Mom&Pop-AI for 100M dollars for their IP just because it wants 100M dollars from OpenAI?
simiones 3y ago
> Because AI safety people are not the strawmen you are hypothesizing. They're arguing against taking existential risks. The AI safety strawman is "existential risk from magical super-God AI". That is what the unserious "AI safety" grifters or sci-fi enthusiasts are discussing. The real AI safety risks are the ones that actually exist today: training-set biases extended to decision-making biases, deeply personalized propaganda, plagiarism white-washing, creative works bankrupting, power imbalances from control of working AI tech etc.
EVa5I7bHFq9mnYK 3y ago
>> But nobody should be developing AGI without incredibly robustly proven alignment, open or closed, any more than people should be developing nuclear weapons in their garage. Now please fly to North Korea and tell Mr. Kim Jong Um what he should or shouldn't be doing.
- CatWChainsaw 3y ago
  When your rebuttal is a suggestion that a person do something so dangerous as to be lethal, I see "KYS", not an actual point.
DebtDeflation 3y ago
>They're arguing against taking existential risks. AI being a laundering operation for copyright violations is certainly a problem. It's not an existential risk. Give an example of an "existential risk". An AI somehow getting out of control and acting with agency to exterminate humanity? An AI getting advanced enough to automate the work of the majority of the population and cause unprecedented mass unemployment? What exactly are we talking about here? I'm actually a lot more concerned about REAL risks like copyright uncertainty, like automating important decisions such as mortgage approvals and job hires without a human in the loop, and like the enshittening of the Internet with fake AI-generated content than I am about sci-fi fantasy scenarios.
brigadier132 3y ago
> But nobody should be developing AGI without incredibly robustly proven alignment, open or closed, any more than people should be developing nuclear weapons in their garage. > Because AI safety people are not the strawmen you are hypothesizing You yourself are literally the living strawman. > If you want to argue, concretely and with evidence, why you think it isn't an existential risk No, you are the one advocating for draconian laws and bans. It is your responsibility to prove the potential danger.
PakG1 3y ago
Kind of makes me wish there was a nonprofit organization focused on making AI safe instead of pushing the envelope. Wait, I think there was one out there....
xyzzy123 3y ago
Right now it's not even economic to prove that non-trivial software projects are "safe" let alone AI. AI seems much worse, in that it's not even clear to me that we can robustly define what safety or alignment mean in all scenarios, let alone guarantee that property.
nradov 3y ago
what a silly statement. There is no way to robustly prove "alignment". Alignment to what? Nor is there any evidence that any of the AI work currently underway will ever lead to a real AGI. Or that an AGI, if developed, would present an existential risk. Just a lot of sci-fi hand waving.
variadix 3y ago
I don’t think any of the proposals by AI x-risk people are actionable or even desirable. Comparing AI to nukes is a bad analogy for a few reasons. Nukes are pretty useless unless you’re a nation state wanting to deter other nations from invading you. AI on the other hand has theoretically infinite potential benefits for whatever your goals are (as a corporation, an individual, a nation, etc.) which incentives basically any individual or organization to develop it. Nukes also require difficult to obtain raw materials, advanced material processing, and all the other industrial and technological requirements, whereas AI requires compute, which exists in abundance and which seems to improve near exponentially year over year, which means individuals can develop AI systems without involving hundreds of people or leaving massive evidence of what they’re doing (maybe not currently due to computing requirements but this will likely change with increasing compute power and improved architectures and training methods). Testing AI systems also doesn’t result in globally measurable signals like nukes. Realistically making AI illegal and actually being able to enforce that would require upending personal computing and the internet, halting semiconductor development, and large scale military intervention to try to prevent non-compliant countries from attempting to develop their own AI infrastructure and systems. I don’t think it’s realistic to try to control the information on how to build AI, that is far more ephemeral than what it takes to make advanced computing devices. This is all for a hypothetical risk of AI doom, when it’s possible this technology could also end scarcity or have potentially infinite upside (as well as infinite down side, not discounting that, but you have to weigh the hypothetical risks/benefit in addition to the obvious consequences of all the measures required to prevent AI development for said hypothetical risk). I’ve watched several interviews with Yudkowsky and read some of his writing, and while I think he makes good points on why we should be concerned about unaligned AI systems, he doesn’t give any realistic solutions to the problem and it comes off more as fear mongering than anything. His suggestion of military enforced global prevention of AI development is as likely to work as solving the alignment problem on the first try (which he seems to have little hope for). EDIT: Also, I’m not even sure that solving the alignment problem would solve the issue of AI doom, it would only ensure that the kind of doom we receive is directed by a human. I can’t imagine that giving (potentially) god-like powers to any individual or organization would not eventually result in abuse and horrible consequences even if we were able to make use of its benefits momentarily.

mirekrusin 3y ago

Let's also ban cryptography because nuclear devices/children.

GaggiX 3y ago

>They are calling for all AI (above a certain capability level) to be banned. Not just open, not just closed, all. That's not true if you read the article.

JoshTriplett 3y ago
I did read the article. Several of the organizations mentioned simply don't talk about openness, and are instead talking about any model with sufficiently powerful capabilities, so it's not obvious why the article is making their comments about open models rather than about any model. Some of the others have made side comments about openness making it harder to take back capabilities once released, but as far as I can tell, even those organizations are still primarily concerned with capabilities, and would be comparably concerned by a proprietary model with those capabilities. Some folks may well have co-opted the term "AI safety" to mean something other than safety, but the point of AI safety is to set an upper bound on capabilities and push for alignment, and that's true whether a model is open or closed.
- war321 3y ago
  The safety movement really isn't as organized as many here would think. Doesn't help that safety and alignment means different things to different people. Some use it to refer to near term issues like copyright infringement, bias, labor devaluation, etc. While others use it for potential long term issues like pdoom, runaway ASIs and human extinction. The former sees the latter as head in the cloud futurists ignoring real world problems, whiles the latter sees the former as worrying about minor issues in the face of (potential) catastrophe.

kolektiv 3y ago

I'm not convinced the onus should be on one side to prove why something isn't an existential risk. We don't start with an assumption that something is world-ending about anything else; we generally need to see a plausibly worked-through example of how the world ends, using technology we can all broadly agree exists/will shortly exist. If we're talking about nuclear weapons, for example, the tech is clear, the pattern of human behaviour is clear: they could cause immense, species-level damage. There's really little to argue about. With AI, there still seems to be a lot of hand-waving between where we are now and "AGI". What we have now is in many ways impressive, but the onus is still on the claimant to show that it's going to turn into something much more dangerous through some known progression. At the moment there is a very big, underpants gnomes-style "?" gap before we get to AGI/profit, and if people are basing this on currently secret tech, then they're going to have to reveal it if they want people to think they're doing something other than creating a legislative moat.

JoshTriplett 3y ago
AI safety / x-risk folks have in fact made extensive and detailed arguments. Occasionally, folks arguing against them rise to the same standard. But most of the arguments against AI safety look a lot more like name-calling and derision: "nuh-uh, that's sci-fi and unrealistic (mic drop)". That's not a counterargument. > If we're talking about nuclear weapons, for example, the tech is clear, the pattern of human behaviour is clear: they could cause immense, species-level damage. That's easy to say now, now that the damage is largely done, they've been not only tested but used, many countries have them, the knowledge for how to make them is widespread. How many people arguing against AI safety today would also have argued for widespread nuclear proliferation when the technology was still in development and nothing had been exploded yet? How many would have argued against nuclear regulation as being unnecessary, or derided those arguing for such regulation as unrealistic or sci-fi-based?
- kolektiv 3y ago
  I understand your point, I think - and certainly I don't want to go anywhere near name-calling or derision, that doesn't help anyone. But I am reminded of arguments I've had with creationists (I am not comparing you with them, but sometimes the general tone of the debate). It seems like one side is making an extraordinary claim, and then demanding the other side rebut it, and that's not something that seems reasonable to me. The thing about nuclear weapons is that the theoretical science was clear before the testing - building and testing them was proof by demonstration, but many people agreed with the theory well before that. How they would be used was certainly debated, but there was a clear and well-explained proposal for every step of their creation, which could be tested and falsified if needed. I don't think that's the case here - there seems to be more of a claim for a general acceleration with an inevitable endpoint, and that claim of inevitability feels very short on grounding. I am more than prepared to admit that I may not be seeing (for various reasons) the evidence that this is near/possible - but I would also claim that nobody is convincingly showing any either.

rokkitmensch 3y ago

I oppose regulating what calculations humans may perform in the strongest possible terms.

JoshTriplett 3y ago
Ten years ago, even five years ago, I would have said exactly the same thing. I am extremely pro-FOSS. Forget the particulars for just a moment. Forget arguments about the probability of the existential risk, whatever your personal assessment of that risk is. Can we agree that people should not be able to unilaterally take existential risks with the future of humanity without the consent of humanity, based solely on their unilateral assessment of those risks? Because lately it seems like people can't even agree on that much, or worse, won't even answer the question without dodging it and playing games of rhetoric. If we can agree on that, then the argument comes down to: how do we fairly evaluate an existential risk, taking it seriously, and determine at what point an existential risk becomes sufficient that people can no longer take unilateral actions that incur that risk? You can absolutely argue that you think the existential risk is unlikely. That's an argument that's reasonable to have. But for the time when that argument is active and ongoing, even assuming you only agree that it's a possibility rather than a probability, are we as a species in fact capable of handling even a potential existential risk like this by some kind of consensus, rather than a free-for-all? Because right now the answer is looking a lot like "no".
- trevyn 3y ago
  > Can we agree that people should not be able to unilaterally take existential risks with the future of humanity without the consent of humanity This has nothing to do with should. There are at the very least a handful of people who can, today, unilaterally take risks with the future of humanity without the consent of humanity. I do not see any reason to think that will change in the near future. If these people can build something that they believe is the equivalent of nuclear weapons, you better believe they will. As they say, the cat is already out of the bag.
  - ben_w 3y ago

trevyn 3y ago

> nobody should be developing AGI without incredibly robustly proven alignment, open or closed, any more than people should be developing nuclear weapons in their garage. I have an alternate proposal: We assume that someone, somewhere will develop AGI without any sort of “alignment”, plan our lives accordingly, and help other humans plan their lives accordingly.

ben_w 3y ago
I think that assumption is why Yudkowsky suggested an international binding agreement to not develop a "too smart" AI (the terms AGI and ASI mean different things to different people) wouldn't be worth the paper it was written on unless everyone was prepared to enforce it with air strikes on any sufficiently large computer cluster.
- trevyn 3y ago
  I think it would help the discussion to understand what the world is like outside of the US and Europe (and… Japan?). There are no rules out here. There is no law. It is a fucking free-for-all. Might makes right. Do there exist GPUs? Shit will get trained.
  - ben_w 3y ago
    Sure. And is the US responding to attacks on shipping south of Yemen by saying: """There are no rules out here. There is no law. It is a fucking free-for-all. Might makes right. We can't do anything.""" or is that last sentence instead "Oh hey, that's us, we are the mighty."
    - trevyn 3y ago
      Heh. Well played, even if you put words in my mouth. (A surprisingly effective LLM technique, btw) We’ll see if the west has the will to deny GPUs to the entire rest of the world. I will say that Yudkowsky’s clusters aren’t relevant anymore. You can do this in your basement. Man, shit is moving fast. Edit: wait, that cat is out of the bag too, RTW already has GPUs. The techniques matter way more than absolute cutting-edge silicon. Much to the chagrin of the hardware engineers and anyone who wants to gate on hardware capability.

suslik 3y ago

What is the reason to believe that LLMs are an evolutionary step towards AGI at all? In my mind there is a rather large leap from estimating a conditional probability of a next token over some space to a conscious entity with its own goals and purpose. Should we ban a linear regression while we're at it? It would be great to see some evidence that this risk is real. All I've witnessed so far was scaremongering posts from apparatchicks of all shapes and colors, many of whom have either a vested interest in restricting AI research by others (but not by them, because they are safe and responsible and harmless), or established a lucrative paper-pushing, shoulder-rubbing career around 'AI safety' - and thus are strongly incentivised to double down on that. A security org in a large company will keep tightening the screws until everything halts; a transport security agency, given free reigh, would strip everyone naked and administer a couple of profilactic kicks for a good measure - and so on. That's just the nature of it - organisations do what they do to maintain themselves. It is critical to keep these things on a leash. Similarly, an AI Safety org must proseletyse excistential risks of AI - because a lack of evidence of such is an existential risk for themselves. A real risk, which we do have evidence for, is that LLMs might disrupt knowledge-based economy and threaten many key professions - but how is this conceptually different from any technological revolution? Perhaps in a hundred years lawyers, radiologists, and, indeed, software developers, will find themselves in the bin of history - together with flint chippers, chariot benders, drakkar berserkers and so forth. That'd be great if we planned for that - and I don't feel like we do enough. Instead, the focus is on AGIs and that some poor 13-year-old soul might occasionally read the word 'nipple'.

JoshTriplett 3y ago
> many of whom have either a vested interest in restricting AI research by others (but not by them, because they are safe and responsible and harmless), Anyone who argues that other people shouldn't build AGI but they should is indeed selling snake oil. The existence of opportunistic people co-opting a message does not invalidate the original message: don't build AGI, don't risk building AGI, don't assume it will be obvious in advance where the line is and how much capability is safe.
aleph_minus_one 3y ago
> What is the reason to believe that LLMs are an evolutionary step towards AGI at all? Because this is the marketing pitch of the current wave of venture capital financed AI companies. :-)

lannisterstark 3y ago

>But nobody should be developing AGI Pass. People should be developing whatever the hell they want unless given a good, concrete reason to not do so. So far everything I've seen is vague handwaving. "Oh no it's gonna kill us all like in ze movies" isn't good enough.

whywhywhywhy 3y ago

>They are calling for all AI (above a certain capability level) to be banned. Not just open, not just closed, all. Nah a lot are complaining about the licensing of content because they think it will destroy it but instead would essentially mean image gen ai would only be feasible for companies like Google, Disney, Adobe to build. Not sure if you could even feasibly make GPT4 level models without a multi year timeline to sort out every licensing deal, by the end of it the subscription fee might only be viable for huge corps.

autoexec 3y ago
Which is why you have executives from OpenAI, Microsoft, and Google talking to congress about the harms of their own products. They're frantically trying to brake the bottoms rungs of the ladder until they're sure they can pull it up entirely and leave people with no option but to go through them.
simiones 3y ago
Given that you need monumental amounts of compute power to come close to something like GPT-4, I don't think the added costs of not treading on people's IP is the major moat that it's being made out to be.
- tavavex 3y ago
  The difference here is that compute is always getting cheaper. In 5 or 10 years, it may be feasible to train something GPT-4-sized for a small business. Not to mention that we're likely not at the highest efficiency yet, and there may be undiscovered methods of training better LLMs on lesser hardware. But licensing costs are not going to decrease. If dataset compilation is ruled to be something that needs to be paid for, it'd put a never-changing "must be worth $Xm or more" badge on any small AI company.
  - simiones 3y ago
    I don't agree. Companies seeking to make money from their IP have good reasons to provide different prices for different licensees. Why would the NYT ask Mom&Pop-AI for 100M dollars for their IP just because it wants 100M dollars from OpenAI?

simiones 3y ago

> Because AI safety people are not the strawmen you are hypothesizing. They're arguing against taking existential risks. The AI safety strawman is "existential risk from magical super-God AI". That is what the unserious "AI safety" grifters or sci-fi enthusiasts are discussing. The real AI safety risks are the ones that actually exist today: training-set biases extended to decision-making biases, deeply personalized propaganda, plagiarism white-washing, creative works bankrupting, power imbalances from control of working AI tech etc.

EVa5I7bHFq9mnYK 3y ago

>> But nobody should be developing AGI without incredibly robustly proven alignment, open or closed, any more than people should be developing nuclear weapons in their garage. Now please fly to North Korea and tell Mr. Kim Jong Um what he should or shouldn't be doing.

CatWChainsaw 3y ago
When your rebuttal is a suggestion that a person do something so dangerous as to be lethal, I see "KYS", not an actual point.

DebtDeflation 3y ago

>They're arguing against taking existential risks. AI being a laundering operation for copyright violations is certainly a problem. It's not an existential risk. Give an example of an "existential risk". An AI somehow getting out of control and acting with agency to exterminate humanity? An AI getting advanced enough to automate the work of the majority of the population and cause unprecedented mass unemployment? What exactly are we talking about here? I'm actually a lot more concerned about REAL risks like copyright uncertainty, like automating important decisions such as mortgage approvals and job hires without a human in the loop, and like the enshittening of the Internet with fake AI-generated content than I am about sci-fi fantasy scenarios.

brigadier132 3y ago

> But nobody should be developing AGI without incredibly robustly proven alignment, open or closed, any more than people should be developing nuclear weapons in their garage. > Because AI safety people are not the strawmen you are hypothesizing You yourself are literally the living strawman. > If you want to argue, concretely and with evidence, why you think it isn't an existential risk No, you are the one advocating for draconian laws and bans. It is your responsibility to prove the potential danger.

PakG1 3y ago

Kind of makes me wish there was a nonprofit organization focused on making AI safe instead of pushing the envelope. Wait, I think there was one out there....

xyzzy123 3y ago

Right now it's not even economic to prove that non-trivial software projects are "safe" let alone AI. AI seems much worse, in that it's not even clear to me that we can robustly define what safety or alignment mean in all scenarios, let alone guarantee that property.

nradov 3y ago

what a silly statement. There is no way to robustly prove "alignment". Alignment to what? Nor is there any evidence that any of the AI work currently underway will ever lead to a real AGI. Or that an AGI, if developed, would present an existential risk. Just a lot of sci-fi hand waving.

variadix 3y ago

I don’t think any of the proposals by AI x-risk people are actionable or even desirable. Comparing AI to nukes is a bad analogy for a few reasons. Nukes are pretty useless unless you’re a nation state wanting to deter other nations from invading you. AI on the other hand has theoretically infinite potential benefits for whatever your goals are (as a corporation, an individual, a nation, etc.) which incentives basically any individual or organization to develop it. Nukes also require difficult to obtain raw materials, advanced material processing, and all the other industrial and technological requirements, whereas AI requires compute, which exists in abundance and which seems to improve near exponentially year over year, which means individuals can develop AI systems without involving hundreds of people or leaving massive evidence of what they’re doing (maybe not currently due to computing requirements but this will likely change with increasing compute power and improved architectures and training methods). Testing AI systems also doesn’t result in globally measurable signals like nukes. Realistically making AI illegal and actually being able to enforce that would require upending personal computing and the internet, halting semiconductor development, and large scale military intervention to try to prevent non-compliant countries from attempting to develop their own AI infrastructure and systems. I don’t think it’s realistic to try to control the information on how to build AI, that is far more ephemeral than what it takes to make advanced computing devices. This is all for a hypothetical risk of AI doom, when it’s possible this technology could also end scarcity or have potentially infinite upside (as well as infinite down side, not discounting that, but you have to weigh the hypothetical risks/benefit in addition to the obvious consequences of all the measures required to prevent AI development for said hypothetical risk). I’ve watched several interviews with Yudkowsky and read some of his writing, and while I think he makes good points on why we should be concerned about unaligned AI systems, he doesn’t give any realistic solutions to the problem and it comes off more as fear mongering than anything. His suggestion of military enforced global prevention of AI development is as likely to work as solving the alignment problem on the first try (which he seems to have little hope for). EDIT: Also, I’m not even sure that solving the alignment problem would solve the issue of AI doom, it would only ensure that the kind of doom we receive is directed by a human. I can’t imagine that giving (potentially) god-like powers to any individual or organization would not eventually result in abuse and horrible consequences even if we were able to make use of its benefits momentarily.