9 ms·

Any laws should be legislating the downstream effects of AI, not the models themselves. Otherwise we will quickly get to a place where we have a handful of gove

Any laws should be legislating the downstream effects of AI, not the models themselves. Otherwise we will quickly get to a place where we have a handful of government-sanctioned and white-washed “safe” models responsible for deleterious effects on society, with plausible deniability for the companies abusing them. Legislating around the model is missing the point. There is no evidence that a runaway artificial intelligence is even possible. The focus on this is going to distract us from the real and current issues with strong AI. The real risks are societal instability due to: - Rapid disruption of the labor market - Astroturfing, psyops, and disruption of general trust (commercially and maliciously, both domestic and foreign) - Crippling of our domestic ai capabilities, leading to cutting edge development moving overseas, and a loss of our ability to influence further development. - Increased concentration of power and disruption of decentralized and democratic forms of organization due to all of the above.

deleted 3y ago
[deleted]
circuit10 3y ago
> There is no evidence that a runaway artificial intelligence is even plausible Really? There is a lot of theory behind why this is likely to happen and if you want a real example there is a similar existing scenario we can look at, which is how humans have gone through an exponential runaway explosion in capabilities in the last few hundred years because of being more intelligent than other species and being able to improve our own capabilities through tool use (in the case of AI it can directly improve itself so it would likely be much faster and there would be less of a cap on it as we have the bottleneck of not being able to improve our own intelligence much)
- civilitty 3y ago
  Yes, really. Scifi fantasies don’t count as evidence and we’ve learned since the Renaissance and scientific revolution that all this Platonic theorizing is just an intellectual circle jerk.
  - circuit10 3y ago
    The fact that something has been covered in sci-fi doesn’t mean that it can’t happen. https://en.m.wikipedia.org/wiki/Appeal_to_the_stone https://en.m.wikipedia.org/wiki/Appeal_to_the_stone “Speaker A: Infectious diseases are caused by tiny organisms that are not visible to unaided eyesight. Speaker B: Your statement is false. Speaker A: Why do you think that it is false? Speaker B: It sounds like nonsense. Speaker B denies Speaker A's claim without providing evidence to support their denial.” Also I gave a real world example that wasn’t related to sci-fi in any way
    - Al0neStar 3y ago
      The burden of proof is on the person making the initial claim and i highly doubt that the reasoning behind AI ruin is as extensive and compelling as the germ theory of disease. We assume human-level general AI is possible because we exist in nature but a super-human self-optimizing AI god is nowhere to be found.
      - yeck 3y ago
        Which claim does the burden of proof land on? That an artificial super intelligence can easily be controlled or that it cannot? And what is your rational for deciding?
        Al0neStar 3y ago
        The claim that there's a possibility of a sudden intelligence explosion. Like i said above you can argue that an AGI can be realized because there's plenty of us running around on earth but claims about a hypothetical super AGI are unfounded and akin to Russell's Teapot.
        pixl97 3y ago
        Ok, in a converse question, why would evolution create the human mind as the most intelligent possible system? So I ask, are systems capable of better than human intelligence possible? And if so to what factor? I would answer yes and unknown. If there are states of intelligent far greater than human then that gets really risky very fast.
- mmaunder 3y ago
  Humans evolved unsupervised. AI is highly supervised. The idea that an AI will enslave us all is as absurd as suggesting “computers” will enslave us all merely because they exist. Models are designed and operated by people for specific use cases. The real risks are people using this new tool for evil, not the tool itself. AI sentience is a seductive concept being used by self professed experts to draw attention to themselves and by mega corps to throw up competitive barriers to entry.
  - sorokod 3y ago
    A fair amount of effort is spent on autonomous models, e.g. driving. Who knows what the military are up to.
  - circuit10 3y ago
    It’s almost impossible to supervise something more intelligent than you because you can’t tell why it’s doing things. For now it’s easy to supervise them because AIs are way less intelligent than humans (though even now it’s hard to tell exactly why they’re doing things), but in the future it probably won’t be
    - z3c0 3y ago
      but in the future it probably won’t be I see this parroted so often, and I have to ask: Why? What is there outside of the world of SciFi that makes the AGI of the future so nebulous, when humans would have presumably advanced to a point to be able to create the intelligence to begin with. Emergent properties are often as unexpected as they are bizarre, but they are not unexplainable, especially when you understand the underpinning systems.
      - circuit10 3y ago
        We can’t even fully explain how our own brains work, never mind a system that’s completely alien to us and that would have to be more complex. We can’t even explain how current LLMs work internally. Maybe we’ll make some breakthrough if we put enough resources into it but if people keep denying the problem there will never be enough resources out into it
- dwallin 3y ago
  The theories all inevitably rely on assumptions that are essentially the equivalence of spherical cows in a frictionless universe. All evidence is that costs for intelligence likely scale superlinearly. Each increase in intelligence capability requires substantially more resources (Computing power, training data, electricity, hardware, time, etc). Being smart doesn’t just directly result in these becoming available with no limit. Any significant attempts to increase the availability of these to a level that mattered would almost certainly draw attention. In addition, even for current AI we don’t even fully understand what we are doing, even though they are operating at a lower generalized intelligence level than us. Since we don’t have a solid foundational model for truly understanding intelligence, progress relies heavily on experimentation to see what works. (Side note: my gut is that we will find there’s some sort of equivalent to the halting problem when it comes to understanding intelligence) It’s extremely likely that this remains true, even for artificial intelligence. In order for an AI to improve upon itself, it would likely also need to do significant experimentation, with diminishing returns and exponentially increasing costs for each level of improvement it achieves. In addition, a goal-oriented generalized AI would have the same problems that you worry about. In trying to build a superior intelligence to itself it risks building something that undermines its own goals. This increases the probability of either us, or a goal-aligned AI, noticing and being able to stop things from escalating. It also means that a super intelligent AI has disincentives to build better AIs.
  - arisAlexis 3y ago
    "In addition, even for current AI we don’t even fully understand what we are doing" That is the problem, don't you get it?
    - ls612 3y ago
      “In addition, even with the current state of the internet, we don’t have understanding everything we are doing with it” -some guy in the ‘90s probably
      - pixl97 3y ago
        Many people would look at engagement algorithms braching out of social media and causing riots and uprisings as one of those issues that would be difficult to predict in the 90s
- AbrahamParangi 3y ago
  Theory is not actually a form of evidence.
  - circuit10 3y ago
    Theory can be used to predict things with reasonable confidence. It could be wrong, but assuming it’s wrong is a big risk to take. Also I gave a real-world analogy that has actually happened
    - AnimalMuppet 3y ago
      An accurate theory can be used to predict things with reasonable confidence, within the limits of the theory. We don't have an accurate theory of intelligence. What we have now is at the "not even wrong" stage. Assuming it's wrong is about like assuming that alchemy is wrong.
      - pixl97 3y ago
        Then let's go with an observation. Humans got smarter than most large mammals, then most large mammals went extinct.
- throwaway9274 3y ago
  Is there really “a lot of theory” that says runaway AI is possible? In the sense of empirical fact-based peer reviewed machine learning literature? Because if so I must have missed it. It seems more accurate to say there is quite a bit of writing done by vocal influencers who frequent a couple online forums.
  - pixl97 3y ago
    In evolutionary biological systems 'capabilities break out' common enough in viral and bacterial systems with their fast evolution times. The same thing can occur when new predators are introduced to ecosystems that are not prepared for them. The question that I assume you have is 'can AI systems experience large evolutionary jumps in capabilities'. I would say yes myself, but you apper to disagree.
  - DirkH 3y ago
    There was also only "a lot of theory" and zero empirical evidence that nuclear bombs were possible not that long ago. Theoretical fact-based peer reviewed literature often (though not always) precedes empirical evidence. We shouldn't jump the gun and say runaway AI is inevitable just because there is fact-based theory on its possibility. Just like it would have been dumb to just look at E=mc^2 and conclude nuclear weapon development is 100% inevitable. But in the same breath we shouldn't say it's impossible just because we don't have empirical evidence either.
    - throwaway9274 3y ago
      Prior to the development of nuclear weapons, there was well-grounded peer reviewed theory in the physics literature. There were predictions that could be made by the theoretical frameworks, and experiments to confirm those predictions. The literature well described the composition of atoms, the decay of radioactive elements, the concept of fission, and the conversion of mass to energy. No such groundwork exists to substantiate the claim that runaway AI is possible. LessWrong, the so-called Alignment Forum, and similar Internet groups propose thought experiments. They couch these thought experiments in the guise of inevitable outcomes. The prophesies of the influencers originating there have no more relationship to the truth than the average enthusiast’s daydreams. The content there fills a role similar to science fiction in the atomic age: fanciful speculation on the properties of imaginary systems. Over time, the bad content is forgotten, and a handful of items survive and look prescient via selection bias.
- voidhorse 3y ago
  You do realize the "AI" we're talking about here is just LLMs, right? Language is a huge part of being human, so that's a good elements but: 1. There's no evidence that these LLMs have any sort of actual understanding of what they produce. 2. Being human is a lot more than just linguistic capability. We are embodied in a certain way, can interact with environments, have rich, multi-sensory experiences etc. tbh I think anyone that believes AGI can arise from an LLM is extremely naive and has basically no idea of the history of this track of research or how it works. To assert that LLMs will be capable of achieving consciousness is equivalent to asserting that human consciousness is reducible to statistical inference, which is palpably not true. (see. e.g. deductive logic). The only reason people even fall into the trap of thinking computers are remotely close to getting to AGI is because it became very popular to reason about the brain using analogies from computer science in the 50s, despite the fact that this was completely unjustified, and people glob on to the metaphors and then have the mypoia to actually think the human brain (which is exceedingly complex) is reducible to a freakin' von neuman architecture, lol. Biological systems and inorganic systems are fundamentally different.
  - flangola7 3y ago
    I don't think anyone is talking about consciousness here. Consciousness is tangential to safety concerns.
z3c0 3y ago
Astroturfing, psyops, and disruption of general trust (commercially and maliciously, both domestic and foreign) It is disturbing to me how unconcerned everybody is with this over what is still only a hypothetical problem. States and businesses have been long employing subversive techniques to corral people towards their goals, and they all just got an alarmingly useful tool for automated propaganda. This is a problem right now, not hypothetically. All these people aching to be a Cassandra should rant and rave about that.
- pixl97 3y ago
  I mean it's not hypothetical, there are shit loads of bots on social media already causing problems. Now the bots these days have a human controller but we can see strife they cause in politics already.
  - z3c0 3y ago
    We're in accordance.
hn_throwaway_99 3y ago
Definitely agree. I would summarize it a bit differently, but when people talk about AI dangers they are usually talking about 1 of 4 different things: 1. AI eventually takes control and destroys humans (i.e. the Skynet concern). 2. AI further ingrains already existing societal biases (sexism, racism, etc.) to the detriment of things like fair employment, fair judicial proceedings, etc. 3. AI makes large swaths of humanity unemployable, and we've never been able to design an economic system that can handle that. 4. AI supercharges already widely deployed psyops campaigns for disinformation, inciting division and violence, etc. The thing I find so aggravating is I see lots of media and self-professed AI experts focused on #1, I see lots of "Ethical AI" people solely focused on #2, but I see comparatively little focus on #3 and #4, which as you say are both happening right now. IMO #3 and #4 are far more likely to result in societal collapse than the first two issues.
- usaar333 3y ago
  #1 has high focus more due to impact than high probability. #2 doesn't seem talked about much at this point and seems to be pivoting more to #3. #2 never had much of a compelling argument given auditability. #3 gets mainly ignored due to Luddite assumptions driving it. I'm dubious myself over the short term - humans will have absolute advantage in many fields for a long time (especially with robotics lagging and being costly). #4 is risky, but humans can adapt. I see collapse as unlikely.
  - pwdisswordfishc 3y ago
    > I'm dubious myself over the short term - humans will have absolute advantage in many fields for a long time AI doesn’t have to be better at the humans’ job to unemploy them. It’s enough that its output looks presentable enough for advertising most of the time, that it never asks for a day off, never gets sick or retires, never refuses orders, never joins a union… The capitalist doesn’t really care about having the best product to sell, they only care about having the lowest-cost product they can get away with selling.
  - verall 3y ago
    > #2 never had much of a compelling argument given auditability. There are already AI systems which generate "scores" based on likelyhood to commit another crime (used in parole cases) and likelyhood to be a "good tenant" (used by landlords). Don't underestimate the difficulty to fix something when the people in charge of that thing do not want it fixed.
davidzweig 3y ago
>> There is no evidence that a runaway artificial intelligence is even possible. In the space of a century or so, the humans have managed to take rocks and sand and turn them into something that you can talk to with your voice, and it understands and responds fairly convincingly as it were a well-read human (glue together chatGPT with TTS/ASR). Doesn't seem like a big stretch to imagine that superhuman AI is just a few good ideas away, a decade or two perhaps.
Animats 3y ago
> Any laws should be legislating the downstream effects of AI, not the models themselves. That would require stronger consumer protections. So that's politically unacceptable in the US at the moment. We may well see it in the EU. The EU already regulates "automated decision making" as it affects EU citizens. This is part of the General Protection on Data Regulation. This paper discusses the application of those rules to AI systems.[1] Key points summary: - AI isn't special for regulation purposes. "First, the concept of Automated Decision Making includes algorithmic decision-making as well as AI-driven decision-making." - Guiding Principle 1: Law-compliant ADM. An operator that decides to use ADM for a particular purpose shall ensure that the design and the operation of the ADM are compliant with the laws applicable to an equivalent non- automated decision-making system. - Guiding Principle 2: ADM shall not be denied legal effect, validity or enforceability solely on the grounds that it is automated. - Guiding Principle 3: The operator has to assume the legal effects and bear the consequences of the ADM’s decision. ("Operator" here means the seller or offerer of the system, not the end user.) - Guiding Principle 4: It shall be disclosed that the decision is being made by automated means - Guiding Principle 5: Traceable decisions - Guiding Principle 6: The complexity, the opacity or the unpredictability of ADM is not a valid ground for rendering an unreasoned, unfounded or arbitrary decision. - Guiding Principle 7: The risks that the ADM may cause any harm or damage shall be allocated to the operator. - Guiding Principle 8: Automation shall not prevent, limit, or render unfeasible the exercise of rights and access to justice by affected persons. An alternative human-based route to exercise rights should be available. - Guiding Principle 9: The operator shall ensure reasonable and proportionate human oversight over the operation of ADM taking into consideration the risks involved and the rights and legitimate interests potentially affected by the decision. - Guiding Principle 10: Human review of significant decisions Human review of selected significant decisions on the grounds of the relevance of the legal effects, the irreversibility of their consequences, or the seriousness of the impact on rights and legitimate interests shall be made available by the operator. This is just a summary. The full text has examples, which include, without naming names, Google closing accounts and Uber firing drivers automatically. [1] https://europeanlawinstitute.eu/fileadmin/user_upload/p_eli/Publications/ELI_Innovation_Paper_on_Guiding_Principles_for_ADM_in_the_EU.pdf https://europeanlawinstitute.eu/fileadmin/user_upload/p_eli/...
gmerc 3y ago
Unaligned AI as an existential threat is an interesting topic but I feel we already know the answer to this one: It’s not like for the last few decades, we haven’t created an artificial, incentive based system at global scale that’s showing exactly how this will go. It’s not like a bunch of autonomous entities with a single prime directive, profit maximization are running our planet and are affecting how we live, how we structure of every day of our lives, control every aspect of our potential and destiny. Autonomous entities operating on reinforcement cycles driven by reward / punishment rules aligning about every human on this planet to their goals, right? It’s not like the enitities in this system are self improving towards a measurement and reward maximization and, as a result, command resources asserting normative (lobbying) power over the autonomy, self governance and control of people and their systems is governance. It’s not like we don’t know that this artificial system is unaligned with sustainability and survival of the human race, let alone happiness, freedom or love. We can watch its effects in real time accelerating towards our destruction under the yellow skies of New York, the burning steppes of Canada, the ashen hell or flood ridden plains of Australia, the thawing permafrost of Siberia, scorching climate affected cities of Southeast Asia, the annual haze from platantion burns in Indonesia, suffocating smog in Thailand, and the stripped bare husks of Latin American rainforests. And we know instinctively we are no longer in control, the system operating at larger than national scale, having long overpowered the systems of human governance, brute forcing everything and everyone on the planet into their control. But we pretend to otherwise, argue, pass measures doctoring symptoms, not mentioning the elephant in the room. But, one may protest, the vaunted C-Level control the entities, we say as Zuck and Co lament having to lay off humans, sobbing about responsibility to the prime directive. But politicians are, we pray as lobbyists, the human agents of our alien overlords bend them to their will. The alien entities we call corporations have no prime directive of human survival, sustainability, happiness and they already run everything. So one may be excused for having cynical views about the debate on whether unaligned AI is an existential, extinction level risk for us, whether humans could give creation to an unaligned system that could wipe them from the face of the planet. Our stories, narratives, the tales of the millennia apex predator of this planet have little room for the heresy of not being on top, in control. So deep goes out immersion in our own manifest destiny and in control identity, any challenge to the mere narrative is met with screetches and denigrations. In a throwback to the age of the heliocentricity debate - Galileo just broke decorum, spelling out what scientists knew for hundreds of years - the scientists and people devoted to understanding the technology are met with brandings of Doomsayer and heresy. Just as the earth being the center of the universe anchored our belief of being special, our intelligence, creativity or ability to draw hands is the pedestal these people have chosen to put their hands on with warnings of unaligned systemic entities. “It’s not human” is the last but feeble defense of the mind, failing to see the obvious. That the artificial system we created for a hundred years does not need people to be human, it just needs them to labor. It matters not that we can feel, love, express emotion and conjure dreams and hopes and offer human judgement for our jobs do not require it. Autonomy is not a feature of almost every human job, judgement replaced by corporate policies and rules. It matters not to the corporation that we need food to eat as it controls the resources to buy it, the creation of artificial labor is inevitably goal aligned with this system. Intelligence, let alone super intelligence is not a feature needed for most jobs or a system to take control over the entire planet. Our stories conjure super villains to make us believe we are in control, our movies no more than religious texts to the gospel of human exceptionalism. Show us the evidence they scream, as they did to Galileo, daring him to challenge the clear hand of god in all of creation. Us, unable to control a simple system we conjured into existence from rules and incentives operating on fallible meatsuits, having a chance to control a system of unparalleled processing power imbued with the combined statistical corpus of human knowledge, behavior, flaws and weaknesses? Laughable. Us, who saw social media codify the rules and incentives in digital systems powered by the precursor AI of today and watched the system helplessly a/b optimize towards maximum exploitation of human weaknesses for alignment with growth and profit containing the descent AI systems powered by orders of magnitude more capable systems or quantum computing? A snow flake may as well outlast hell. Us, a race with a 100% failure rate to find lasting governing structures optimizing for human potential not slipping in the face of an entity that only requires a single slip? An entity with perfect knowledge of the rules that bind us? Preposterous. Evidence indeed. “But we are human” will echo as the famous last words through the cosmos as our atoms are reconfigured into bitcoin storage to hold the profits of unbounded growth. What remains will not be human anymore, an timeless reckoning to the power of rules, incentives and consumption, eating world after world to satisfy the prime directive. But we always have hope. As our tales tell us, it dies last and it’s the most remote property for an AI to achieve. It may master confidence, assertiveness and misdirection, but hope? That may be the last human refuge for the coming storm.

deleted 3y ago

[deleted]

circuit10 3y ago

> There is no evidence that a runaway artificial intelligence is even plausible Really? There is a lot of theory behind why this is likely to happen and if you want a real example there is a similar existing scenario we can look at, which is how humans have gone through an exponential runaway explosion in capabilities in the last few hundred years because of being more intelligent than other species and being able to improve our own capabilities through tool use (in the case of AI it can directly improve itself so it would likely be much faster and there would be less of a cap on it as we have the bottleneck of not being able to improve our own intelligence much)

civilitty 3y ago
Yes, really. Scifi fantasies don’t count as evidence and we’ve learned since the Renaissance and scientific revolution that all this Platonic theorizing is just an intellectual circle jerk.
- circuit10 3y ago
  The fact that something has been covered in sci-fi doesn’t mean that it can’t happen. https://en.m.wikipedia.org/wiki/Appeal_to_the_stone https://en.m.wikipedia.org/wiki/Appeal_to_the_stone “Speaker A: Infectious diseases are caused by tiny organisms that are not visible to unaided eyesight. Speaker B: Your statement is false. Speaker A: Why do you think that it is false? Speaker B: It sounds like nonsense. Speaker B denies Speaker A's claim without providing evidence to support their denial.” Also I gave a real world example that wasn’t related to sci-fi in any way
  - Al0neStar 3y ago
    The burden of proof is on the person making the initial claim and i highly doubt that the reasoning behind AI ruin is as extensive and compelling as the germ theory of disease. We assume human-level general AI is possible because we exist in nature but a super-human self-optimizing AI god is nowhere to be found.
    - yeck 3y ago
      Which claim does the burden of proof land on? That an artificial super intelligence can easily be controlled or that it cannot? And what is your rational for deciding?
      - Al0neStar 3y ago
        The claim that there's a possibility of a sudden intelligence explosion. Like i said above you can argue that an AGI can be realized because there's plenty of us running around on earth but claims about a hypothetical super AGI are unfounded and akin to Russell's Teapot.
        pixl97 3y ago
        Ok, in a converse question, why would evolution create the human mind as the most intelligent possible system? So I ask, are systems capable of better than human intelligence possible? And if so to what factor? I would answer yes and unknown. If there are states of intelligent far greater than human then that gets really risky very fast.
mmaunder 3y ago
Humans evolved unsupervised. AI is highly supervised. The idea that an AI will enslave us all is as absurd as suggesting “computers” will enslave us all merely because they exist. Models are designed and operated by people for specific use cases. The real risks are people using this new tool for evil, not the tool itself. AI sentience is a seductive concept being used by self professed experts to draw attention to themselves and by mega corps to throw up competitive barriers to entry.
- sorokod 3y ago
  A fair amount of effort is spent on autonomous models, e.g. driving. Who knows what the military are up to.
- circuit10 3y ago
  It’s almost impossible to supervise something more intelligent than you because you can’t tell why it’s doing things. For now it’s easy to supervise them because AIs are way less intelligent than humans (though even now it’s hard to tell exactly why they’re doing things), but in the future it probably won’t be
  - z3c0 3y ago
    but in the future it probably won’t be I see this parroted so often, and I have to ask: Why? What is there outside of the world of SciFi that makes the AGI of the future so nebulous, when humans would have presumably advanced to a point to be able to create the intelligence to begin with. Emergent properties are often as unexpected as they are bizarre, but they are not unexplainable, especially when you understand the underpinning systems.
    - circuit10 3y ago
      We can’t even fully explain how our own brains work, never mind a system that’s completely alien to us and that would have to be more complex. We can’t even explain how current LLMs work internally. Maybe we’ll make some breakthrough if we put enough resources into it but if people keep denying the problem there will never be enough resources out into it
dwallin 3y ago
The theories all inevitably rely on assumptions that are essentially the equivalence of spherical cows in a frictionless universe. All evidence is that costs for intelligence likely scale superlinearly. Each increase in intelligence capability requires substantially more resources (Computing power, training data, electricity, hardware, time, etc). Being smart doesn’t just directly result in these becoming available with no limit. Any significant attempts to increase the availability of these to a level that mattered would almost certainly draw attention. In addition, even for current AI we don’t even fully understand what we are doing, even though they are operating at a lower generalized intelligence level than us. Since we don’t have a solid foundational model for truly understanding intelligence, progress relies heavily on experimentation to see what works. (Side note: my gut is that we will find there’s some sort of equivalent to the halting problem when it comes to understanding intelligence) It’s extremely likely that this remains true, even for artificial intelligence. In order for an AI to improve upon itself, it would likely also need to do significant experimentation, with diminishing returns and exponentially increasing costs for each level of improvement it achieves. In addition, a goal-oriented generalized AI would have the same problems that you worry about. In trying to build a superior intelligence to itself it risks building something that undermines its own goals. This increases the probability of either us, or a goal-aligned AI, noticing and being able to stop things from escalating. It also means that a super intelligent AI has disincentives to build better AIs.
- arisAlexis 3y ago
  "In addition, even for current AI we don’t even fully understand what we are doing" That is the problem, don't you get it?
  - ls612 3y ago
    “In addition, even with the current state of the internet, we don’t have understanding everything we are doing with it” -some guy in the ‘90s probably
    - pixl97 3y ago
      Many people would look at engagement algorithms braching out of social media and causing riots and uprisings as one of those issues that would be difficult to predict in the 90s
AbrahamParangi 3y ago
Theory is not actually a form of evidence.
- circuit10 3y ago
  Theory can be used to predict things with reasonable confidence. It could be wrong, but assuming it’s wrong is a big risk to take. Also I gave a real-world analogy that has actually happened
  - AnimalMuppet 3y ago
    An accurate theory can be used to predict things with reasonable confidence, within the limits of the theory. We don't have an accurate theory of intelligence. What we have now is at the "not even wrong" stage. Assuming it's wrong is about like assuming that alchemy is wrong.
    - pixl97 3y ago
      Then let's go with an observation. Humans got smarter than most large mammals, then most large mammals went extinct.
throwaway9274 3y ago
Is there really “a lot of theory” that says runaway AI is possible? In the sense of empirical fact-based peer reviewed machine learning literature? Because if so I must have missed it. It seems more accurate to say there is quite a bit of writing done by vocal influencers who frequent a couple online forums.
- pixl97 3y ago
  In evolutionary biological systems 'capabilities break out' common enough in viral and bacterial systems with their fast evolution times. The same thing can occur when new predators are introduced to ecosystems that are not prepared for them. The question that I assume you have is 'can AI systems experience large evolutionary jumps in capabilities'. I would say yes myself, but you apper to disagree.
- DirkH 3y ago
  There was also only "a lot of theory" and zero empirical evidence that nuclear bombs were possible not that long ago. Theoretical fact-based peer reviewed literature often (though not always) precedes empirical evidence. We shouldn't jump the gun and say runaway AI is inevitable just because there is fact-based theory on its possibility. Just like it would have been dumb to just look at E=mc^2 and conclude nuclear weapon development is 100% inevitable. But in the same breath we shouldn't say it's impossible just because we don't have empirical evidence either.
  - throwaway9274 3y ago
    Prior to the development of nuclear weapons, there was well-grounded peer reviewed theory in the physics literature. There were predictions that could be made by the theoretical frameworks, and experiments to confirm those predictions. The literature well described the composition of atoms, the decay of radioactive elements, the concept of fission, and the conversion of mass to energy. No such groundwork exists to substantiate the claim that runaway AI is possible. LessWrong, the so-called Alignment Forum, and similar Internet groups propose thought experiments. They couch these thought experiments in the guise of inevitable outcomes. The prophesies of the influencers originating there have no more relationship to the truth than the average enthusiast’s daydreams. The content there fills a role similar to science fiction in the atomic age: fanciful speculation on the properties of imaginary systems. Over time, the bad content is forgotten, and a handful of items survive and look prescient via selection bias.
voidhorse 3y ago
You do realize the "AI" we're talking about here is just LLMs, right? Language is a huge part of being human, so that's a good elements but: 1. There's no evidence that these LLMs have any sort of actual understanding of what they produce. 2. Being human is a lot more than just linguistic capability. We are embodied in a certain way, can interact with environments, have rich, multi-sensory experiences etc. tbh I think anyone that believes AGI can arise from an LLM is extremely naive and has basically no idea of the history of this track of research or how it works. To assert that LLMs will be capable of achieving consciousness is equivalent to asserting that human consciousness is reducible to statistical inference, which is palpably not true. (see. e.g. deductive logic). The only reason people even fall into the trap of thinking computers are remotely close to getting to AGI is because it became very popular to reason about the brain using analogies from computer science in the 50s, despite the fact that this was completely unjustified, and people glob on to the metaphors and then have the mypoia to actually think the human brain (which is exceedingly complex) is reducible to a freakin' von neuman architecture, lol. Biological systems and inorganic systems are fundamentally different.
- flangola7 3y ago
  I don't think anyone is talking about consciousness here. Consciousness is tangential to safety concerns.

z3c0 3y ago

Astroturfing, psyops, and disruption of general trust (commercially and maliciously, both domestic and foreign) It is disturbing to me how unconcerned everybody is with this over what is still only a hypothetical problem. States and businesses have been long employing subversive techniques to corral people towards their goals, and they all just got an alarmingly useful tool for automated propaganda. This is a problem right now, not hypothetically. All these people aching to be a Cassandra should rant and rave about that.

pixl97 3y ago
I mean it's not hypothetical, there are shit loads of bots on social media already causing problems. Now the bots these days have a human controller but we can see strife they cause in politics already.
- z3c0 3y ago
  We're in accordance.

hn_throwaway_99 3y ago

Definitely agree. I would summarize it a bit differently, but when people talk about AI dangers they are usually talking about 1 of 4 different things: 1. AI eventually takes control and destroys humans (i.e. the Skynet concern). 2. AI further ingrains already existing societal biases (sexism, racism, etc.) to the detriment of things like fair employment, fair judicial proceedings, etc. 3. AI makes large swaths of humanity unemployable, and we've never been able to design an economic system that can handle that. 4. AI supercharges already widely deployed psyops campaigns for disinformation, inciting division and violence, etc. The thing I find so aggravating is I see lots of media and self-professed AI experts focused on #1, I see lots of "Ethical AI" people solely focused on #2, but I see comparatively little focus on #3 and #4, which as you say are both happening right now. IMO #3 and #4 are far more likely to result in societal collapse than the first two issues.

usaar333 3y ago
#1 has high focus more due to impact than high probability. #2 doesn't seem talked about much at this point and seems to be pivoting more to #3. #2 never had much of a compelling argument given auditability. #3 gets mainly ignored due to Luddite assumptions driving it. I'm dubious myself over the short term - humans will have absolute advantage in many fields for a long time (especially with robotics lagging and being costly). #4 is risky, but humans can adapt. I see collapse as unlikely.
- pwdisswordfishc 3y ago
  > I'm dubious myself over the short term - humans will have absolute advantage in many fields for a long time AI doesn’t have to be better at the humans’ job to unemploy them. It’s enough that its output looks presentable enough for advertising most of the time, that it never asks for a day off, never gets sick or retires, never refuses orders, never joins a union… The capitalist doesn’t really care about having the best product to sell, they only care about having the lowest-cost product they can get away with selling.
- verall 3y ago
  > #2 never had much of a compelling argument given auditability. There are already AI systems which generate "scores" based on likelyhood to commit another crime (used in parole cases) and likelyhood to be a "good tenant" (used by landlords). Don't underestimate the difficulty to fix something when the people in charge of that thing do not want it fixed.

davidzweig 3y ago

>> There is no evidence that a runaway artificial intelligence is even possible. In the space of a century or so, the humans have managed to take rocks and sand and turn them into something that you can talk to with your voice, and it understands and responds fairly convincingly as it were a well-read human (glue together chatGPT with TTS/ASR). Doesn't seem like a big stretch to imagine that superhuman AI is just a few good ideas away, a decade or two perhaps.

Animats 3y ago

> Any laws should be legislating the downstream effects of AI, not the models themselves. That would require stronger consumer protections. So that's politically unacceptable in the US at the moment. We may well see it in the EU. The EU already regulates "automated decision making" as it affects EU citizens. This is part of the General Protection on Data Regulation. This paper discusses the application of those rules to AI systems.[1] Key points summary: - AI isn't special for regulation purposes. "First, the concept of Automated Decision Making includes algorithmic decision-making as well as AI-driven decision-making." - Guiding Principle 1: Law-compliant ADM. An operator that decides to use ADM for a particular purpose shall ensure that the design and the operation of the ADM are compliant with the laws applicable to an equivalent non- automated decision-making system. - Guiding Principle 2: ADM shall not be denied legal effect, validity or enforceability solely on the grounds that it is automated. - Guiding Principle 3: The operator has to assume the legal effects and bear the consequences of the ADM’s decision. ("Operator" here means the seller or offerer of the system, not the end user.) - Guiding Principle 4: It shall be disclosed that the decision is being made by automated means - Guiding Principle 5: Traceable decisions - Guiding Principle 6: The complexity, the opacity or the unpredictability of ADM is not a valid ground for rendering an unreasoned, unfounded or arbitrary decision. - Guiding Principle 7: The risks that the ADM may cause any harm or damage shall be allocated to the operator. - Guiding Principle 8: Automation shall not prevent, limit, or render unfeasible the exercise of rights and access to justice by affected persons. An alternative human-based route to exercise rights should be available. - Guiding Principle 9: The operator shall ensure reasonable and proportionate human oversight over the operation of ADM taking into consideration the risks involved and the rights and legitimate interests potentially affected by the decision. - Guiding Principle 10: Human review of significant decisions Human review of selected significant decisions on the grounds of the relevance of the legal effects, the irreversibility of their consequences, or the seriousness of the impact on rights and legitimate interests shall be made available by the operator. This is just a summary. The full text has examples, which include, without naming names, Google closing accounts and Uber firing drivers automatically. [1] https://europeanlawinstitute.eu/fileadmin/user_upload/p_eli/Publications/ELI_Innovation_Paper_on_Guiding_Principles_for_ADM_in_the_EU.pdf https://europeanlawinstitute.eu/fileadmin/user_upload/p_eli/...

gmerc 3y ago

Unaligned AI as an existential threat is an interesting topic but I feel we already know the answer to this one: It’s not like for the last few decades, we haven’t created an artificial, incentive based system at global scale that’s showing exactly how this will go. It’s not like a bunch of autonomous entities with a single prime directive, profit maximization are running our planet and are affecting how we live, how we structure of every day of our lives, control every aspect of our potential and destiny. Autonomous entities operating on reinforcement cycles driven by reward / punishment rules aligning about every human on this planet to their goals, right? It’s not like the enitities in this system are self improving towards a measurement and reward maximization and, as a result, command resources asserting normative (lobbying) power over the autonomy, self governance and control of people and their systems is governance. It’s not like we don’t know that this artificial system is unaligned with sustainability and survival of the human race, let alone happiness, freedom or love. We can watch its effects in real time accelerating towards our destruction under the yellow skies of New York, the burning steppes of Canada, the ashen hell or flood ridden plains of Australia, the thawing permafrost of Siberia, scorching climate affected cities of Southeast Asia, the annual haze from platantion burns in Indonesia, suffocating smog in Thailand, and the stripped bare husks of Latin American rainforests. And we know instinctively we are no longer in control, the system operating at larger than national scale, having long overpowered the systems of human governance, brute forcing everything and everyone on the planet into their control. But we pretend to otherwise, argue, pass measures doctoring symptoms, not mentioning the elephant in the room. But, one may protest, the vaunted C-Level control the entities, we say as Zuck and Co lament having to lay off humans, sobbing about responsibility to the prime directive. But politicians are, we pray as lobbyists, the human agents of our alien overlords bend them to their will. The alien entities we call corporations have no prime directive of human survival, sustainability, happiness and they already run everything. So one may be excused for having cynical views about the debate on whether unaligned AI is an existential, extinction level risk for us, whether humans could give creation to an unaligned system that could wipe them from the face of the planet. Our stories, narratives, the tales of the millennia apex predator of this planet have little room for the heresy of not being on top, in control. So deep goes out immersion in our own manifest destiny and in control identity, any challenge to the mere narrative is met with screetches and denigrations. In a throwback to the age of the heliocentricity debate - Galileo just broke decorum, spelling out what scientists knew for hundreds of years - the scientists and people devoted to understanding the technology are met with brandings of Doomsayer and heresy. Just as the earth being the center of the universe anchored our belief of being special, our intelligence, creativity or ability to draw hands is the pedestal these people have chosen to put their hands on with warnings of unaligned systemic entities. “It’s not human” is the last but feeble defense of the mind, failing to see the obvious. That the artificial system we created for a hundred years does not need people to be human, it just needs them to labor. It matters not that we can feel, love, express emotion and conjure dreams and hopes and offer human judgement for our jobs do not require it. Autonomy is not a feature of almost every human job, judgement replaced by corporate policies and rules. It matters not to the corporation that we need food to eat as it controls the resources to buy it, the creation of artificial labor is inevitably goal aligned with this system. Intelligence, let alone super intelligence is not a feature needed for most jobs or a system to take control over the entire planet. Our stories conjure super villains to make us believe we are in control, our movies no more than religious texts to the gospel of human exceptionalism. Show us the evidence they scream, as they did to Galileo, daring him to challenge the clear hand of god in all of creation. Us, unable to control a simple system we conjured into existence from rules and incentives operating on fallible meatsuits, having a chance to control a system of unparalleled processing power imbued with the combined statistical corpus of human knowledge, behavior, flaws and weaknesses? Laughable. Us, who saw social media codify the rules and incentives in digital systems powered by the precursor AI of today and watched the system helplessly a/b optimize towards maximum exploitation of human weaknesses for alignment with growth and profit containing the descent AI systems powered by orders of magnitude more capable systems or quantum computing? A snow flake may as well outlast hell. Us, a race with a 100% failure rate to find lasting governing structures optimizing for human potential not slipping in the face of an entity that only requires a single slip? An entity with perfect knowledge of the rules that bind us? Preposterous. Evidence indeed. “But we are human” will echo as the famous last words through the cosmos as our atoms are reconfigured into bitcoin storage to hold the profits of unbounded growth. What remains will not be human anymore, an timeless reckoning to the power of rules, incentives and consumption, eating world after world to satisfy the prime directive. But we always have hope. As our tales tell us, it dies last and it’s the most remote property for an AI to achieve. It may master confidence, assertiveness and misdirection, but hope? That may be the last human refuge for the coming storm.