6 ms·

re your last sentence: why do you disapprove of funding research into AI safety

michaelmrose 4y ago
I'm not the poster but there is no reason to believe that we possess the tools to contemplate AI safety worse it may be a fools errand to contemplate how to control something much smarter than you are. Ergo at this point in time money spent to pay people to contemplate it is almost certainly entirely wasted.
- TedDoesntTalk 4y ago
  It’s much like spending millions on studying how to save humanity from the sun when it expands in 5b years.
  - c1ccccc1 4y ago
    Interestingly, the sun would kill us much sooner than that: in less than 1 billion years the sun's increased output will heat up the Earth's surface to the point of becoming unlivable. Related to the topics of AI apocalypse and putting things off because they seem far away: I personally find it amusing to contemplate a strong AGI being developed in 2035 and wiping out humanity. Why? Because it would eliminate the Y2038 problem, retroactively vindicating all the engineers who only allocated 32 bits to timestamps. It would mean the justification of "oh, it's so far off, the world will have changed unimaginably by then", would for once turn out to be correct!
    - notahacker 4y ago
      I quite like the idea of strong AI being developed in 2038 just in time for its plans to be thwarted by time disorientation...
- FeepingCreature 4y ago
  Maybe we should try to acquire the tools before we build systems much smarter than we are
  - michaelmrose 4y ago
    Current AI seems to be just a box of math churning out correlations that we can put to effective use but not understand wherein the ability to actually act is entirely reliant on us deliberately wiring the machine into the world in a way it can even have negative effects or any effect at all wherein said negative effects are entirely and trivially predictable and avoidable. This is true whether its a robot that accidentally breaks a kids finger or a system that bakes in the inherent racism of prior decisions into future sentencing or mortgage approvals. In most cases the logical thing is just don't use AI for that. Its not at all clear that future actual AI will be GPT7 now with 10,000x as much processing power. In fact not much seems to be clear at all. It would seem that the insight to control future tools will come from the experience in building said tools with little chance that it will just accidentally suddenly become skynet. If we doom ourselves thus it will be a long laborious process with years of striving, setbacks, and thousands of people involved. We should focus on the basic science and let it become clear what avenues exist over the probable decades it will take to even reach the vicinity of our goal. Money spent specifically on paying PhDs to imagine how to secure a technology we don't have and don't remotely understand is probably wasted.
    - FeepingCreature 4y ago
      None of this seems to argue against focusing on safety tools now, given the enormous downside risk. And maybe you're wrong about the probable decades. The possible outcomes here are "probably we waste a lot of money" and "the world ends."
raverbashing 4y ago
At this point I think that giving $5 to a homeless person in SF (even if they go straight to doing another shot) might be more productive than this self-aggrandizing vapid BS about some theoretical risk that might come in the future Right now what's threatening the most people is war, famine and diseases. Climate change is a tangible future threat but I'd bet a couple $ that the JSO organizers are full EA proponents and are flying private jets to convince people to throw soup into works of art
- concordDance 4y ago
  > BS about some theoretical risk that might come in the future Like increased rate of hurricanes due to climate change? One does need to seriously consider low probability, high impact events, like pandemics, in one's future planning and resource allocation.
  - raverbashing 4y ago
    > why do you disapprove of funding research into AI safety This is what I'm replying to. And yes Climate Change is a worrying problem, but it won't be solved by people throwing tantrums in museums
  - AlexandrB 4y ago
    > Like increased rate of hurricanes due to climate change? This is a bad analogy. The mechanisms and risks of climate change are reasonably well understood as are the solutions available to us now and even some possible future solutions. When it comes to AGI, what's the risk? Something bad might happen? The mechanisms of AGI arising is also unknown since it has never happened before and it's not even known if it's possible. Finally, what's the solution? Prevent AGI research? Make sure every AGI project has a killswitch? How do you prevent something when you don't know if it is possible nor how it will arise? Compare this to the concrete, measurable things we can do to slow down climate change and "AGI safety" seems like a make-work money pit.
froggychairs 4y ago
Not OP, but I do work in reducing harms of deployed AI systems at a big tech company When most EA/Rationalist folks discuss AI Safety. They’re talking about AGI, a hypothetical construct that we have no way of proving is even possible Now there’s nothing inherently wrong with funding research of that sort. The problem comes is that many EA/Rationalist folks begin downplaying legitimate tangible risks on the horizon (I’ve had prominent EA folks tell me that Climate Change is not an existential risk) or that many of these AI Safety researchers actively ignore how to reduce harms of the AI we have deployed today. This existential measuring contest is just so odd and unnecessary. I really don’t care if you think AI Safety is a legitimate problem. I personally don’t but it’s when they begin to downplay others actual catastrophes that many of us take issue.
- concordDance 4y ago
  > I’ve had prominent EA folks tell me that Climate Change is not an existential risk Have you seen anything to indicate otherwise? The IPCC reports don't contain even a whiff in that direction. To the best of my knowledge zero people who know anything about climate change think its existential.
  - andybak 4y ago
    Is there an accepted unambiguous definition of "existential" here? Is it: 1. All life on the planet dies 2. All advanced life on the planet dies? 3. All humans die? 4. Advanced civilization is destroyed irretrievably? 5. Advanced civilization is destroyed for a long period of time? etc And I'm not sure the group "people who know anything about climate change" are neccesarily better equipped to answer this question - it's a complex systems question. I'm not actually sure who is equipped to answer it accurately.
    - a3_nm 4y ago
      It looks like there are research works trying to define precisely this notion of existential risk in the context of climate change, see e.g. https://link.springer.com/article/10.1007/s10584-022-03430-y https://link.springer.com/article/10.1007/s10584-022-03430-y

I'm not the poster but there is no reason to believe that we possess the tools to contemplate AI safety worse it may be a fools errand to contemplate how to control something much smarter than you are. Ergo at this point in time money spent to pay people to contemplate it is almost certainly entirely wasted.

TedDoesntTalk 4y ago
It’s much like spending millions on studying how to save humanity from the sun when it expands in 5b years.
- c1ccccc1 4y ago
  Interestingly, the sun would kill us much sooner than that: in less than 1 billion years the sun's increased output will heat up the Earth's surface to the point of becoming unlivable. Related to the topics of AI apocalypse and putting things off because they seem far away: I personally find it amusing to contemplate a strong AGI being developed in 2035 and wiping out humanity. Why? Because it would eliminate the Y2038 problem, retroactively vindicating all the engineers who only allocated 32 bits to timestamps. It would mean the justification of "oh, it's so far off, the world will have changed unimaginably by then", would for once turn out to be correct!
  - notahacker 4y ago
    I quite like the idea of strong AI being developed in 2038 just in time for its plans to be thwarted by time disorientation...
FeepingCreature 4y ago
Maybe we should try to acquire the tools before we build systems much smarter than we are
- michaelmrose 4y ago
  Current AI seems to be just a box of math churning out correlations that we can put to effective use but not understand wherein the ability to actually act is entirely reliant on us deliberately wiring the machine into the world in a way it can even have negative effects or any effect at all wherein said negative effects are entirely and trivially predictable and avoidable. This is true whether its a robot that accidentally breaks a kids finger or a system that bakes in the inherent racism of prior decisions into future sentencing or mortgage approvals. In most cases the logical thing is just don't use AI for that. Its not at all clear that future actual AI will be GPT7 now with 10,000x as much processing power. In fact not much seems to be clear at all. It would seem that the insight to control future tools will come from the experience in building said tools with little chance that it will just accidentally suddenly become skynet. If we doom ourselves thus it will be a long laborious process with years of striving, setbacks, and thousands of people involved. We should focus on the basic science and let it become clear what avenues exist over the probable decades it will take to even reach the vicinity of our goal. Money spent specifically on paying PhDs to imagine how to secure a technology we don't have and don't remotely understand is probably wasted.
  - FeepingCreature 4y ago
    None of this seems to argue against focusing on safety tools now, given the enormous downside risk. And maybe you're wrong about the probable decades. The possible outcomes here are "probably we waste a lot of money" and "the world ends."

raverbashing 4y ago

At this point I think that giving $5 to a homeless person in SF (even if they go straight to doing another shot) might be more productive than this self-aggrandizing vapid BS about some theoretical risk that might come in the future Right now what's threatening the most people is war, famine and diseases. Climate change is a tangible future threat but I'd bet a couple $ that the JSO organizers are full EA proponents and are flying private jets to convince people to throw soup into works of art

concordDance 4y ago
> BS about some theoretical risk that might come in the future Like increased rate of hurricanes due to climate change? One does need to seriously consider low probability, high impact events, like pandemics, in one's future planning and resource allocation.
- raverbashing 4y ago
  > why do you disapprove of funding research into AI safety This is what I'm replying to. And yes Climate Change is a worrying problem, but it won't be solved by people throwing tantrums in museums
- AlexandrB 4y ago
  > Like increased rate of hurricanes due to climate change? This is a bad analogy. The mechanisms and risks of climate change are reasonably well understood as are the solutions available to us now and even some possible future solutions. When it comes to AGI, what's the risk? Something bad might happen? The mechanisms of AGI arising is also unknown since it has never happened before and it's not even known if it's possible. Finally, what's the solution? Prevent AGI research? Make sure every AGI project has a killswitch? How do you prevent something when you don't know if it is possible nor how it will arise? Compare this to the concrete, measurable things we can do to slow down climate change and "AGI safety" seems like a make-work money pit.

froggychairs 4y ago

Not OP, but I do work in reducing harms of deployed AI systems at a big tech company When most EA/Rationalist folks discuss AI Safety. They’re talking about AGI, a hypothetical construct that we have no way of proving is even possible Now there’s nothing inherently wrong with funding research of that sort. The problem comes is that many EA/Rationalist folks begin downplaying legitimate tangible risks on the horizon (I’ve had prominent EA folks tell me that Climate Change is not an existential risk) or that many of these AI Safety researchers actively ignore how to reduce harms of the AI we have deployed today. This existential measuring contest is just so odd and unnecessary. I really don’t care if you think AI Safety is a legitimate problem. I personally don’t but it’s when they begin to downplay others actual catastrophes that many of us take issue.

concordDance 4y ago
> I’ve had prominent EA folks tell me that Climate Change is not an existential risk Have you seen anything to indicate otherwise? The IPCC reports don't contain even a whiff in that direction. To the best of my knowledge zero people who know anything about climate change think its existential.
- andybak 4y ago
  Is there an accepted unambiguous definition of "existential" here? Is it: 1. All life on the planet dies 2. All advanced life on the planet dies? 3. All humans die? 4. Advanced civilization is destroyed irretrievably? 5. Advanced civilization is destroyed for a long period of time? etc And I'm not sure the group "people who know anything about climate change" are neccesarily better equipped to answer this question - it's a complex systems question. I'm not actually sure who is equipped to answer it accurately.
  - a3_nm 4y ago
    It looks like there are research works trying to define precisely this notion of existential risk in the context of climate change, see e.g. https://link.springer.com/article/10.1007/s10584-022-03430-y https://link.springer.com/article/10.1007/s10584-022-03430-y