4 ms·

I’m intuitively not all that worried about LLMs in particular. I am worried about the commercial imperative to produce AI (by whatever path we may eventually fi

I’m intuitively not all that worried about LLMs in particular. I am worried about the commercial imperative to produce AI (by whatever path we may eventually find it) being at odds with the technical and sociopolitical ability to steer that AI toward good outcomes (whatever that may mean). I have no clue whether LLMs will get us there (neither do you) and I have no clue whether we’re 6 months away or 600 years away (neither do you). What LLMs have revealed is that very reasonable checkpoints will be blown through for commercial purposes. We are reliably producing unexpected emergent capabilities with each new scale up. We are reliably failing to get these systems to behave as their creators intend. We are continually deploying these systems to the public anyway, and we are continually connecting them to other, increasingly vital, systems. If your argument is “we’ll never reach a superhuman or self-improving AI,” I’m open to hearing that argument. Tell me how you know. If nothing precludes us from building it, then I know that we will eventually build it due to the commercial incentives. If your argument is “we’ll get there, but don’t worry because we’ll stop minutes before deployment and figure out all the alignment/emergent behaviors/social distribution/airgapping problems before pressing ‘deploy’,” then I ask what is your evidence? What capability have we shown to make such a decision at such a moment?

zer00eyz 3y ago
> Tell me how you know. Congratulations, you're performing one of the major functions of intellect that an LLM can't. Learning. You aren't a deterministic system. LLM's aren't a model that is suddenly going to change all the past failures of "make it bigger and AGI will emerge from it"... At its core what an LLM is produces something interesting, but it's NOT a path to intellect... It's more or less turning language into a mathematical model. Are we so foolish, so hubris driven that something bigger will emerge from the digitized version of our language, and our vision alone, we made god in our image and how did THAT work out? And if a super intelligent AI emerges tomorrow so what? Is it going to be able to manipulate us all based on a deep understand of people, that it gained from the piles of bad psych research? Is it going to melt down reactors and launch nukes, cause we live in a brittle world and that is suicide for your AGI (and improbable to impossible). Is it going to kill us with paperclips... cause we're so stupid we keep feeding it metal till we drown ourselves. If AGI shows up tomorrow how many decades to a tipping point where it could "take over" where it could "cause harm"? Walk me through that process where the world is automated enough that paperclips are what kills us. That AGI is gonna be stuck in a cradle, a data center for a LONG time. It's gonna need to be fed power, hardware, water (for cooling). A can of gas and a bag of rags will stop, A few people with shovels or one person with a backhoe cuts it off from the world. It takes 50 to 100 years for that AGI to get to a place where it does not need us at all. I would be more worried about a doomer mudering it in its cradle than it going off the rails in that time frame.
- ethanbond 3y ago
  Okay, let's say it takes 100 years. Hey let's say it takes 200 years for AGI to get to a place where it doesn't need us at all. What then?
  - kridsdale3 3y ago
    Nuke it. It exists in a physical (if distributed) location and requires inputs and outputs in electric cables and fibre cables. Destroy the brain.
    - staunton 3y ago
      > It exists in a physical (if distributed) location and requires inputs and outputs in electric cables and fibre cables "Nuke all of civilization right now!"

zer00eyz 3y ago

> Tell me how you know. Congratulations, you're performing one of the major functions of intellect that an LLM can't. Learning. You aren't a deterministic system. LLM's aren't a model that is suddenly going to change all the past failures of "make it bigger and AGI will emerge from it"... At its core what an LLM is produces something interesting, but it's NOT a path to intellect... It's more or less turning language into a mathematical model. Are we so foolish, so hubris driven that something bigger will emerge from the digitized version of our language, and our vision alone, we made god in our image and how did THAT work out? And if a super intelligent AI emerges tomorrow so what? Is it going to be able to manipulate us all based on a deep understand of people, that it gained from the piles of bad psych research? Is it going to melt down reactors and launch nukes, cause we live in a brittle world and that is suicide for your AGI (and improbable to impossible). Is it going to kill us with paperclips... cause we're so stupid we keep feeding it metal till we drown ourselves. If AGI shows up tomorrow how many decades to a tipping point where it could "take over" where it could "cause harm"? Walk me through that process where the world is automated enough that paperclips are what kills us. That AGI is gonna be stuck in a cradle, a data center for a LONG time. It's gonna need to be fed power, hardware, water (for cooling). A can of gas and a bag of rags will stop, A few people with shovels or one person with a backhoe cuts it off from the world. It takes 50 to 100 years for that AGI to get to a place where it does not need us at all. I would be more worried about a doomer mudering it in its cradle than it going off the rails in that time frame.

ethanbond 3y ago
Okay, let's say it takes 100 years. Hey let's say it takes 200 years for AGI to get to a place where it doesn't need us at all. What then?
- kridsdale3 3y ago
  Nuke it. It exists in a physical (if distributed) location and requires inputs and outputs in electric cables and fibre cables. Destroy the brain.
  - staunton 3y ago
    > It exists in a physical (if distributed) location and requires inputs and outputs in electric cables and fibre cables "Nuke all of civilization right now!"