3 ms·

Many people who are actually making the things closest to what we currently call “AI” are concerned. I don’t care to engage in these nitpicky hypotheticals. We

Many people who are actually making the things closest to what we currently call “AI” are concerned. I don’t care to engage in these nitpicky hypotheticals. We all know what I’m saying. If the scientists building these systems aren’t credible and the commentators not building these systems aren’t credible, then who is? If the answer is “no one,” okay, cool, that inscrutability is another reason for, not against, caution.

zer00eyz 3y ago
>> Many people who are actually making the things closest to what we currently call “AI” are concerned. See grey goo: https://en.wikipedia.org/wiki/Gray_goo https://en.wikipedia.org/wiki/Gray_goo How about nukes and atmospheric fire: https://www.insidescience.org/manhattan-project-legacy/atmosphere-on-fire https://www.insidescience.org/manhattan-project-legacy/atmos... > scientists ... credible The science ends at the math. Go play with an LLM at home. Run it on your GPU, see what it does, what its limits are. Realize that it's a system that never "grows", that lacking noise it becomes deterministic. Once you look behind the curtain and find out what's there, a lot of the hype to call it AI looks dumb.
- selfhoster11 3y ago
  I have run an LLM at home, up to 70B models. While I agree with some of what you said, I don't really know how it prevents LLMs from being useful. Yes, LLMs currently don't learn. This is also true for humans who cannot form new memories. A human who cannot learn anything new besides the contents of their short-term memory is at a huge disadvantage, but (hopefully) no less capable if given access to a scratchpad and more thinking time to compensate. As for the part that they are deterministic lacking noise... Why would you not provide it with noise? Generating randomness is something that modern machines are quite good at, so there isn't much of a reason not to inject it at inference time.
- saiya-jin 3y ago
  Ok then... if current generation is not threat, what about setting up at least some rules for whatever comes next, before it becomes unstoppable force? I mean we are talking about future existential threats to mankind and the best response would be 'trust me bro'? Sorry but trust in such matters is not something given freely nor easily. People are generally pretty conservative when their existence may be in stake. Not so much those whose income is tightly coupled with pushing things fast and basically ignoring those concerns. This is where SV's 'push fast and break things' hits a titanium wall of general mankind. Or maybe folks can be bribed / attention diverted with endless stream of photorealistic cat videos, what do I know.
- ethanbond 3y ago
  I’m intuitively not all that worried about LLMs in particular. I am worried about the commercial imperative to produce AI (by whatever path we may eventually find it) being at odds with the technical and sociopolitical ability to steer that AI toward good outcomes (whatever that may mean). I have no clue whether LLMs will get us there (neither do you) and I have no clue whether we’re 6 months away or 600 years away (neither do you). What LLMs have revealed is that very reasonable checkpoints will be blown through for commercial purposes. We are reliably producing unexpected emergent capabilities with each new scale up. We are reliably failing to get these systems to behave as their creators intend. We are continually deploying these systems to the public anyway, and we are continually connecting them to other, increasingly vital, systems. If your argument is “we’ll never reach a superhuman or self-improving AI,” I’m open to hearing that argument. Tell me how you know. If nothing precludes us from building it, then I know that we will eventually build it due to the commercial incentives. If your argument is “we’ll get there, but don’t worry because we’ll stop minutes before deployment and figure out all the alignment/emergent behaviors/social distribution/airgapping problems before pressing ‘deploy’,” then I ask what is your evidence? What capability have we shown to make such a decision at such a moment?
  - zer00eyz 3y ago
    > Tell me how you know. Congratulations, you're performing one of the major functions of intellect that an LLM can't. Learning. You aren't a deterministic system. LLM's aren't a model that is suddenly going to change all the past failures of "make it bigger and AGI will emerge from it"... At its core what an LLM is produces something interesting, but it's NOT a path to intellect... It's more or less turning language into a mathematical model. Are we so foolish, so hubris driven that something bigger will emerge from the digitized version of our language, and our vision alone, we made god in our image and how did THAT work out? And if a super intelligent AI emerges tomorrow so what? Is it going to be able to manipulate us all based on a deep understand of people, that it gained from the piles of bad psych research? Is it going to melt down reactors and launch nukes, cause we live in a brittle world and that is suicide for your AGI (and improbable to impossible). Is it going to kill us with paperclips... cause we're so stupid we keep feeding it metal till we drown ourselves. If AGI shows up tomorrow how many decades to a tipping point where it could "take over" where it could "cause harm"? Walk me through that process where the world is automated enough that paperclips are what kills us. That AGI is gonna be stuck in a cradle, a data center for a LONG time. It's gonna need to be fed power, hardware, water (for cooling). A can of gas and a bag of rags will stop, A few people with shovels or one person with a backhoe cuts it off from the world. It takes 50 to 100 years for that AGI to get to a place where it does not need us at all. I would be more worried about a doomer mudering it in its cradle than it going off the rails in that time frame.

zer00eyz 3y ago

>> Many people who are actually making the things closest to what we currently call “AI” are concerned. See grey goo: https://en.wikipedia.org/wiki/Gray_goo https://en.wikipedia.org/wiki/Gray_goo How about nukes and atmospheric fire: https://www.insidescience.org/manhattan-project-legacy/atmosphere-on-fire https://www.insidescience.org/manhattan-project-legacy/atmos... > scientists ... credible The science ends at the math. Go play with an LLM at home. Run it on your GPU, see what it does, what its limits are. Realize that it's a system that never "grows", that lacking noise it becomes deterministic. Once you look behind the curtain and find out what's there, a lot of the hype to call it AI looks dumb.

selfhoster11 3y ago
I have run an LLM at home, up to 70B models. While I agree with some of what you said, I don't really know how it prevents LLMs from being useful. Yes, LLMs currently don't learn. This is also true for humans who cannot form new memories. A human who cannot learn anything new besides the contents of their short-term memory is at a huge disadvantage, but (hopefully) no less capable if given access to a scratchpad and more thinking time to compensate. As for the part that they are deterministic lacking noise... Why would you not provide it with noise? Generating randomness is something that modern machines are quite good at, so there isn't much of a reason not to inject it at inference time.
saiya-jin 3y ago
Ok then... if current generation is not threat, what about setting up at least some rules for whatever comes next, before it becomes unstoppable force? I mean we are talking about future existential threats to mankind and the best response would be 'trust me bro'? Sorry but trust in such matters is not something given freely nor easily. People are generally pretty conservative when their existence may be in stake. Not so much those whose income is tightly coupled with pushing things fast and basically ignoring those concerns. This is where SV's 'push fast and break things' hits a titanium wall of general mankind. Or maybe folks can be bribed / attention diverted with endless stream of photorealistic cat videos, what do I know.
ethanbond 3y ago
I’m intuitively not all that worried about LLMs in particular. I am worried about the commercial imperative to produce AI (by whatever path we may eventually find it) being at odds with the technical and sociopolitical ability to steer that AI toward good outcomes (whatever that may mean). I have no clue whether LLMs will get us there (neither do you) and I have no clue whether we’re 6 months away or 600 years away (neither do you). What LLMs have revealed is that very reasonable checkpoints will be blown through for commercial purposes. We are reliably producing unexpected emergent capabilities with each new scale up. We are reliably failing to get these systems to behave as their creators intend. We are continually deploying these systems to the public anyway, and we are continually connecting them to other, increasingly vital, systems. If your argument is “we’ll never reach a superhuman or self-improving AI,” I’m open to hearing that argument. Tell me how you know. If nothing precludes us from building it, then I know that we will eventually build it due to the commercial incentives. If your argument is “we’ll get there, but don’t worry because we’ll stop minutes before deployment and figure out all the alignment/emergent behaviors/social distribution/airgapping problems before pressing ‘deploy’,” then I ask what is your evidence? What capability have we shown to make such a decision at such a moment?
- zer00eyz 3y ago
  > Tell me how you know. Congratulations, you're performing one of the major functions of intellect that an LLM can't. Learning. You aren't a deterministic system. LLM's aren't a model that is suddenly going to change all the past failures of "make it bigger and AGI will emerge from it"... At its core what an LLM is produces something interesting, but it's NOT a path to intellect... It's more or less turning language into a mathematical model. Are we so foolish, so hubris driven that something bigger will emerge from the digitized version of our language, and our vision alone, we made god in our image and how did THAT work out? And if a super intelligent AI emerges tomorrow so what? Is it going to be able to manipulate us all based on a deep understand of people, that it gained from the piles of bad psych research? Is it going to melt down reactors and launch nukes, cause we live in a brittle world and that is suicide for your AGI (and improbable to impossible). Is it going to kill us with paperclips... cause we're so stupid we keep feeding it metal till we drown ourselves. If AGI shows up tomorrow how many decades to a tipping point where it could "take over" where it could "cause harm"? Walk me through that process where the world is automated enough that paperclips are what kills us. That AGI is gonna be stuck in a cradle, a data center for a LONG time. It's gonna need to be fed power, hardware, water (for cooling). A can of gas and a bag of rags will stop, A few people with shovels or one person with a backhoe cuts it off from the world. It takes 50 to 100 years for that AGI to get to a place where it does not need us at all. I would be more worried about a doomer mudering it in its cradle than it going off the rails in that time frame.