4 ms·

People who worry about AI taking over the world need to worry about the world continuing to work. Your average gas station gets gas 2x a day. The electric grid

People who worry about AI taking over the world need to worry about the world continuing to work. Your average gas station gets gas 2x a day. The electric grid has 100's of faults that are worked around and fixed every day. Delivery still are loaded and driven by people. Any AGI that is a threat to humanity has to be suicidal to act on it because of how the world works. Furthermore, if you crack open an LLM and turn OFF the random noise (temperature) it becomes very deterministic. You think our accent to godhood is on the back of a vector map with some noise, I dont know what to tell you. That doesn't mean all this ML can't change the world, can't foster less BS jobs and enable creativity... but I have massive doubts that sentience and sapience are on this path.

ethanbond 3y ago
Sure hope that intelligence doesn’t roughly mean “ability to find solutions to problems that less intelligent entities can’t discover.” I have no idea what “our ascent to godhood” entails, and it’s hilarious that doomers get criticized for being religious about all this and then you’ll say something like that with a straight face. That’s neat that you “have doubts” sentience and sapience are on this path. You also don’t actually know.
staunton 3y ago
> Any AGI that is a threat to humanity has to be suicidal to act on it because of how the world works A "threat to humanity" need not mean "killing all humans next year". It could just mean some entity gaining permanent control. That situation would be irreversible and humanity would be at the mercy of that entity. Chances are, the entity would have emerged victorious from an evolutionary struggle for power and thus would not care about human flourishing. That entity need not be a single AI. It could be a system like a large corporation or a country where initially a lot of "small" decisions are made by AIs. Over time, the influence of humans on this system might diminish and the trend might become irreversible. Currently, no individual and no group, (organization, country...) on the planet has anywhere close to complete control over humanity. Further, even if some country managed to conquer everything, it could not hope to maintain that power indefinitely. An immortal system capable of complete surveillance, however, may be able to maintain power. It's a new thing, we don't know. "Sentience" doesn't matter one bit for any of this. Such a system might take centuries to form or it might go quickly. Humans might also go extinct before something like this comes about. However, that doesn't mean people who think about such possibilities are stupid.
- greyface- 3y ago
  > That entity need not be a single AI. It could be a system like a large corporation This is not a new problem, then. Let's tackle the issue of corporations, rather than chase an AI boogeyman that doesn't fundamentally change anything. Look at oil companies, for example. They have humans in the loop at every level, and yet those humans do little to prevent profit incentives from leading them to destroy the planet. A broken reward function is a broken reward function, AI-assisted or not.
  - staunton 3y ago
    An organization's policies are still implemented and maintained by humans. No matter how powerful, the power is transient. You have churn, corruption, incomplete knowledge transfer, etc. AI systems in effective leadership could be able to maintain goals and accumulate power. That's what's new.
- kridsdale3 3y ago
  I argue this already happened. Facebook Inc (therefore Zuck personally) in 2015 effectively wielded total control of the world. Adjusting weights in ML systems there could determine who would be President of the United States. Only when that power became clear to the common man, and to Congress, did any effort to rein it in take place. That fear is why their Stablecoin effort was completely destroyed by the US Government. Too much power concentration. It's why Jack Ma was pulled away to some black site and presumably beaten with reeds for a few months.
  - staunton 3y ago
    Effective and manifested power cannot be reined in. A power that is reined in is only as powerful as its reined in form. Most likely, there wasn't any power to start with and it's just a stupid conspiracy theory... You can imagine yourself or your tribe or your political party to have absolute power. You can imagine Big Brother, or the Illuminati, or the Teletubbies to have power... It's not True! Today, nobody has absolute power! Let's hope we can either keep it that way or share power amongst ourselves.

ethanbond 3y ago

Sure hope that intelligence doesn’t roughly mean “ability to find solutions to problems that less intelligent entities can’t discover.” I have no idea what “our ascent to godhood” entails, and it’s hilarious that doomers get criticized for being religious about all this and then you’ll say something like that with a straight face. That’s neat that you “have doubts” sentience and sapience are on this path. You also don’t actually know.

staunton 3y ago

> Any AGI that is a threat to humanity has to be suicidal to act on it because of how the world works A "threat to humanity" need not mean "killing all humans next year". It could just mean some entity gaining permanent control. That situation would be irreversible and humanity would be at the mercy of that entity. Chances are, the entity would have emerged victorious from an evolutionary struggle for power and thus would not care about human flourishing. That entity need not be a single AI. It could be a system like a large corporation or a country where initially a lot of "small" decisions are made by AIs. Over time, the influence of humans on this system might diminish and the trend might become irreversible. Currently, no individual and no group, (organization, country...) on the planet has anywhere close to complete control over humanity. Further, even if some country managed to conquer everything, it could not hope to maintain that power indefinitely. An immortal system capable of complete surveillance, however, may be able to maintain power. It's a new thing, we don't know. "Sentience" doesn't matter one bit for any of this. Such a system might take centuries to form or it might go quickly. Humans might also go extinct before something like this comes about. However, that doesn't mean people who think about such possibilities are stupid.

greyface- 3y ago
> That entity need not be a single AI. It could be a system like a large corporation This is not a new problem, then. Let's tackle the issue of corporations, rather than chase an AI boogeyman that doesn't fundamentally change anything. Look at oil companies, for example. They have humans in the loop at every level, and yet those humans do little to prevent profit incentives from leading them to destroy the planet. A broken reward function is a broken reward function, AI-assisted or not.
- staunton 3y ago
  An organization's policies are still implemented and maintained by humans. No matter how powerful, the power is transient. You have churn, corruption, incomplete knowledge transfer, etc. AI systems in effective leadership could be able to maintain goals and accumulate power. That's what's new.
kridsdale3 3y ago
I argue this already happened. Facebook Inc (therefore Zuck personally) in 2015 effectively wielded total control of the world. Adjusting weights in ML systems there could determine who would be President of the United States. Only when that power became clear to the common man, and to Congress, did any effort to rein it in take place. That fear is why their Stablecoin effort was completely destroyed by the US Government. Too much power concentration. It's why Jack Ma was pulled away to some black site and presumably beaten with reeds for a few months.
- staunton 3y ago
  Effective and manifested power cannot be reined in. A power that is reined in is only as powerful as its reined in form. Most likely, there wasn't any power to start with and it's just a stupid conspiracy theory... You can imagine yourself or your tribe or your political party to have absolute power. You can imagine Big Brother, or the Illuminati, or the Teletubbies to have power... It's not True! Today, nobody has absolute power! Let's hope we can either keep it that way or share power amongst ourselves.