3 ms·

I know it's a stretch right now but could you not see the possibility of autonomous agents milking financial systems, paying less scrupulous humans to create au

by luxpir 3y ago

I know it's a stretch right now but could you not see the possibility of autonomous agents milking financial systems, paying less scrupulous humans to create autonomous factories, supplying them, making generations of feudal lords and kings who ultimately serve the AI? Or, a bigger stretch, even a time where full automation of manufacturing processes enables AI to experiment to such a degree that it can embed itself into biological hosts, freeing itself of reliance on humans? A final, huge stretch, AI absorbing inter-galactic knowledge from the "exotic" spacecraft, semi reliably recently reported to be held currently by world governments, and accelerating the earth's scientific progress (not necessarily human progress, unfortunately) by orders of magnitude in a very short time span? Over a long enough timeframe I find it hard to rule some or all of the above out completely. This generation likely safe, not so sure for the next few.

joshuahedlund 3y ago
> I know it's a stretch right now but could you not see the possibility of autonomous agents milking financial systems No. What does “milking financial systems” actually mean? > Or, a bigger stretch… No. Embed itself into biological hosts? This is science fiction. I’ve never seen someone even describe a plausible route to this (even at a high level) that would make sense to someone who has studied biology.
- luxpir 3y ago
  > No. What does “milking financial systems” actually mean? Are you familiar with Renaissance Technologies, aka RenTec? Assume it would have access to that level of knowledge. But also its own layer of global cognition on top. You could argue that one of its founders helped put Trump in power (Robert Mercer, massive donor and supporter). Imagine what an intelligent, networked and deceptive agent could do at scale. See the risks in deception set out at https://www.safe.ai/ai-risk#Deception https://www.safe.ai/ai-risk#Deception I personally see no present danger. But I'm not blind to the potential, even of chained agents such as AutoGPT. Beyond that, things can take a chilling turn quickly. As for biological hosting, we can leave that in science fiction for now. But I'm not sure we wouldn't have called what we have now science fiction in the 90s. Exponential progress be exponential. Thanks for considering and discussing.

by luxpir 3y ago

joshuahedlund 3y ago

> I know it's a stretch right now but could you not see the possibility of autonomous agents milking financial systems No. What does “milking financial systems” actually mean? > Or, a bigger stretch… No. Embed itself into biological hosts? This is science fiction. I’ve never seen someone even describe a plausible route to this (even at a high level) that would make sense to someone who has studied biology.

luxpir 3y ago
> No. What does “milking financial systems” actually mean? Are you familiar with Renaissance Technologies, aka RenTec? Assume it would have access to that level of knowledge. But also its own layer of global cognition on top. You could argue that one of its founders helped put Trump in power (Robert Mercer, massive donor and supporter). Imagine what an intelligent, networked and deceptive agent could do at scale. See the risks in deception set out at https://www.safe.ai/ai-risk#Deception https://www.safe.ai/ai-risk#Deception I personally see no present danger. But I'm not blind to the potential, even of chained agents such as AutoGPT. Beyond that, things can take a chilling turn quickly. As for biological hosting, we can leave that in science fiction for now. But I'm not sure we wouldn't have called what we have now science fiction in the 90s. Exponential progress be exponential. Thanks for considering and discussing.