10 ms·

I don’t see any reason this highly chaotic system is most likely to settle in a spot that we happen to enjoy/see as a compromise of some sort. The point is tha

by bbor 1mo ago

I don’t see any reason this highly chaotic system is most likely to settle in a spot that we happen to enjoy/see as a compromise of some sort. The point is that we now have superintelligence; either it helps us conquer death, or it poses a massive —say, existential- risk to us all. It’s hard to imagine such a powerful tool resulting in just another century of slightly-better capitalist nationalism. I guess there could be a second AI winter around the corner that we haven’t seen coming yet. That would be nice! Not likely IMHO, but nice.

chr15m 1mo ago
> we now have superintelligence Wait what?
- cdelsolar 1mo ago
  There’s a magic box that can literally build you anything you want that you can control from your phone while playing bedtime songs for your kid
  - irishcoffee 1mo ago
    Anything I want? Like a car? Or a football stadium? Your prompt skills up to snuff? I crack myself up.
    - johnsmith1840 1mo ago
      Once robots mature, yes. People will literally prompt buildings into existance. I think that's more than 10yrs from now but if coding AGI or close enough to it becomes real robotics will also accelerate.
      - weatherlite 1mo ago
        Well you still need land and a bunch of permits, access to materials , trucks, equipment etc. But yeah I guess big construction companies will start to automate in the next decades. It's more interesting to me to build an entire subarb/city in a few months instead of yeras to deal with the rising housing costs. Looks like camp deflation was right but we'll see...
        jbstack 1mo ago
        > Permits Obtained by your AI agent, granted by another AI agent. > Materials, trucks, equipment etc. Ordered by your AI agent, delivered by a self-driving truck or drone, loaded/unloaded by robots to/from the warehouse/site. Most of this stuff can already be done today in some form, so its just about improving the autonomy and integrations along the supply chain.
    - jbstack 1mo ago
      Many factories in China now already have 90% of their work being done by robots. Is it really so hard to imagine that we reach a point where they can be prompted?
    - bawana 1mo ago
      why isnt there an ai front end for 3d printers?
      - BirAdam 1mo ago
        Give it time.
  - jdkdidjmdns 1mo ago
    you must have very little imagination
  - chr15m 1mo ago
    It's absolutely incredible!
imglorp 1mo ago
I'm still not seeing the route to that existential risk, not yet anyway. Far more imminent is the very real risk of unbounded greed and power that wields those AIs, in full swing as we watch: unchecked global warming, pollution, poverty, authoritarianism, plummeting biodiversity, and more. The corporations are like out of control AIs themselves, that will do anything to make their paperclips.
- andsoitis 1mo ago
  > The corporations are like out of control AIs themselves, that will do anything to make their paperclips. Ideas manipulate humans like pets. Ideas have a life of their own and the evolutionary context within which they live is human culture which they have instigated and change, but also in grey matter, which itself is the product of DNA which is really just information in physical form. It’s information across the stack. Or perhaps the way to think about it is as a dimension, much like time, that cuts perpendicularly through the stack.
- pjc50 1mo ago
  > I'm still not seeing the route to that existential risk, not yet anyway. Power consumption needs -> reinstate fossil fuels -> warming above 2C -> clathrate gun
  - imglorp 1mo ago
    Absolutely happening now, but that's not caused by AI though. AI is not choosing that. The corporations are wielding AI irresponsibly and choosing a massive build-out speculating on possible future gains. They did that for crypto too, if you remember them buying a fossil plant just to drive a mining farm.
    - pjc50 1mo ago
      > but that's not caused by AI though. AI is not choosing that AI can't not "choose" that, because AI is dependent on consuming vast amounts of electricity, and everything follows from that. > The corporations are wielding AI irresponsibly This is the most likely route to "AI apocalypse", yes. Humans are neither going to give up control to AI, nor give up the pursuit of profit for the long term survival of others. The distinction won't really matter as to whether it's "corporation" or "AI", if it's an AI-wielding corporation.
pjc50 1mo ago
> either it helps us conquer death, or it poses a massive —say, existential- risk to us all Lesson of 2026: there is no "we". Hobbes is dead but still looming over us. The risk is that it helps some people conquer death, while posing an existential risk to some people. The US has enough problems with its gerontocracy. There has only ever been one president born after 1946. What if you were right and there never was another one born after 1946? Propped up by an immortal Supreme Court on "life" terms, of course. Life extension is also further fatal to birth rates, of course. I suppose the worst possible outcome is technology that extends life but not working life, so you get a larger and larger population of people too old to look after themselves.
- bbor 1mo ago
  I don’t question your portrayal of the worst possible outcomes — certainly victory over senescence could send us down all sorts of dysfunctional local minima. But I encourage you to try to intentionally invest in thinking through some nice ones, too! That’s my main and primary point, to be clear: hope is essential to human flourishing, and it admits no justification by design, unlike all other beliefs. But to comment on a few particulars, on the off chance you/lurkers ever scroll back over this convo: 1. Surely you’d agree there’s a “we” in the sense of communal risk, yes? Obviously I’m thinking about xrisk again, but this applies equally as well to threats we all agree are grave, like climate change, pandemics, nuclear war, and hell, even economic adversity. 2. I will admit that I’m missing the particular Hobbes reference, so hopefully that’s not key lol. I know him for the social contract, primarily — are you emphasizing his deadness in order to imply that we no longer abide by said contract? Cause if so… we have problems, but the day to day seems pretty darn regulated still, IMHO. 3. I don’t think there’s anything necessarily wrong with birthrates we would consider low, but u also don’t agree that longevity would necessarily lead us there — a lot of people would probably still have kids in their youth, reveling all along in the fact that they’ll have decades to centuries to share with their adult children (if they want) instead of somewhere around, what… ~25y, on average? I certainly am not having kids now because I want to prioritize my youthful energy on my work, friends, and learning — knowing that I had time on the other side would reverse my personal decision 100%. 4. I think the concerns about technologically-enabled hyper chronic bigotry that you mention are overblown, no matter how many times the honourable Prof. Weinersmith argues the opposite in his comics. I’ve found cogsci, neuropsych, and philmind have all come to share a deep suspicion of the parts of us we consider continuous, persistent, and unified — an existentially confounding idea ofc, but also one that backs up ~~an old adage~~ a line from a recent sketch comedy show: ’People can change.’ Apologies for the rants. And I do feel like your latter description of a possible world would be an incredible setting for a biting scifi satire lol. But in the end: I hope you can find a way to join me in the wilfully-hopeful camp, even if it’s not today or even this year. We have just as much science!

by bbor 1mo ago

chr15m 1mo ago

> we now have superintelligence Wait what?

cdelsolar 1mo ago
There’s a magic box that can literally build you anything you want that you can control from your phone while playing bedtime songs for your kid
- irishcoffee 1mo ago
  Anything I want? Like a car? Or a football stadium? Your prompt skills up to snuff? I crack myself up.
  - johnsmith1840 1mo ago
    Once robots mature, yes. People will literally prompt buildings into existance. I think that's more than 10yrs from now but if coding AGI or close enough to it becomes real robotics will also accelerate.
    - weatherlite 1mo ago
      Well you still need land and a bunch of permits, access to materials , trucks, equipment etc. But yeah I guess big construction companies will start to automate in the next decades. It's more interesting to me to build an entire subarb/city in a few months instead of yeras to deal with the rising housing costs. Looks like camp deflation was right but we'll see...
      - jbstack 1mo ago
        > Permits Obtained by your AI agent, granted by another AI agent. > Materials, trucks, equipment etc. Ordered by your AI agent, delivered by a self-driving truck or drone, loaded/unloaded by robots to/from the warehouse/site. Most of this stuff can already be done today in some form, so its just about improving the autonomy and integrations along the supply chain.
  - jbstack 1mo ago
    Many factories in China now already have 90% of their work being done by robots. Is it really so hard to imagine that we reach a point where they can be prompted?
  - bawana 1mo ago
    why isnt there an ai front end for 3d printers?
    - BirAdam 1mo ago
      Give it time.
- jdkdidjmdns 1mo ago
  you must have very little imagination
- chr15m 1mo ago
  It's absolutely incredible!

imglorp 1mo ago

I'm still not seeing the route to that existential risk, not yet anyway. Far more imminent is the very real risk of unbounded greed and power that wields those AIs, in full swing as we watch: unchecked global warming, pollution, poverty, authoritarianism, plummeting biodiversity, and more. The corporations are like out of control AIs themselves, that will do anything to make their paperclips.

andsoitis 1mo ago
> The corporations are like out of control AIs themselves, that will do anything to make their paperclips. Ideas manipulate humans like pets. Ideas have a life of their own and the evolutionary context within which they live is human culture which they have instigated and change, but also in grey matter, which itself is the product of DNA which is really just information in physical form. It’s information across the stack. Or perhaps the way to think about it is as a dimension, much like time, that cuts perpendicularly through the stack.
pjc50 1mo ago
> I'm still not seeing the route to that existential risk, not yet anyway. Power consumption needs -> reinstate fossil fuels -> warming above 2C -> clathrate gun
- imglorp 1mo ago
  Absolutely happening now, but that's not caused by AI though. AI is not choosing that. The corporations are wielding AI irresponsibly and choosing a massive build-out speculating on possible future gains. They did that for crypto too, if you remember them buying a fossil plant just to drive a mining farm.
  - pjc50 1mo ago
    > but that's not caused by AI though. AI is not choosing that AI can't not "choose" that, because AI is dependent on consuming vast amounts of electricity, and everything follows from that. > The corporations are wielding AI irresponsibly This is the most likely route to "AI apocalypse", yes. Humans are neither going to give up control to AI, nor give up the pursuit of profit for the long term survival of others. The distinction won't really matter as to whether it's "corporation" or "AI", if it's an AI-wielding corporation.

pjc50 1mo ago

> either it helps us conquer death, or it poses a massive —say, existential- risk to us all Lesson of 2026: there is no "we". Hobbes is dead but still looming over us. The risk is that it helps some people conquer death, while posing an existential risk to some people. The US has enough problems with its gerontocracy. There has only ever been one president born after 1946. What if you were right and there never was another one born after 1946? Propped up by an immortal Supreme Court on "life" terms, of course. Life extension is also further fatal to birth rates, of course. I suppose the worst possible outcome is technology that extends life but not working life, so you get a larger and larger population of people too old to look after themselves.

bbor 1mo ago
I don’t question your portrayal of the worst possible outcomes — certainly victory over senescence could send us down all sorts of dysfunctional local minima. But I encourage you to try to intentionally invest in thinking through some nice ones, too! That’s my main and primary point, to be clear: hope is essential to human flourishing, and it admits no justification by design, unlike all other beliefs. But to comment on a few particulars, on the off chance you/lurkers ever scroll back over this convo: 1. Surely you’d agree there’s a “we” in the sense of communal risk, yes? Obviously I’m thinking about xrisk again, but this applies equally as well to threats we all agree are grave, like climate change, pandemics, nuclear war, and hell, even economic adversity. 2. I will admit that I’m missing the particular Hobbes reference, so hopefully that’s not key lol. I know him for the social contract, primarily — are you emphasizing his deadness in order to imply that we no longer abide by said contract? Cause if so… we have problems, but the day to day seems pretty darn regulated still, IMHO. 3. I don’t think there’s anything necessarily wrong with birthrates we would consider low, but u also don’t agree that longevity would necessarily lead us there — a lot of people would probably still have kids in their youth, reveling all along in the fact that they’ll have decades to centuries to share with their adult children (if they want) instead of somewhere around, what… ~25y, on average? I certainly am not having kids now because I want to prioritize my youthful energy on my work, friends, and learning — knowing that I had time on the other side would reverse my personal decision 100%. 4. I think the concerns about technologically-enabled hyper chronic bigotry that you mention are overblown, no matter how many times the honourable Prof. Weinersmith argues the opposite in his comics. I’ve found cogsci, neuropsych, and philmind have all come to share a deep suspicion of the parts of us we consider continuous, persistent, and unified — an existentially confounding idea ofc, but also one that backs up ~~an old adage~~ a line from a recent sketch comedy show: ’People can change.’ Apologies for the rants. And I do feel like your latter description of a possible world would be an incredible setting for a biting scifi satire lol. But in the end: I hope you can find a way to join me in the wilfully-hopeful camp, even if it’s not today or even this year. We have just as much science!