5 ms·

Unfortunately, this is essentially high level clickbait. At the beginning he immediately portrays the issue as black and white: >AI will either be the greates

Unfortunately, this is essentially high level clickbait. At the beginning he immediately portrays the issue as black and white: >AI will either be the greatest equalizer ever invented, or the worst source of injustice. In all likelihood it will probably fall in the middle, with the likelihood that it will tilt power and money towards the already ultra wealthy (to the surprise of no one). But it will enable ordinary people and small businesses to do cool shit and make their lives better.

enos_feedler 1mo ago
Nailed it. Instead of reading this long form post I will eat popcorn and watch the show unfold as laid out here.
- erickhill 1mo ago
  There's also the video version... https://www.youtube.com/watch?v=U4zGLSlLo5A https://www.youtube.com/watch?v=U4zGLSlLo5A
bbor 1mo ago
I don’t see any reason this highly chaotic system is most likely to settle in a spot that we happen to enjoy/see as a compromise of some sort. The point is that we now have superintelligence; either it helps us conquer death, or it poses a massive —say, existential- risk to us all. It’s hard to imagine such a powerful tool resulting in just another century of slightly-better capitalist nationalism. I guess there could be a second AI winter around the corner that we haven’t seen coming yet. That would be nice! Not likely IMHO, but nice.
- chr15m 1mo ago
  > we now have superintelligence Wait what?
  - cdelsolar 1mo ago
    There’s a magic box that can literally build you anything you want that you can control from your phone while playing bedtime songs for your kid
    - irishcoffee 1mo ago
      Anything I want? Like a car? Or a football stadium? Your prompt skills up to snuff? I crack myself up.
      - johnsmith1840 1mo ago
        Once robots mature, yes. People will literally prompt buildings into existance. I think that's more than 10yrs from now but if coding AGI or close enough to it becomes real robotics will also accelerate.
        weatherlite 1mo ago
        Well you still need land and a bunch of permits, access to materials , trucks, equipment etc. But yeah I guess big construction companies will start to automate in the next decades. It's more interesting to me to build an entire subarb/city in a few months instead of yeras to deal with the rising housing costs. Looks like camp deflation was right but we'll see...
        jbstack 29d ago
        > Permits Obtained by your AI agent, granted by another AI agent. > Materials, trucks, equipment etc. Ordered by your AI agent, delivered by a self-driving truck or drone, loaded/unloaded by robots to/from the warehouse/site. Most of this stuff can already be done today in some form, so its just about improving the autonomy and integrations along the supply chain.
      - jbstack 29d ago
        Many factories in China now already have 90% of their work being done by robots. Is it really so hard to imagine that we reach a point where they can be prompted?
- imglorp 1mo ago
  I'm still not seeing the route to that existential risk, not yet anyway. Far more imminent is the very real risk of unbounded greed and power that wields those AIs, in full swing as we watch: unchecked global warming, pollution, poverty, authoritarianism, plummeting biodiversity, and more. The corporations are like out of control AIs themselves, that will do anything to make their paperclips.
  - andsoitis 1mo ago
    > The corporations are like out of control AIs themselves, that will do anything to make their paperclips. Ideas manipulate humans like pets. Ideas have a life of their own and the evolutionary context within which they live is human culture which they have instigated and change, but also in grey matter, which itself is the product of DNA which is really just information in physical form. It’s information across the stack. Or perhaps the way to think about it is as a dimension, much like time, that cuts perpendicularly through the stack.
  - pjc50 29d ago
    > I'm still not seeing the route to that existential risk, not yet anyway. Power consumption needs -> reinstate fossil fuels -> warming above 2C -> clathrate gun
    - imglorp 29d ago
      Absolutely happening now, but that's not caused by AI though. AI is not choosing that. The corporations are wielding AI irresponsibly and choosing a massive build-out speculating on possible future gains. They did that for crypto too, if you remember them buying a fossil plant just to drive a mining farm.
      - pjc50 29d ago
        > but that's not caused by AI though. AI is not choosing that AI can't not "choose" that, because AI is dependent on consuming vast amounts of electricity, and everything follows from that. > The corporations are wielding AI irresponsibly This is the most likely route to "AI apocalypse", yes. Humans are neither going to give up control to AI, nor give up the pursuit of profit for the long term survival of others. The distinction won't really matter as to whether it's "corporation" or "AI", if it's an AI-wielding corporation.
- pjc50 29d ago
  > either it helps us conquer death, or it poses a massive —say, existential- risk to us all Lesson of 2026: there is no "we". Hobbes is dead but still looming over us. The risk is that it helps some people conquer death, while posing an existential risk to some people. The US has enough problems with its gerontocracy. There has only ever been one president born after 1946. What if you were right and there never was another one born after 1946? Propped up by an immortal Supreme Court on "life" terms, of course. Life extension is also further fatal to birth rates, of course. I suppose the worst possible outcome is technology that extends life but not working life, so you get a larger and larger population of people too old to look after themselves.
  - bbor 27d ago
    I don’t question your portrayal of the worst possible outcomes — certainly victory over senescence could send us down all sorts of dysfunctional local minima. But I encourage you to try to intentionally invest in thinking through some nice ones, too! That’s my main and primary point, to be clear: hope is essential to human flourishing, and it admits no justification by design, unlike all other beliefs. But to comment on a few particulars, on the off chance you/lurkers ever scroll back over this convo: 1. Surely you’d agree there’s a “we” in the sense of communal risk, yes? Obviously I’m thinking about xrisk again, but this applies equally as well to threats we all agree are grave, like climate change, pandemics, nuclear war, and hell, even economic adversity. 2. I will admit that I’m missing the particular Hobbes reference, so hopefully that’s not key lol. I know him for the social contract, primarily — are you emphasizing his deadness in order to imply that we no longer abide by said contract? Cause if so… we have problems, but the day to day seems pretty darn regulated still, IMHO. 3. I don’t think there’s anything necessarily wrong with birthrates we would consider low, but u also don’t agree that longevity would necessarily lead us there — a lot of people would probably still have kids in their youth, reveling all along in the fact that they’ll have decades to centuries to share with their adult children (if they want) instead of somewhere around, what… ~25y, on average? I certainly am not having kids now because I want to prioritize my youthful energy on my work, friends, and learning — knowing that I had time on the other side would reverse my personal decision 100%. 4. I think the concerns about technologically-enabled hyper chronic bigotry that you mention are overblown, no matter how many times the honourable Prof. Weinersmith argues the opposite in his comics. I’ve found cogsci, neuropsych, and philmind have all come to share a deep suspicion of the parts of us we consider continuous, persistent, and unified — an existentially confounding idea ofc, but also one that backs up ~~an old adage~~ a line from a recent sketch comedy show: ’People can change.’ Apologies for the rants. And I do feel like your latter description of a possible world would be an incredible setting for a biting scifi satire lol. But in the end: I hope you can find a way to join me in the wilfully-hopeful camp, even if it’s not today or even this year. We have just as much science!
pydry 1mo ago
It reads like that kind of AI marketing which focuses on how "scary" a new model supposedly is. Bill Gates' fear is clearly that everybody thinks this stuff is just kind of mediocre because then the real horror story unfolds - Microsoft stock price collapses into blue chip territory where it firmly belongs.
- erickhill 1mo ago
  Actually his real worry is the lack of any kind of oversight and regulation - that the inmates are allowed to police themselves.
- dudinax 1mo ago
  He calls for a tax on tokens.
  - pydry 1mo ago
    Yeah, he called for a tax on "job eliminating" robots too. Neither are in the slightest bit practical to implement and he knows it. He's just grandstanding while talking his book.
    - pjc50 29d ago
      Token tax is fairly straightforward to implement on the American AI companies. The international aspects of it are a lot more complicated.
      - pydry 29d ago
        it's not straightforward at all. the AI companies would just redefine what a token is to miminize their tax bill. or, relocate abroad. Bill Gates' wealth would be nice and easy to tax though and would provide many of the same rewards he purported to want from an AI or robot tax.
  - BirAdam 29d ago
    Tax and regulation is how you pull the ladder up behind yourself, so... naturally Gates is in favor of this.
theptip 1mo ago
IMO, this is a highly bimodal probability distribution. I model this as two very strong attractor states (radical abundance and totalitarian power concentration). I believe the latter is way more likely without a concerted effort that we currently seem incapable of. The “somewhere in the middle” outcomes in my opinion are not AGI-pilled, and only come about if the exponential dies hard in the next year or maybe two - as in, it’s a very specific timing prediction that needs to come true for this scenario to occur.
- fluoridation 1mo ago
  What exponential?
  - theptip 1mo ago
    LLM capability improvement, eg as measured by METR task time.
    - fluoridation 29d ago
      So it's an exponential with a negative exponent? The thing about those kinds of curves is that since they have asymptotes to zero, it's hard to tell when you've plateaued.
      - theptip 29d ago
        No, the metric is “p50 task duration”. 7-month doubling time, recently 4-month: https://metr.org/ https://metr.org/ Plenty of other exponential metrics too, compute built, AI revenue, etc.
  - KellyCriterion 28d ago
    monthly AI bills :-)
- newsomix9xl 1mo ago
  Why is totalitarianism always the "easy likely" outcome? These are often hard fought bloody and some times brilliantly organized strategic and tactical victories. That took years. Years. Its portrayed like "oh, good guys stayed home, I guess the dictatorship wins". Really?
skew-aberration 29d ago
I don't think that's necessarily unlikely, but people making this argument rarely seem to understand just how radical a change is required to make that a possibility. e.g most small business rent their premises, and buy goods and services from other tenant small businesses, who buy their raw materials from large landowners. How can a small business survive when all the land is monopolized by the technocrats? All value you add gets taken away as rent. This is why they always talk about taxes on outputs (tokens) instead of inputs (land, energy usage). It's not an ungrounded hypothetical, that's exactly how society worked for hundreds (thousands?) of years.
taskforcegemini 29d ago
isn't "making lives better" basically the reason for pretty much any companies existence? arguable of course whise lives are meant to become better, but if anything ai just moves the goalpost
bithammerthunde 29d ago
> In all likelihood it will probably fall in the middle AI era has been called the new industrial revolution. I'd just like to point out that the industrial revolution created the British Empire. That's wasn't exactly "in the middle".
francisofascii 29d ago
>AI will either be the greatest equalizer ever invented, or the worst source of injustice. It can be both. With vanishing white collar work and decent wages, the middle class gets dragged down to become equal to the lower class.
frigg 29d ago
I was also disappointed and stopped reading mid-way. There is, unfortunately, no original idea or thought in this post.
KellyCriterion 28d ago
My guess is that we will run out of cheap/available energy earlier, so neither of these extreme states can be reached.

enos_feedler 1mo ago

Nailed it. Instead of reading this long form post I will eat popcorn and watch the show unfold as laid out here.

erickhill 1mo ago
There's also the video version... https://www.youtube.com/watch?v=U4zGLSlLo5A https://www.youtube.com/watch?v=U4zGLSlLo5A

bbor 1mo ago

I don’t see any reason this highly chaotic system is most likely to settle in a spot that we happen to enjoy/see as a compromise of some sort. The point is that we now have superintelligence; either it helps us conquer death, or it poses a massive —say, existential- risk to us all. It’s hard to imagine such a powerful tool resulting in just another century of slightly-better capitalist nationalism. I guess there could be a second AI winter around the corner that we haven’t seen coming yet. That would be nice! Not likely IMHO, but nice.

chr15m 1mo ago
> we now have superintelligence Wait what?
- cdelsolar 1mo ago
  There’s a magic box that can literally build you anything you want that you can control from your phone while playing bedtime songs for your kid
  - irishcoffee 1mo ago
    Anything I want? Like a car? Or a football stadium? Your prompt skills up to snuff? I crack myself up.
    - johnsmith1840 1mo ago
      Once robots mature, yes. People will literally prompt buildings into existance. I think that's more than 10yrs from now but if coding AGI or close enough to it becomes real robotics will also accelerate.
      - weatherlite 1mo ago
        Well you still need land and a bunch of permits, access to materials , trucks, equipment etc. But yeah I guess big construction companies will start to automate in the next decades. It's more interesting to me to build an entire subarb/city in a few months instead of yeras to deal with the rising housing costs. Looks like camp deflation was right but we'll see...
        jbstack 29d ago
        > Permits Obtained by your AI agent, granted by another AI agent. > Materials, trucks, equipment etc. Ordered by your AI agent, delivered by a self-driving truck or drone, loaded/unloaded by robots to/from the warehouse/site. Most of this stuff can already be done today in some form, so its just about improving the autonomy and integrations along the supply chain.
    - jbstack 29d ago
      Many factories in China now already have 90% of their work being done by robots. Is it really so hard to imagine that we reach a point where they can be prompted?
imglorp 1mo ago
I'm still not seeing the route to that existential risk, not yet anyway. Far more imminent is the very real risk of unbounded greed and power that wields those AIs, in full swing as we watch: unchecked global warming, pollution, poverty, authoritarianism, plummeting biodiversity, and more. The corporations are like out of control AIs themselves, that will do anything to make their paperclips.
- andsoitis 1mo ago
  > The corporations are like out of control AIs themselves, that will do anything to make their paperclips. Ideas manipulate humans like pets. Ideas have a life of their own and the evolutionary context within which they live is human culture which they have instigated and change, but also in grey matter, which itself is the product of DNA which is really just information in physical form. It’s information across the stack. Or perhaps the way to think about it is as a dimension, much like time, that cuts perpendicularly through the stack.
- pjc50 29d ago
  > I'm still not seeing the route to that existential risk, not yet anyway. Power consumption needs -> reinstate fossil fuels -> warming above 2C -> clathrate gun
  - imglorp 29d ago
    Absolutely happening now, but that's not caused by AI though. AI is not choosing that. The corporations are wielding AI irresponsibly and choosing a massive build-out speculating on possible future gains. They did that for crypto too, if you remember them buying a fossil plant just to drive a mining farm.
    - pjc50 29d ago
      > but that's not caused by AI though. AI is not choosing that AI can't not "choose" that, because AI is dependent on consuming vast amounts of electricity, and everything follows from that. > The corporations are wielding AI irresponsibly This is the most likely route to "AI apocalypse", yes. Humans are neither going to give up control to AI, nor give up the pursuit of profit for the long term survival of others. The distinction won't really matter as to whether it's "corporation" or "AI", if it's an AI-wielding corporation.
pjc50 29d ago
> either it helps us conquer death, or it poses a massive —say, existential- risk to us all Lesson of 2026: there is no "we". Hobbes is dead but still looming over us. The risk is that it helps some people conquer death, while posing an existential risk to some people. The US has enough problems with its gerontocracy. There has only ever been one president born after 1946. What if you were right and there never was another one born after 1946? Propped up by an immortal Supreme Court on "life" terms, of course. Life extension is also further fatal to birth rates, of course. I suppose the worst possible outcome is technology that extends life but not working life, so you get a larger and larger population of people too old to look after themselves.
- bbor 27d ago
  I don’t question your portrayal of the worst possible outcomes — certainly victory over senescence could send us down all sorts of dysfunctional local minima. But I encourage you to try to intentionally invest in thinking through some nice ones, too! That’s my main and primary point, to be clear: hope is essential to human flourishing, and it admits no justification by design, unlike all other beliefs. But to comment on a few particulars, on the off chance you/lurkers ever scroll back over this convo: 1. Surely you’d agree there’s a “we” in the sense of communal risk, yes? Obviously I’m thinking about xrisk again, but this applies equally as well to threats we all agree are grave, like climate change, pandemics, nuclear war, and hell, even economic adversity. 2. I will admit that I’m missing the particular Hobbes reference, so hopefully that’s not key lol. I know him for the social contract, primarily — are you emphasizing his deadness in order to imply that we no longer abide by said contract? Cause if so… we have problems, but the day to day seems pretty darn regulated still, IMHO. 3. I don’t think there’s anything necessarily wrong with birthrates we would consider low, but u also don’t agree that longevity would necessarily lead us there — a lot of people would probably still have kids in their youth, reveling all along in the fact that they’ll have decades to centuries to share with their adult children (if they want) instead of somewhere around, what… ~25y, on average? I certainly am not having kids now because I want to prioritize my youthful energy on my work, friends, and learning — knowing that I had time on the other side would reverse my personal decision 100%. 4. I think the concerns about technologically-enabled hyper chronic bigotry that you mention are overblown, no matter how many times the honourable Prof. Weinersmith argues the opposite in his comics. I’ve found cogsci, neuropsych, and philmind have all come to share a deep suspicion of the parts of us we consider continuous, persistent, and unified — an existentially confounding idea ofc, but also one that backs up ~~an old adage~~ a line from a recent sketch comedy show: ’People can change.’ Apologies for the rants. And I do feel like your latter description of a possible world would be an incredible setting for a biting scifi satire lol. But in the end: I hope you can find a way to join me in the wilfully-hopeful camp, even if it’s not today or even this year. We have just as much science!

pydry 1mo ago

It reads like that kind of AI marketing which focuses on how "scary" a new model supposedly is. Bill Gates' fear is clearly that everybody thinks this stuff is just kind of mediocre because then the real horror story unfolds - Microsoft stock price collapses into blue chip territory where it firmly belongs.

erickhill 1mo ago
Actually his real worry is the lack of any kind of oversight and regulation - that the inmates are allowed to police themselves.
dudinax 1mo ago
He calls for a tax on tokens.
- pydry 1mo ago
  Yeah, he called for a tax on "job eliminating" robots too. Neither are in the slightest bit practical to implement and he knows it. He's just grandstanding while talking his book.
  - pjc50 29d ago
    Token tax is fairly straightforward to implement on the American AI companies. The international aspects of it are a lot more complicated.
    - pydry 29d ago
      it's not straightforward at all. the AI companies would just redefine what a token is to miminize their tax bill. or, relocate abroad. Bill Gates' wealth would be nice and easy to tax though and would provide many of the same rewards he purported to want from an AI or robot tax.
- BirAdam 29d ago
  Tax and regulation is how you pull the ladder up behind yourself, so... naturally Gates is in favor of this.

theptip 1mo ago

IMO, this is a highly bimodal probability distribution. I model this as two very strong attractor states (radical abundance and totalitarian power concentration). I believe the latter is way more likely without a concerted effort that we currently seem incapable of. The “somewhere in the middle” outcomes in my opinion are not AGI-pilled, and only come about if the exponential dies hard in the next year or maybe two - as in, it’s a very specific timing prediction that needs to come true for this scenario to occur.

fluoridation 1mo ago
What exponential?
- theptip 1mo ago
  LLM capability improvement, eg as measured by METR task time.
  - fluoridation 29d ago
    So it's an exponential with a negative exponent? The thing about those kinds of curves is that since they have asymptotes to zero, it's hard to tell when you've plateaued.
    - theptip 29d ago
      No, the metric is “p50 task duration”. 7-month doubling time, recently 4-month: https://metr.org/ https://metr.org/ Plenty of other exponential metrics too, compute built, AI revenue, etc.
- KellyCriterion 28d ago
  monthly AI bills :-)
newsomix9xl 1mo ago
Why is totalitarianism always the "easy likely" outcome? These are often hard fought bloody and some times brilliantly organized strategic and tactical victories. That took years. Years. Its portrayed like "oh, good guys stayed home, I guess the dictatorship wins". Really?

skew-aberration 29d ago

I don't think that's necessarily unlikely, but people making this argument rarely seem to understand just how radical a change is required to make that a possibility. e.g most small business rent their premises, and buy goods and services from other tenant small businesses, who buy their raw materials from large landowners. How can a small business survive when all the land is monopolized by the technocrats? All value you add gets taken away as rent. This is why they always talk about taxes on outputs (tokens) instead of inputs (land, energy usage). It's not an ungrounded hypothetical, that's exactly how society worked for hundreds (thousands?) of years.

taskforcegemini 29d ago

isn't "making lives better" basically the reason for pretty much any companies existence? arguable of course whise lives are meant to become better, but if anything ai just moves the goalpost

bithammerthunde 29d ago

> In all likelihood it will probably fall in the middle AI era has been called the new industrial revolution. I'd just like to point out that the industrial revolution created the British Empire. That's wasn't exactly "in the middle".

francisofascii 29d ago

>AI will either be the greatest equalizer ever invented, or the worst source of injustice. It can be both. With vanishing white collar work and decent wages, the middle class gets dragged down to become equal to the lower class.

frigg 29d ago

I was also disappointed and stopped reading mid-way. There is, unfortunately, no original idea or thought in this post.

KellyCriterion 28d ago

My guess is that we will run out of cheap/available energy earlier, so neither of these extreme states can be reached.