4 ms·

But it makes perfect sense to drop out and enjoy last couple years of pre-AGI bliss. Advances in AI even without AGI will lead to unemployment, recession, coll

by vl 2y ago

But it makes perfect sense to drop out and enjoy last couple years of pre-AGI bliss. Advances in AI even without AGI will lead to unemployment, recession, collapse of our economic structure, and then our social structure. Whatever is on the other side is not pretty. If you are on the forefront, know it’s coming imminently, and made your money, it makes perfect sense to leave and enjoy money and leisures money allows while money still worth something.

bossyTeacher 2y ago
I highly doubt that's the case. The US government will undoubtly seize OpenAI, the assets and employees way before it happens in the name of national security. I am pretty sure that they got a special team keeping an eye on the internal comms at openai to make sure they are on top of their internal affairs.
- sva_ 2y ago
  They already got a retired army general who was head of the NSA on the board lol https://en.wikipedia.org/wiki/Paul_Nakasone https://en.wikipedia.org/wiki/Paul_Nakasone
- cudgy 2y ago
  They don’t have to seize the company. They are likely embedded already and can simply blackmail, legally harass, or “disappear” the uncooperative.
andrepd 2y ago
I'm having genuine trouble understanding if this is real or ironic.
tirant 2y ago
The potential risks to humankind do not come from the development of AGI, but from the availability of AGI with a cost orders of magnitude inferior to the equivalent capacity coming from humans.
- bmitc 2y ago
  In my opinion, the risks are from people treating something that is decidely not AGI as if it is AGI. It's the same folly humans repeat over and over, and this will be the worst yet.
- sumtechguy 2y ago
  It is not AGI I am worried about. It is 'good enough' AI. I am doing some self introspection and trying to decide what I am going to do next. As at some point what I do is going to be wildly automated. We can cope or whine or complain about it. But at some point I need to pay the bills. So it needs to be something that is value add and decently difficult to automate. Software was that but not for long. Now mix getting cheap fresh out of college kids with the ability to write decent software in hours instead of weeks. That is a lot of jobs that are going to go away. There is no 'right or wrong' about this. It is just simple economics of cost to produce is going to drop thru the floor. Because us old farts cost more, and not all of us are really good at this we just have been doing it for awhile. So I need to find out what is next for me.
  - StefanWestfal 2y ago
    In simple economics, a decrease in price typically results in an increase in demand, unless the demand is inelastic. Anecdotal experience: Onset of tools such as NumPy, which made it more feasible for a wider range of people to write their own simulations due to drop in cost (time/complexity). This, in turn, increased the demand for tooling, infrastructure, optimisation, etc. and demand for software engineers increased. Yes our jobs will change but there are way to many problems to be solve to assume demand will not increase.
    - sumtechguy 2y ago
      I do understand that. But in this case the supply of who can do the work is about to increase wildly too. That should mean a decrease in the price the 'programmer' can demand. I was mostly thinking along your lines until the other day when someone typed into the latest chat gpt 'write me a game of tetris in python' pasted the error that hallucinated out of it back into the thing and it spat out an acceptable program that did tetris. It compiled and ran and was a roughly decent copy of the game tetris. All in about 5-10 mins. That is 'good enough' I am looking at. Throw away code to do one or two bespoke things and moving on. Why keep it when the next version of this can just make a better version next time. Why keep that expensive programmer on staff to do this when I can hire a couple of dudes from india to type a few prompts in or do it myself? The value of programming is dropping very fast. Or in economic terms the price someone is willing to pay for a given amount of code is going to go down. But the demand for the amount of code will go up. That on the surface looks like a wash but I am leaning to a reduction of what I can charge. One of the basics of an economy is trading money for time. If it takes 5 mins to make and just about anyone can do it. How much money are you willing to come up with to pay for that?
trashtester 2y ago
Nobody really knows what Earth will look like once AGI arrives. It could be anything from extinction, through some Cyberpunk corporate dystopia (like you seem to think) to some kind of Techno-Socialist utopia. One thing it's not likely to be, is a neo-classical capitalist system based on the value of human labor.
- gnulinux 2y ago
  > One thing it's not likely to be, is a neo-classical capitalist system based on the value of human labor. I'm finding it difficult to believe this. For me, your comment is accurate (and very insightful) except even a mostly vanilla continuation of the neoliberal capitalist system seems possible. I think we're literally talking about a "singularity" where by definition our fate is not dependent on our actions, and of something we don't have the full capacity to understand, and next to no capacity to influence. It needs tremendous amount of evidence to claim anything in such an indeterminate system. Maybe 100 rich people will own all the AI and the rest will be fixing bullshit that AI doesn't even bother fixing like roads, rusty farms etc, similar to Kurt Vonnegut's first novel "Player Piano". Not that the world described in that novel is particularly neoliberal capitalist (I suppose it's a bit more "socialistic" (whatever it means)) than that, but I don't think such a future can be ruled out. My bias is that, of course, it's going to be a bleak future. Because when humanity loses all control, it seems unlikely to me a system that protects the interests of individual or collective humans will take place. So whether it's extinction, cyberpunk, techno-socialism, techno-capitalist libertarian anarchy, neoclassical capitalism... whatever it is, it will be something that'll protect the interest of something inhuman, so much more so than the current system. It goes without saying, I'm an extreme AI pessimist: just making my biases clear. AGI -- while it's unclear if it's technically feasible -- will be the death of humanity as we know it now, but perhaps something else humanity-like, something worse and more painful will follow.
  - trashtester 2y ago
    > I'm finding it difficult to believe this. Pay attention to the whole sentence, especially the last section : "... based on the value of human labor." It's not that I'm ruling out capitalism as the outcome. I'm simply ruling out the combined JOINT possibility of capitalism COMBINED WITH human labor remaining the base resource within it. If robotics is going in the direction I expect there will simply be no jobs left that will be done more efficiently by humans than by machines. (ie that robots will match or exceed the robustness, flexibility and cost efficiency of all biology based life forms through breakthroughs in either nanotech or by simply using organic chemistry, DNA, etc to build the robots). Why pay even $1/day for a human to do a job when a robot can do it for $1/week? Also, such a capitalist system will almost certainly lead to AGI's becoming increasingly like a new life form, as capitalism between AGI's introduce a Darwinian selection pressure. That will make it hard even for the 100 richest people to retain permanent control. IF humanity is to survive (for at least a few thousand more years, not just the next 100), we either need some way to ensure alignment. And to do that, we have to make sure that AGI's that optimize resource-control-seeking behaviours have an advantage over those who don't. We may even have to define some level of sophistication where further development is completly halted. At least until we find ways for humans to merge with them in a way that allows us (at least some of us) to retain our humanity.

by vl 2y ago

bossyTeacher 2y ago

I highly doubt that's the case. The US government will undoubtly seize OpenAI, the assets and employees way before it happens in the name of national security. I am pretty sure that they got a special team keeping an eye on the internal comms at openai to make sure they are on top of their internal affairs.

sva_ 2y ago
They already got a retired army general who was head of the NSA on the board lol https://en.wikipedia.org/wiki/Paul_Nakasone https://en.wikipedia.org/wiki/Paul_Nakasone
cudgy 2y ago
They don’t have to seize the company. They are likely embedded already and can simply blackmail, legally harass, or “disappear” the uncooperative.

andrepd 2y ago

I'm having genuine trouble understanding if this is real or ironic.

tirant 2y ago

The potential risks to humankind do not come from the development of AGI, but from the availability of AGI with a cost orders of magnitude inferior to the equivalent capacity coming from humans.

bmitc 2y ago
In my opinion, the risks are from people treating something that is decidely not AGI as if it is AGI. It's the same folly humans repeat over and over, and this will be the worst yet.
sumtechguy 2y ago
It is not AGI I am worried about. It is 'good enough' AI. I am doing some self introspection and trying to decide what I am going to do next. As at some point what I do is going to be wildly automated. We can cope or whine or complain about it. But at some point I need to pay the bills. So it needs to be something that is value add and decently difficult to automate. Software was that but not for long. Now mix getting cheap fresh out of college kids with the ability to write decent software in hours instead of weeks. That is a lot of jobs that are going to go away. There is no 'right or wrong' about this. It is just simple economics of cost to produce is going to drop thru the floor. Because us old farts cost more, and not all of us are really good at this we just have been doing it for awhile. So I need to find out what is next for me.
- StefanWestfal 2y ago
  In simple economics, a decrease in price typically results in an increase in demand, unless the demand is inelastic. Anecdotal experience: Onset of tools such as NumPy, which made it more feasible for a wider range of people to write their own simulations due to drop in cost (time/complexity). This, in turn, increased the demand for tooling, infrastructure, optimisation, etc. and demand for software engineers increased. Yes our jobs will change but there are way to many problems to be solve to assume demand will not increase.
  - sumtechguy 2y ago
    I do understand that. But in this case the supply of who can do the work is about to increase wildly too. That should mean a decrease in the price the 'programmer' can demand. I was mostly thinking along your lines until the other day when someone typed into the latest chat gpt 'write me a game of tetris in python' pasted the error that hallucinated out of it back into the thing and it spat out an acceptable program that did tetris. It compiled and ran and was a roughly decent copy of the game tetris. All in about 5-10 mins. That is 'good enough' I am looking at. Throw away code to do one or two bespoke things and moving on. Why keep it when the next version of this can just make a better version next time. Why keep that expensive programmer on staff to do this when I can hire a couple of dudes from india to type a few prompts in or do it myself? The value of programming is dropping very fast. Or in economic terms the price someone is willing to pay for a given amount of code is going to go down. But the demand for the amount of code will go up. That on the surface looks like a wash but I am leaning to a reduction of what I can charge. One of the basics of an economy is trading money for time. If it takes 5 mins to make and just about anyone can do it. How much money are you willing to come up with to pay for that?

trashtester 2y ago

Nobody really knows what Earth will look like once AGI arrives. It could be anything from extinction, through some Cyberpunk corporate dystopia (like you seem to think) to some kind of Techno-Socialist utopia. One thing it's not likely to be, is a neo-classical capitalist system based on the value of human labor.

gnulinux 2y ago
> One thing it's not likely to be, is a neo-classical capitalist system based on the value of human labor. I'm finding it difficult to believe this. For me, your comment is accurate (and very insightful) except even a mostly vanilla continuation of the neoliberal capitalist system seems possible. I think we're literally talking about a "singularity" where by definition our fate is not dependent on our actions, and of something we don't have the full capacity to understand, and next to no capacity to influence. It needs tremendous amount of evidence to claim anything in such an indeterminate system. Maybe 100 rich people will own all the AI and the rest will be fixing bullshit that AI doesn't even bother fixing like roads, rusty farms etc, similar to Kurt Vonnegut's first novel "Player Piano". Not that the world described in that novel is particularly neoliberal capitalist (I suppose it's a bit more "socialistic" (whatever it means)) than that, but I don't think such a future can be ruled out. My bias is that, of course, it's going to be a bleak future. Because when humanity loses all control, it seems unlikely to me a system that protects the interests of individual or collective humans will take place. So whether it's extinction, cyberpunk, techno-socialism, techno-capitalist libertarian anarchy, neoclassical capitalism... whatever it is, it will be something that'll protect the interest of something inhuman, so much more so than the current system. It goes without saying, I'm an extreme AI pessimist: just making my biases clear. AGI -- while it's unclear if it's technically feasible -- will be the death of humanity as we know it now, but perhaps something else humanity-like, something worse and more painful will follow.
- trashtester 2y ago
  > I'm finding it difficult to believe this. Pay attention to the whole sentence, especially the last section : "... based on the value of human labor." It's not that I'm ruling out capitalism as the outcome. I'm simply ruling out the combined JOINT possibility of capitalism COMBINED WITH human labor remaining the base resource within it. If robotics is going in the direction I expect there will simply be no jobs left that will be done more efficiently by humans than by machines. (ie that robots will match or exceed the robustness, flexibility and cost efficiency of all biology based life forms through breakthroughs in either nanotech or by simply using organic chemistry, DNA, etc to build the robots). Why pay even $1/day for a human to do a job when a robot can do it for $1/week? Also, such a capitalist system will almost certainly lead to AGI's becoming increasingly like a new life form, as capitalism between AGI's introduce a Darwinian selection pressure. That will make it hard even for the 100 richest people to retain permanent control. IF humanity is to survive (for at least a few thousand more years, not just the next 100), we either need some way to ensure alignment. And to do that, we have to make sure that AGI's that optimize resource-control-seeking behaviours have an advantage over those who don't. We may even have to define some level of sophistication where further development is completly halted. At least until we find ways for humans to merge with them in a way that allows us (at least some of us) to retain our humanity.