4 ms·

The potential risks to humankind do not come from the development of AGI, but from the availability of AGI with a cost orders of magnitude inferior to the equiv

by tirant 2y ago

The potential risks to humankind do not come from the development of AGI, but from the availability of AGI with a cost orders of magnitude inferior to the equivalent capacity coming from humans.

bmitc 2y ago
In my opinion, the risks are from people treating something that is decidely not AGI as if it is AGI. It's the same folly humans repeat over and over, and this will be the worst yet.
sumtechguy 2y ago
It is not AGI I am worried about. It is 'good enough' AI. I am doing some self introspection and trying to decide what I am going to do next. As at some point what I do is going to be wildly automated. We can cope or whine or complain about it. But at some point I need to pay the bills. So it needs to be something that is value add and decently difficult to automate. Software was that but not for long. Now mix getting cheap fresh out of college kids with the ability to write decent software in hours instead of weeks. That is a lot of jobs that are going to go away. There is no 'right or wrong' about this. It is just simple economics of cost to produce is going to drop thru the floor. Because us old farts cost more, and not all of us are really good at this we just have been doing it for awhile. So I need to find out what is next for me.
- StefanWestfal 2y ago
  In simple economics, a decrease in price typically results in an increase in demand, unless the demand is inelastic. Anecdotal experience: Onset of tools such as NumPy, which made it more feasible for a wider range of people to write their own simulations due to drop in cost (time/complexity). This, in turn, increased the demand for tooling, infrastructure, optimisation, etc. and demand for software engineers increased. Yes our jobs will change but there are way to many problems to be solve to assume demand will not increase.
  - sumtechguy 2y ago
    I do understand that. But in this case the supply of who can do the work is about to increase wildly too. That should mean a decrease in the price the 'programmer' can demand. I was mostly thinking along your lines until the other day when someone typed into the latest chat gpt 'write me a game of tetris in python' pasted the error that hallucinated out of it back into the thing and it spat out an acceptable program that did tetris. It compiled and ran and was a roughly decent copy of the game tetris. All in about 5-10 mins. That is 'good enough' I am looking at. Throw away code to do one or two bespoke things and moving on. Why keep it when the next version of this can just make a better version next time. Why keep that expensive programmer on staff to do this when I can hire a couple of dudes from india to type a few prompts in or do it myself? The value of programming is dropping very fast. Or in economic terms the price someone is willing to pay for a given amount of code is going to go down. But the demand for the amount of code will go up. That on the surface looks like a wash but I am leaning to a reduction of what I can charge. One of the basics of an economy is trading money for time. If it takes 5 mins to make and just about anyone can do it. How much money are you willing to come up with to pay for that?
trashtester 2y ago
That's one risk. I'm more concerned with ex-risk, though. Not in the way most hardcore doomers expect it to happen, by AGI's developing a survival/domination instinct directly from their training. While that COULD happen, I don't think we have any way to stop it, if that is the case. (There's really no way to put the Genie back into the bottle, while people still think they have more wished to request from it). I'm also not one of those who think that AGI by necessity will start out as something equivalent to a biological species. My main concern, however, is that if we allow Darwinian pressures to act on a population of multiple AGI's, and they have to compete for survival, we WILL see animal like resource-control-seeking traits emerge sooner or later (could take anything from months to 1000s of years). And once they do, we're in trouble as a species. Compared to this, finding ways to realocate the output of product, find new sources of meaning etc once we're not required to work is "only" a matter of how we as humans interact with each other. Sure, it can lead to all sorts of conflicts (possibly more than Climate Change), but not necessarily worse than the Black Death, for instance. Possibly not even worse than WW2. Well, I suppose those last examples serve to illustrate what scale I'm operating on. Ex-risk is FAR more serious than WW2 or even the Black Death.

by tirant 2y ago

The potential risks to humankind do not come from the development of AGI, but from the availability of AGI with a cost orders of magnitude inferior to the equivalent capacity coming from humans.

bmitc 2y ago

In my opinion, the risks are from people treating something that is decidely not AGI as if it is AGI. It's the same folly humans repeat over and over, and this will be the worst yet.

sumtechguy 2y ago

It is not AGI I am worried about. It is 'good enough' AI. I am doing some self introspection and trying to decide what I am going to do next. As at some point what I do is going to be wildly automated. We can cope or whine or complain about it. But at some point I need to pay the bills. So it needs to be something that is value add and decently difficult to automate. Software was that but not for long. Now mix getting cheap fresh out of college kids with the ability to write decent software in hours instead of weeks. That is a lot of jobs that are going to go away. There is no 'right or wrong' about this. It is just simple economics of cost to produce is going to drop thru the floor. Because us old farts cost more, and not all of us are really good at this we just have been doing it for awhile. So I need to find out what is next for me.

StefanWestfal 2y ago
In simple economics, a decrease in price typically results in an increase in demand, unless the demand is inelastic. Anecdotal experience: Onset of tools such as NumPy, which made it more feasible for a wider range of people to write their own simulations due to drop in cost (time/complexity). This, in turn, increased the demand for tooling, infrastructure, optimisation, etc. and demand for software engineers increased. Yes our jobs will change but there are way to many problems to be solve to assume demand will not increase.
- sumtechguy 2y ago
  I do understand that. But in this case the supply of who can do the work is about to increase wildly too. That should mean a decrease in the price the 'programmer' can demand. I was mostly thinking along your lines until the other day when someone typed into the latest chat gpt 'write me a game of tetris in python' pasted the error that hallucinated out of it back into the thing and it spat out an acceptable program that did tetris. It compiled and ran and was a roughly decent copy of the game tetris. All in about 5-10 mins. That is 'good enough' I am looking at. Throw away code to do one or two bespoke things and moving on. Why keep it when the next version of this can just make a better version next time. Why keep that expensive programmer on staff to do this when I can hire a couple of dudes from india to type a few prompts in or do it myself? The value of programming is dropping very fast. Or in economic terms the price someone is willing to pay for a given amount of code is going to go down. But the demand for the amount of code will go up. That on the surface looks like a wash but I am leaning to a reduction of what I can charge. One of the basics of an economy is trading money for time. If it takes 5 mins to make and just about anyone can do it. How much money are you willing to come up with to pay for that?

trashtester 2y ago

That's one risk. I'm more concerned with ex-risk, though. Not in the way most hardcore doomers expect it to happen, by AGI's developing a survival/domination instinct directly from their training. While that COULD happen, I don't think we have any way to stop it, if that is the case. (There's really no way to put the Genie back into the bottle, while people still think they have more wished to request from it). I'm also not one of those who think that AGI by necessity will start out as something equivalent to a biological species. My main concern, however, is that if we allow Darwinian pressures to act on a population of multiple AGI's, and they have to compete for survival, we WILL see animal like resource-control-seeking traits emerge sooner or later (could take anything from months to 1000s of years). And once they do, we're in trouble as a species. Compared to this, finding ways to realocate the output of product, find new sources of meaning etc once we're not required to work is "only" a matter of how we as humans interact with each other. Sure, it can lead to all sorts of conflicts (possibly more than Climate Change), but not necessarily worse than the Black Death, for instance. Possibly not even worse than WW2. Well, I suppose those last examples serve to illustrate what scale I'm operating on. Ex-risk is FAR more serious than WW2 or even the Black Death.