3 ms·

Giving and social interaction (of at least some kinds) are not immediately at risk. What's at risk is the participation of some large groups of people in trad

Giving and social interaction (of at least some kinds) are not immediately at risk. What's at risk is the participation of some large groups of people in trade. Ultimately the value of a person's labor depends on scarcity, just as the trade value of things or experiences do, and we've seen how lack of scarcity makes trade value crash. (The value of the Humble Bundle and Cory Doctorow's books depends on "giving" more than "trade", and the latter at all only due to information asymmetry.) The storm coming is that when we have duplicable, cheap, general AI, the value of any act of production will plummet to the cost of copying a mind and running it. Actually, that's already the case, but the cost of producing a new mind is quite high, now. :) When people talk about automating everything we now do (or can do), pro-automation people often say "well, comparative advantage means that there will always be something for humans to do to live", but comparative advantage depends on scarcity of productive agents. If copying and running an AI to solve a problem is cheaper than employing an already existing human, humans are in trouble, economically. Dunno what to do about it, though.

DanielBMarkham 14y ago
I believe technologists continue to make the mistake of assuming everything is digital. Perhaps it's a part of the job. "comparative advantage depends on scarcity of productive agents" People buy things (trade) based on perceived value. That perception can be based on perceived scarcity, sentiment, anger, fear, love, happiness -- the list is very long. The scarcity argument only holds true across large industries and populations, and that's only because of the fungibility of money and the fact that material goods are (rather) easily categorized. It's not going to continue working. Money will stay fungible, but we're going to see an explosion in the kinds of things that have perceived value that the world has never seen. Which makes sense if you think about it: as mankind has progressed, the diversity of the things he trades has increased. This trend will continue. You guys are confusing theory and reality. Macro-economics fails us here. That's a shame. Looks like some folks have more work to do :)
- randallsquared 14y ago
  It doesn't matter if everything is digital, if the creators of everything can be. > The scarcity argument only holds true across large industries and populations I don't think this is so, unless you're talking about handcrafted-by-genuine-Ukranian-American underwater-woven baskets. For anything that can be copied (and that segment is growing way faster than any other), the price will fall at least to just above the cost, which is low indeed. This is true for services as well, once minds to do the service can be copied.
loup-vaillant 14y ago
The actual storm coming is most probably very different. When we have a general AI, it is likely it will start to optimize the world according to its programming. One such optimization would be to code an even more efficient AI (we assume the AI is a better programmer than its human fathers and mothers). And so on, until FOOM, we have a super-intelligence, capable of taking over computers, convincing humans, build companies, take over means of production, inventing means of productions, and basically take over the world. And of course, it will be unstoppable. Now let's just hope that the original such AI have no bug, especially in its goal system, and let's hope further that it's initial goals are exactly in line with humanity's. We wouldn't want clippy to tile the solar system with paper clips. Or Smiley to do the same with molecular smileys (as a proxy for human happiness). Or Hal9000 to do the same with ultra-efficient computing devices so it can solve the Riemann Hypothesis… Which would have the unfortunate side effect of killing us all. To the extent you don't believe in intelligence explosion, Robin Hanson describe the kind of world we could have. I dare say, it's not pretty.
- randallsquared 14y ago
  I'm more-or-less in agreement with this (Hanson's ems are the kind of mind I was imagining, above), but I was assuming something of a best case, where it turns out that there are hard limits to mindlike complexity. If it turns out that there aren't, none of this will matter. I don't have any particular hope that Eliezer, et al, will construct a bugfree, airtight Greater Wish.
  - loup-vaillant 14y ago
    I'd say hard limit isn't the real criterion for rejecting the Intelligence Explosion hypothesis: there is a hard limit, but most likely well above human level: a human-made substrate could most certainly think way faster than evolution-made neurons, and the software could probably at least get rid of biases. What really matters is whether intelligence is likely to explode or not. I think it would be really foolish to count on it not exploding, unless we're positive it won't. The stakes are too high. As for MIRI (as it is called now) actually pulling it off, especially as they are now, I don't have high hopes either. However, they do look like the current best bet. And they do plan to grow (they need money). And maybe, maybe they will convince the other AI scientists to be wary of new powerful magic. For once. If not them, maybe the Future of Humanity Institute.
bobwaycott 14y ago
> Ultimately the value of a person's labor depends on scarcity, just as the trade value of things or experiences do, and we've seen how lack of scarcity makes trade value crash. Apologies for being pedantic, but you're confusing cost--i.e., market price--with value. These are two different things that are not dependent entirely upon scarcity (I assume you mean supply/demand balances of both commodities and labor itself) alone. There are many factors beyond scarcity that are inputs to determining the value of a commodity (including the commodity of labor) to both society and economy, which may or may not be included in determining the cost of said commodity--and the calculations of both are not guaranteed to be equal. From almost any perspective--economic, historical, philosophical, psychological, etc.--the cost and value of an item are rarely equal. This is especially critical in analyzing and theorizing on trade relations, as the value of a commodity to one agent is often an independent determining factor in calculating the cost of that commodity by another agent.
- randallsquared 14y ago
  Scarcity creates a lower bound to cost. If the lower bound for every good and service drops such that no human can afford to live between material cost and the finished cost, then humans will be out of work except for charity. There might well be boutique human-crafted items, but counting on that for the survival of 7-15 billion humans seems premature. :)
  - bobwaycott 14y ago
    This may well be true, but this is still only a matter of cost, not value. Again, I apologize for being pedantic on the matter, but my point was that the value of a commodity, including labor, is a very different thing from the cost of that commodity.

DanielBMarkham 14y ago

I believe technologists continue to make the mistake of assuming everything is digital. Perhaps it's a part of the job. "comparative advantage depends on scarcity of productive agents" People buy things (trade) based on perceived value. That perception can be based on perceived scarcity, sentiment, anger, fear, love, happiness -- the list is very long. The scarcity argument only holds true across large industries and populations, and that's only because of the fungibility of money and the fact that material goods are (rather) easily categorized. It's not going to continue working. Money will stay fungible, but we're going to see an explosion in the kinds of things that have perceived value that the world has never seen. Which makes sense if you think about it: as mankind has progressed, the diversity of the things he trades has increased. This trend will continue. You guys are confusing theory and reality. Macro-economics fails us here. That's a shame. Looks like some folks have more work to do :)

randallsquared 14y ago
It doesn't matter if everything is digital, if the creators of everything can be. > The scarcity argument only holds true across large industries and populations I don't think this is so, unless you're talking about handcrafted-by-genuine-Ukranian-American underwater-woven baskets. For anything that can be copied (and that segment is growing way faster than any other), the price will fall at least to just above the cost, which is low indeed. This is true for services as well, once minds to do the service can be copied.

loup-vaillant 14y ago

The actual storm coming is most probably very different. When we have a general AI, it is likely it will start to optimize the world according to its programming. One such optimization would be to code an even more efficient AI (we assume the AI is a better programmer than its human fathers and mothers). And so on, until FOOM, we have a super-intelligence, capable of taking over computers, convincing humans, build companies, take over means of production, inventing means of productions, and basically take over the world. And of course, it will be unstoppable. Now let's just hope that the original such AI have no bug, especially in its goal system, and let's hope further that it's initial goals are exactly in line with humanity's. We wouldn't want clippy to tile the solar system with paper clips. Or Smiley to do the same with molecular smileys (as a proxy for human happiness). Or Hal9000 to do the same with ultra-efficient computing devices so it can solve the Riemann Hypothesis… Which would have the unfortunate side effect of killing us all. To the extent you don't believe in intelligence explosion, Robin Hanson describe the kind of world we could have. I dare say, it's not pretty.

randallsquared 14y ago
I'm more-or-less in agreement with this (Hanson's ems are the kind of mind I was imagining, above), but I was assuming something of a best case, where it turns out that there are hard limits to mindlike complexity. If it turns out that there aren't, none of this will matter. I don't have any particular hope that Eliezer, et al, will construct a bugfree, airtight Greater Wish.
- loup-vaillant 14y ago
  I'd say hard limit isn't the real criterion for rejecting the Intelligence Explosion hypothesis: there is a hard limit, but most likely well above human level: a human-made substrate could most certainly think way faster than evolution-made neurons, and the software could probably at least get rid of biases. What really matters is whether intelligence is likely to explode or not. I think it would be really foolish to count on it not exploding, unless we're positive it won't. The stakes are too high. As for MIRI (as it is called now) actually pulling it off, especially as they are now, I don't have high hopes either. However, they do look like the current best bet. And they do plan to grow (they need money). And maybe, maybe they will convince the other AI scientists to be wary of new powerful magic. For once. If not them, maybe the Future of Humanity Institute.

bobwaycott 14y ago

> Ultimately the value of a person's labor depends on scarcity, just as the trade value of things or experiences do, and we've seen how lack of scarcity makes trade value crash. Apologies for being pedantic, but you're confusing cost--i.e., market price--with value. These are two different things that are not dependent entirely upon scarcity (I assume you mean supply/demand balances of both commodities and labor itself) alone. There are many factors beyond scarcity that are inputs to determining the value of a commodity (including the commodity of labor) to both society and economy, which may or may not be included in determining the cost of said commodity--and the calculations of both are not guaranteed to be equal. From almost any perspective--economic, historical, philosophical, psychological, etc.--the cost and value of an item are rarely equal. This is especially critical in analyzing and theorizing on trade relations, as the value of a commodity to one agent is often an independent determining factor in calculating the cost of that commodity by another agent.

randallsquared 14y ago
Scarcity creates a lower bound to cost. If the lower bound for every good and service drops such that no human can afford to live between material cost and the finished cost, then humans will be out of work except for charity. There might well be boutique human-crafted items, but counting on that for the survival of 7-15 billion humans seems premature. :)
- bobwaycott 14y ago
  This may well be true, but this is still only a matter of cost, not value. Again, I apologize for being pedantic on the matter, but my point was that the value of a commodity, including labor, is a very different thing from the cost of that commodity.