4 ms·
> Companies have a much stronger incentive to make AGI than deadly weapons. It will be incredibly profitable. Are you sure? Various countries are throwing hund
by scotty79 4y ago
> Companies have a much stronger incentive to make AGI than deadly weapons. It will be incredibly profitable.
Are you sure? Various countries are throwing hundreds of billions of dollars on the table of weapons manufacturers. I think this market grew recently way faster than AI.
> We have zero historical examples of a species developing a technology that is smarter than it is to look back at and establish some sort of probability for how this will go.
We already have so many superhuman technologies under our belt that allow organizations, which I could argue are a form of soulless AIs to compete with interests of whole societies. I don't think neural AI will come to beat our organizations at their game just because if you pour into it significant fraction of our total computing resources might be slightly smarter than a single human. Our bureaucracies surpassed a single human with their cunning, and self-preservation and crushing power centuries ago. We already live in the world made by AIs for AIs. They are just made of humans not silicon.
> To say that "producing words got a bit cheaper" really trivializes what a massive impact AGI will have, and how soon it's coming.
I am not convinced it will be more unique than internet or printing press. I believe AIs will pressure us into changing some things just as any relevant technology have done before. Internet came with its own set of SF scares of hackers taking over entire world or brains fried by man machine interfaces that were supposed to be just around the corner. We lived through so many technological revolutions that impacted our way of life and work. I don't think AIs or even AGIs developed from what we have now will be much different. I think the example of self-driving cars shows us that pace of progress going forward will be more disappointing than anything else.
Another thing is that people vastly overvalue intelligence. Tiger might be less intelligent than human but it will still tear a single unaided human to shreds. People concerned about AIs should focus less on how intelligent it is but instead of how large physical claws are we willing to equip it with.
- AnimalMuppet 4y ago> People concerned about AIs should focus less on how intelligent it is but instead of how large physical claws are we willing to equip it with. This. But also, if it were smarter than us (or even if people believed that!), then it could program people to be its claws. I don't want to give it physical control of weapons, or even the means of production, but I also don't want it to manipulate peoples' minds.