4 ms·

I just found the words to express that people have really conflated AI with the singularity. I agree with you that AI is here. And that doesn't mean Skynet. It

I just found the words to express that people have really conflated AI with the singularity. I agree with you that AI is here. And that doesn't mean Skynet. It means that you can converse with a reasonably intelligent partner who adds to the shared pool of ideas. Just like another person does. People think when AI gets here, and if it's even a little smarter than humans, it will compound over time, make itself smarter, and rule over us all. That ignores that humanity is made up of intelligent people, some of whom are smarter than others, and even organizations (and technologies, like books) that allow our ideas to persist past our individual lifetimes. And yet, no organization has come to dominate the world. Why would an AI?

concordDance 3y ago
The smarter things have indeed dominated the world. Its not like chimps have much of a say in what direction the world goes in. The same is actually true of groups of people, find me two groups of people of similar size where one is on average smarter than the other and I'd expect the smarter one to have far more dominance over the world.
ben_w 3y ago
> And yet, no organization has come to dominate the world. Why would an AI? Depends how smart, how many instances are running, and if those instances are aligned with each other. And if you count the British empire as world domination. (It wasn't in charge everywhere, but it was dominant; two AI empires fighting each other in the same way the British Empire fought its peers still isn't good for the rest of us). A fully general IQ 100 AI on commodity hardware radically changes the world economy, and yet isn't likely to make many breakthroughs. An army of robots guided by an equal number of such minds could well be on par with a human army of similar number and let a dictator fight a brutal war — even a smaller army in the form of assassin bots (which don't have to be humanoid, you can have an explosive kamikaze pigeon) is a radical change to combat on both a nation state level and also for non-governmental actors (organised crime, terrorism, vigilantes, paramilitaries, police, private security). But there are also potential surprise failures modes. A group of replicated minds, no matter how smart, are going to have correlated failures. Figure out how to make one of those soldiers surrender, you know how to make all of them surrender. Make them smarter and have them do R&D or science, instead of bouncing ideas off each other, they all get stuck at the same place. You'll speed up research, but it won't look like an unbounded number of skilled researchers, it will look like one researcher with unbounded time. (Not that I'm expecting IQ to generalise from humans to AI. Hasn't done so far, after all). Also, an AI with no qualia which is really good at only one thing might become dominant just because what it does plays into a prisoners' dilemma where humans defect often enough ("enough" being a standard which depends on the details, e.g. we don't need many humans choosing to release CFCs to cause environmental issues).
arketyp 3y ago
In a way our societies are already super intelligences, honed by natural selection and with their own incentives working on top of, and not necessarily aligned with, the desires of each of its constituent human apes. AI will just seamlessly blend into that, I think.
- rbanffy 3y ago
  Their goals are rather simple. While they are immensely powerful, they aren't much smarter than a fungus.
  - deleted 3y ago
    [deleted]
KineticLensman 3y ago
> And yet, no organization has come to dominate the world. Why would an AI? I think the general point is valid (that AI is an intelligent partner rather than Skynet) but organisations do dominate the world, especially the parts of it (including older organisations) that don't employ smart people or newer technologies. There is a view that organisations are a form of slow AI executing complex business / legal processes (traditionally based on people power).
- sshine 3y ago
  > There is a view that organisations are a form of slow AI executing complex business Multi-national companies do act a lot like hiveminds. There are individual human beings directing at the top, but only because they comply with the hive's interests. Boards act as an averaging mechanism that seeks to maximize self-preservation and growth. The anti-capitalist incentive to not understand why a company would want to grow beyond its positive impact on society think that these companies exist to benefit society. But once something has got big enough to become self-sustaining, and not depending on individual patches of land, it's only natural to see this pattern repeat. I don't really see what's artificial about it, though. Thousands of human beings clicking away on interconnected computers just seems pretty intelligent. Adding LLMs to the mix seems like a way to boost the people who already have a function in an organisation without having to scale with people; for any task where LLMs can make a person better, your organisation will grow in proportion.
  - rbanffy 3y ago
    > Multi-national companies do act a lot like hiveminds. Corporations are AIs. They are just much slower at making decisions than single humans. They also limit competition against themselves by consuming human time. We are at the stage where we give an AI a limited, carefully described, goal and it fulfills it based on patterns it learned from human codified knowledge. At some point, not that far from now, there will be AIs that are capable of selecting goals based on more generic directives and optimize from there better than any human could do. At that point, where motivations become incomprehensible to humans, we can say we have something like a superintelligence. It'll be interesting.
tnecniv 3y ago
It feels pretty close to the ship computer in original Star Trek. It wasn’t really an AI in that it was sentient, but a verbal natural language interface to a computer capable of running all sorts of analyses and ship operations. If it ran into something outside of its knowledge base it’d just say I don’t know and shrug.
- rbanffy 3y ago
  > If it ran into something outside of its knowledge base it’d just say I don’t know and shrug. This is something LLMs are yet to master.
mark_l_watson 3y ago
+1 well said

concordDance 3y ago

The smarter things have indeed dominated the world. Its not like chimps have much of a say in what direction the world goes in. The same is actually true of groups of people, find me two groups of people of similar size where one is on average smarter than the other and I'd expect the smarter one to have far more dominance over the world.

ben_w 3y ago

> And yet, no organization has come to dominate the world. Why would an AI? Depends how smart, how many instances are running, and if those instances are aligned with each other. And if you count the British empire as world domination. (It wasn't in charge everywhere, but it was dominant; two AI empires fighting each other in the same way the British Empire fought its peers still isn't good for the rest of us). A fully general IQ 100 AI on commodity hardware radically changes the world economy, and yet isn't likely to make many breakthroughs. An army of robots guided by an equal number of such minds could well be on par with a human army of similar number and let a dictator fight a brutal war — even a smaller army in the form of assassin bots (which don't have to be humanoid, you can have an explosive kamikaze pigeon) is a radical change to combat on both a nation state level and also for non-governmental actors (organised crime, terrorism, vigilantes, paramilitaries, police, private security). But there are also potential surprise failures modes. A group of replicated minds, no matter how smart, are going to have correlated failures. Figure out how to make one of those soldiers surrender, you know how to make all of them surrender. Make them smarter and have them do R&D or science, instead of bouncing ideas off each other, they all get stuck at the same place. You'll speed up research, but it won't look like an unbounded number of skilled researchers, it will look like one researcher with unbounded time. (Not that I'm expecting IQ to generalise from humans to AI. Hasn't done so far, after all). Also, an AI with no qualia which is really good at only one thing might become dominant just because what it does plays into a prisoners' dilemma where humans defect often enough ("enough" being a standard which depends on the details, e.g. we don't need many humans choosing to release CFCs to cause environmental issues).

arketyp 3y ago

In a way our societies are already super intelligences, honed by natural selection and with their own incentives working on top of, and not necessarily aligned with, the desires of each of its constituent human apes. AI will just seamlessly blend into that, I think.

rbanffy 3y ago
Their goals are rather simple. While they are immensely powerful, they aren't much smarter than a fungus.
- deleted 3y ago
  [deleted]

KineticLensman 3y ago

> And yet, no organization has come to dominate the world. Why would an AI? I think the general point is valid (that AI is an intelligent partner rather than Skynet) but organisations do dominate the world, especially the parts of it (including older organisations) that don't employ smart people or newer technologies. There is a view that organisations are a form of slow AI executing complex business / legal processes (traditionally based on people power).

sshine 3y ago
> There is a view that organisations are a form of slow AI executing complex business Multi-national companies do act a lot like hiveminds. There are individual human beings directing at the top, but only because they comply with the hive's interests. Boards act as an averaging mechanism that seeks to maximize self-preservation and growth. The anti-capitalist incentive to not understand why a company would want to grow beyond its positive impact on society think that these companies exist to benefit society. But once something has got big enough to become self-sustaining, and not depending on individual patches of land, it's only natural to see this pattern repeat. I don't really see what's artificial about it, though. Thousands of human beings clicking away on interconnected computers just seems pretty intelligent. Adding LLMs to the mix seems like a way to boost the people who already have a function in an organisation without having to scale with people; for any task where LLMs can make a person better, your organisation will grow in proportion.
- rbanffy 3y ago
  > Multi-national companies do act a lot like hiveminds. Corporations are AIs. They are just much slower at making decisions than single humans. They also limit competition against themselves by consuming human time. We are at the stage where we give an AI a limited, carefully described, goal and it fulfills it based on patterns it learned from human codified knowledge. At some point, not that far from now, there will be AIs that are capable of selecting goals based on more generic directives and optimize from there better than any human could do. At that point, where motivations become incomprehensible to humans, we can say we have something like a superintelligence. It'll be interesting.

tnecniv 3y ago

It feels pretty close to the ship computer in original Star Trek. It wasn’t really an AI in that it was sentient, but a verbal natural language interface to a computer capable of running all sorts of analyses and ship operations. If it ran into something outside of its knowledge base it’d just say I don’t know and shrug.

rbanffy 3y ago
> If it ran into something outside of its knowledge base it’d just say I don’t know and shrug. This is something LLMs are yet to master.

mark_l_watson 3y ago

+1 well said