4 ms·

AI doesn't have needs any desires, humans do. And no matter how hyped one might be about AI, we're far away from creating an artificial human. As long as that's

by svara 2y ago

AI doesn't have needs any desires, humans do. And no matter how hyped one might be about AI, we're far away from creating an artificial human. As long as that's true, AI is a tool to make humans more effective.

boothby 2y ago
> AI doesn't have needs any desires, humans do. I fear that this won't age well. But to shamelessly riff on Marx, those who control the means of computation will control society.
- visarga 2y ago
  In the AI age, those who own the problems stand to own the AI benefits. Utility is in the application layer, not the hosting or development of AI models.
- svara 2y ago
  I agree the latter part is a risk to consider, but I really think getting an AI to replace human jobs on a vast scale will take much more than just training a bit more. You need to train on a fundamentally different task, which is to be good at the adversarial game of pursuing one's needs and desires in a social environment. And that doesn't yet take into account that the interface to our lives is largely physical, we need bodies. I'm seeing us on track to AGI in the sense of building a universal question answering machine, a system that will be able to answer any unambiguously stated question if given enough time and energy. Stating questions unambiguously gets pretty difficult fast even where it's possible, often it isn't even possible, and getting those answers is just a small part of being a successful human. PS: Needs and desires are totally orthogonal to AI/AGI. Every animal has them, but many animals don't have high intelligence. Needs and desires are a consequence of our evolutionary history, not our intelligence. AGI does not need to mean an artificial human. Whether to pursue or not pursue that research program is up to us, it's not inevitable.
  - the_af 2y ago
    To be clear, I'm not arguing humans will stop being involved in software engineering completely. What I fear is that the pool of employable humans (as code reviewers, prompt engineers and high-level "solution architects") will shrink, because fewer will be needed, and that this will cause ripples in our industry and affect employment. We know this isn't far-fetched. We have strong evidence to suspect during the big layoffs of a couple of years ago, FAANG and startups all colluded to lower engineer salaries across the board, and that their excuse ("the economy is shrinking") was flimsy at best. Now AI presents them with another powerful tool to reduce salaries even more, with a side dish of reducing the size of the cost center that is programmers and engineers.
  - boothby 2y ago
    Honestly, I wasn't even talking about jobs with that. I worry about an intelligent IOT controlled by authoritarian governments or corporate interests. Our phones have already turned society into a panopticon, and that will can get much worse when AGI lands. But yes, the job thing is concerning as well. AI won't scrub a toilet, but it will cheaply and inexhaustibly do every job that humans find meaningful today. It seems that we're heading inexorably towards dystopia.
    - svara 2y ago
      > AI won't scrub a toilet, but it will cheaply and inexhaustibly do every job that humans find meaningful today That's the part I really don't believe. I'm open to being wrong about this, the risk is probably large enough to warrant considering it even if the probability of this happening is low, but I do think it's quite low. We don't actually have to build artificial humans. It's very difficult and very far away. It's a research program that is related to but not identical to the research program leading to tools that have intelligence as a feature. We should be, and in fact we are, building tools. I'm convinced that the mental model many people here and elsewhere are applying is essentially "AGI = artificial human", simply because the human is the only kind of thing in the world that we know that appears to have general intelligence. But that mental model is flawed. We'll be putting intelligence in all sorts of places that are not similar to a human at all, without those devices competing with us at being human.
btilly 2y ago
AI may not have desires, but corporations do. And control more resources than humans. Making corporations more effective is not always in the interest of humans.
- svara 2y ago
  That's fair, but the question was whether AI would destroy or create jobs. You might speculate about a one-person megacorp where everything is done by AIs that a single person runs. What I'm saying is that we're very far from this, because the AI is not a human that can make the CEO's needs and desires their own and execute on them independently. Humans are good at being humans because they've learned to play a complex game, which is to pursue one's needs and desires in a partially adversarial social environment. This is not at all what AI today is being trained for. Maybe a different way to look at it, as a sort of intuition pump: If you were that one man company, and you had an AGI that will correctly answer any unambiguously stated question you could ask, at what point would you need to start hiring?
  - menaerus 2y ago
    You're taking your opinion to extreme because I don't think anyone is talking about replacing all engineers with a single AI computer doing the work for a one-person mega-corporation. The actual question, which is much more realistic, is if an average company of, let'say, 50 engineers will still have a need to hire those 50 engineers if AI turns out to be such an efficiency multiplier? In that case, you will no longer need 10 people to complete 10 tasks in given time-unit but perhaps only 1 engineer + AI compute to do the same. Not all businesses can continue scaling forever, so it's pretty expected that those 9 engineers will become redundant.
    - svara 2y ago
      You took me too literally there, that was intended as a thought experiment to explore the limits. What I was getting at was the question: If we feel intuitively that this extreme isn't realistic, what exactly do we think is missing? My argument is, what's missing is the human ability to play the game of being human, pursuing goals in an adversarial social context. To your point more specifically: Yes, that 10-person team might be replaceable by a single person. More likely than not however, the size of the team was not constrained by lack of ideas or ambition, but by capital and organizational effectiveness. This is how it's played out with every single technology so far that has increased human productivity. They increase demand for labor. Put another way: Businesses in every industry will be able to hire software engineering teams that are so good that in the past, only the big names were able to afford them. The kind of team required for the digital transformation of every old fashioned industry.
strogonoff 2y ago
It should be obvious that technology exists for the sake of humans, not the other way around, but I have already seen an argument for firing humans in favour of LLMs since the latter emit less pollution. LLMs do not have desires, but their existence alters desires of humans, including the ones in charge of businesses.

by svara 2y ago

boothby 2y ago

> AI doesn't have needs any desires, humans do. I fear that this won't age well. But to shamelessly riff on Marx, those who control the means of computation will control society.

visarga 2y ago
In the AI age, those who own the problems stand to own the AI benefits. Utility is in the application layer, not the hosting or development of AI models.
svara 2y ago
I agree the latter part is a risk to consider, but I really think getting an AI to replace human jobs on a vast scale will take much more than just training a bit more. You need to train on a fundamentally different task, which is to be good at the adversarial game of pursuing one's needs and desires in a social environment. And that doesn't yet take into account that the interface to our lives is largely physical, we need bodies. I'm seeing us on track to AGI in the sense of building a universal question answering machine, a system that will be able to answer any unambiguously stated question if given enough time and energy. Stating questions unambiguously gets pretty difficult fast even where it's possible, often it isn't even possible, and getting those answers is just a small part of being a successful human. PS: Needs and desires are totally orthogonal to AI/AGI. Every animal has them, but many animals don't have high intelligence. Needs and desires are a consequence of our evolutionary history, not our intelligence. AGI does not need to mean an artificial human. Whether to pursue or not pursue that research program is up to us, it's not inevitable.
- the_af 2y ago
  To be clear, I'm not arguing humans will stop being involved in software engineering completely. What I fear is that the pool of employable humans (as code reviewers, prompt engineers and high-level "solution architects") will shrink, because fewer will be needed, and that this will cause ripples in our industry and affect employment. We know this isn't far-fetched. We have strong evidence to suspect during the big layoffs of a couple of years ago, FAANG and startups all colluded to lower engineer salaries across the board, and that their excuse ("the economy is shrinking") was flimsy at best. Now AI presents them with another powerful tool to reduce salaries even more, with a side dish of reducing the size of the cost center that is programmers and engineers.
- boothby 2y ago
  Honestly, I wasn't even talking about jobs with that. I worry about an intelligent IOT controlled by authoritarian governments or corporate interests. Our phones have already turned society into a panopticon, and that will can get much worse when AGI lands. But yes, the job thing is concerning as well. AI won't scrub a toilet, but it will cheaply and inexhaustibly do every job that humans find meaningful today. It seems that we're heading inexorably towards dystopia.
  - svara 2y ago
    > AI won't scrub a toilet, but it will cheaply and inexhaustibly do every job that humans find meaningful today That's the part I really don't believe. I'm open to being wrong about this, the risk is probably large enough to warrant considering it even if the probability of this happening is low, but I do think it's quite low. We don't actually have to build artificial humans. It's very difficult and very far away. It's a research program that is related to but not identical to the research program leading to tools that have intelligence as a feature. We should be, and in fact we are, building tools. I'm convinced that the mental model many people here and elsewhere are applying is essentially "AGI = artificial human", simply because the human is the only kind of thing in the world that we know that appears to have general intelligence. But that mental model is flawed. We'll be putting intelligence in all sorts of places that are not similar to a human at all, without those devices competing with us at being human.

btilly 2y ago

AI may not have desires, but corporations do. And control more resources than humans. Making corporations more effective is not always in the interest of humans.

svara 2y ago
That's fair, but the question was whether AI would destroy or create jobs. You might speculate about a one-person megacorp where everything is done by AIs that a single person runs. What I'm saying is that we're very far from this, because the AI is not a human that can make the CEO's needs and desires their own and execute on them independently. Humans are good at being humans because they've learned to play a complex game, which is to pursue one's needs and desires in a partially adversarial social environment. This is not at all what AI today is being trained for. Maybe a different way to look at it, as a sort of intuition pump: If you were that one man company, and you had an AGI that will correctly answer any unambiguously stated question you could ask, at what point would you need to start hiring?
- menaerus 2y ago
  You're taking your opinion to extreme because I don't think anyone is talking about replacing all engineers with a single AI computer doing the work for a one-person mega-corporation. The actual question, which is much more realistic, is if an average company of, let'say, 50 engineers will still have a need to hire those 50 engineers if AI turns out to be such an efficiency multiplier? In that case, you will no longer need 10 people to complete 10 tasks in given time-unit but perhaps only 1 engineer + AI compute to do the same. Not all businesses can continue scaling forever, so it's pretty expected that those 9 engineers will become redundant.
  - svara 2y ago
    You took me too literally there, that was intended as a thought experiment to explore the limits. What I was getting at was the question: If we feel intuitively that this extreme isn't realistic, what exactly do we think is missing? My argument is, what's missing is the human ability to play the game of being human, pursuing goals in an adversarial social context. To your point more specifically: Yes, that 10-person team might be replaceable by a single person. More likely than not however, the size of the team was not constrained by lack of ideas or ambition, but by capital and organizational effectiveness. This is how it's played out with every single technology so far that has increased human productivity. They increase demand for labor. Put another way: Businesses in every industry will be able to hire software engineering teams that are so good that in the past, only the big names were able to afford them. The kind of team required for the digital transformation of every old fashioned industry.

strogonoff 2y ago

It should be obvious that technology exists for the sake of humans, not the other way around, but I have already seen an argument for firing humans in favour of LLMs since the latter emit less pollution. LLMs do not have desires, but their existence alters desires of humans, including the ones in charge of businesses.