8 ms·

> I kind of flipped out, and was motivated to finish this blog piece, when I saw this: “UK government wants to use AI to cut civil service jobs: Yes, you read t

> I kind of flipped out, and was motivated to finish this blog piece, when I saw this: “UK government wants to use AI to cut civil service jobs: Yes, you read that right.” The idea — to have citizen input processed and responded to by an LLM — is hideously toxic and broken; and usefully reveals the kind of thinking that makes morally crippled leaders all across our system love this technology. And this will not just be in government, it will be everywhere. The scariest part is that as people start to spend less time developing a skill set, and instead deferring to AI answers, you will cross a point where this problem can't be fixed (because nobody has the skills to fix it and the AI is trained on the outputs of previous generations of humans). For the "olds" who already have a skillset, this will be incredibly lucrative (as those who can afford to pay to fix it will handsomely). But the potential for this to—at best—plateau humanity and at worst, make it regress, is significant. The dark humor in all this: we thought AI would get us the Terminator, but instead it's going to get us rapid degeneration. --- Edit: an addendum, the overall point I'm making is well encapsulated in this talk https://www.youtube.com/watch?v=ZSRHeXYDLko https://www.youtube.com/watch?v=ZSRHeXYDLko

ildjarn 3y ago
The future was predicted in The Machine Stops and Mockingbird
- dgfitz 3y ago
  I'd argue more of a mix of Brave New World, 1984, and Atlas Shrugged
mritchie712 3y ago
that'd be WALL-E
- temp0826 3y ago
  I was thinking Idiocracy. Time to use AI to generate enough blogspam about the benefits of watering plants with gatorade to "poison the well" of datasets used for future training.
  - cjbgkagh 3y ago
    It’s your well? That sounds like it’ll just result in Gatorade replacing water for crops even sooner.
  - mritchie712 3y ago
    I like to think we'd snap out of it eventually (like in WALL-E)
bluedino 3y ago
On the other hand, in my country we have the stereotypical lazy government employee who is overpaid and doesn't work hard. The government could then save money and provide better service for menial tasks such as "what permit do I need to do such and such"
- candiddevmike 3y ago
  Isn't that the dream though? To be overpaid and not have to work hard?
  - doctor_eval 3y ago
    Yes, but that’s for us tech workers, not the common prole. /s
  - stavros 3y ago
    Not if you're the employer, which, if you're the taxpayer, you are.
    - throwanem 3y ago
      I think you meant 'employer'.
      - stavros 3y ago
        I did, thank you!
        throwanem 3y ago
        Sure thing. You're still wrong on the merits, though. To pick one example, the folks who work at my local MVA ("DMV" in most states) office do not work for me, though they're paid out of my taxes and those of everyone else who earns in Maryland. They don't report to me. Nor should they, because if they did, they would also report to that freak who drives a car plastered with QAnon garbage around Perry Hall. My city councilman and my General Assembly representatives work for me, but the people employed to deliver services managed by the state and city government do not.
willsmith72 3y ago
Doesn't that assume that people will forever be better learners than AI? If the "olds" learnt a skillset at some point, the data they used to learn the skill is presumably available to the AI too. Why can't the AI learn it too? (Not talking about physical labour which clearly has way less potential to be replaced than knowledge work)
- rglover 3y ago
  > Doesn't that assume that people will forever be better learners than AI? Better creators, not learners. AI can't create, it can only remix what's already been produced by humans. Human progress is created, not learned. The olds who are conditioned to try new things when an existing solution doesn't work still have the capacity to create something new (wholly new, not just remixed new).
  - overflow897 3y ago
    AlphaGo and AlphaStar both started out based on human training and then played against versions of themselves to go on and create new strategies in their games. Modern LLMs can't learn/experiment as far as I know in exactly the same way but that may not always be true.
    - rglover 3y ago
      Yeah, but they had a limited set of rules to work within (they were just hyper-efficient at calculating the possible outcomes relative to those rules). Humans, in theory, only have the rules they believe as there technically are no rules (it's all make-believe). For example, what was the "rule" that told people to make a wheel? There wasn't one. The human had to think about it/conceive it, which AI can't (and I'd argue never will be able to) without rules.
    - imtringued 3y ago
      Reinforcement learning is a completely different strategy compared to how most LLMs work.
MyFirstSass 3y ago
Christ this is dark! Imagine a world where everything and everyone will be judged, ranked, evaluated, hired, fired, maybe even as a partner or friend by these AI's. Unfathomably grim even if the alternative is rigid low skill bureaucrats. I find LLM's extremely fascinating but if this is the end game i really hope AI free zones will emerge. You can already see Gen Z being obsessed with face ranking filters, "looksmaxing" from data points and using filters day to day. It's dark.
- cjbgkagh 3y ago
  I think these gig economy workers and algorithmic punishments already have an element of that. It’s entirely possible to do a good job of it but it appears that no one chooses to do so - I presume the same would be true for LLM based processes.
PheonixPharts 3y ago
Anyone who works in AI, especially very closely with models, will tell you that it's not capable of really replacing any jobs yet. All these jobs being "replaced by AI" are simply being eliminated with the consequences of them being eliminated ignored. Customer service jobs aren't being replaced by AI, companies, like Klarna, are just giving up on customer service and using AI to increase their perceived value rather than reducing it.
- rblatz 3y ago
  It's not an all or nothing situation you can put an AI agent in front of real agents, if the AI agent is able to answer the customers question you do not need to escalate to a real human. That could allow 10 humans to now do the job of say 20. Did AI replace those 10 jobs, I'd argue yes.
  - roland35 3y ago
    Yes, exactly like how a sewing machine helped replace sewers. One person can do the job of 2.
  - randomdata 3y ago
    > Did AI replace those 10 jobs, I'd argue yes. Did it, or is it like the automated elevator? The automated elevator didn't reduce the number of elevator operators, it substantially increased the number of elevator operators.
- ilamont 3y ago
  Amazon and others already claim they use AI to handle customer support and marketing copy (https://www.aboutamazon.com/news/innovation-at-amazon/how-amazon-uses-generative-ai https://www.aboutamazon.com/news/innovation-at-amazon/how-am...). Air Canada got burned when an AI chatbot made a mistake (https://news.ycombinator.com/item?id=39378235 https://news.ycombinator.com/item?id=39378235). Legal firms have been fined for hallucinated precedent citations (https://news.ycombinator.com/item?id=39491510 https://news.ycombinator.com/item?id=39491510). The technology is clearly flawed. Regardless, a lower cost option (AI) is replacing a higher priced option (human labor). Ten years ago these tasks would have been handled by human employees or specialists. There is certainly a history of this in the corporate world, when expensive union labor in Detroit is replaced by lower cost workers in Mississippi, manufacturing experts are forced to train their replacements in lower-cost countries, or entire engineering and customer service departments are shifted overseas.
__loam 3y ago
I'm really worried about the implications of this technology for programming, art, and literacy in general. The skills required to be a good programmer or artist are not what's being sampled by these models, merely the output. There's a real danger here of losing these skills if they can no longer be developed professionally, and that even means no more human training data for newer and better models to be trained on. We'd be stuck with whatever trash current models are putting out.
- cjbgkagh 3y ago
  The competency pipeline issue predates the threat of AI, I do agree that AI will make it much worse as it’s hard to see a payoff on investment into skills of AI is going to replace you anyway. I feel like a wise old sage knowing things that new generations will never learn and may even be lost to humanity.
- netsharc 3y ago
  As a programmer, I hope programming remains "sacred", but I can see the flip side: It's a specific way of making a machine do what you want, repeatedly. I can imagine the AI-fuelled world just entering the machine's capabilities/parameters, and asking an LLM in human language to generate code for it. Sure it might not do it in the most elegant or efficient way, but many programmers also use npm.
  - pg_1234 3y ago
    In this AI-fuelled world where everything is run by 2nd rate AI-generated code, talented programmers will be able to hack the whole world. Forget working for anyone, just make the machines do what you want.
happytiger 3y ago
lol. This is how you end up with that scene in idiocracy where you find out you get your college degrees at Costco.
lp4vn 3y ago
There is something very wrong with the current dynamic of the world. Work is deeply devalued while capital has been reaping all the benefits of the increased productivity: if you doubt me, ask anyone who has made any kind of investment if they are making more money from the investment or from their regular job. This is creating a ridiculous wealth disparity and deincentivizing a whole generation to get good with a skillset. I already heard from a lot of young people that working is not worth it, hard to disagree with them when even a basic thing like a piece of land or a house looks out of reach for a regular person. But as you put it unsustainable things are not sustainable, society will regress until the equilibrium is found again. But things didn't need to be like this.
- rglover 3y ago
  You're right, they didn't. > “We can say without exaggeration that the present national ambition of the United States is unemployment. People live for quitting time, for weekends, for vacations, and for retirement; moreover, this ambition seems to be classless, as true in the executive suites as on the assembly lines. One works not because the work is necessary, valuable, useful to a desirable end, or because one loves to do it, but only to be able to quit - a condition that a saner time would regard as infernal, a condemnation.” > - Wendell Berry
  - happyjack 3y ago
    This is so spot on. We (assuming you live in the US) live in a country where the purpose is to work for as little as possible, "grind," then throw money into an index fund and have it support us the rest of our lives. Own property, while having the bottom 50% of society continually support us in the gig and service economy. Look, I'm no Bernie Sanders, but you have to be honest about the morality of it, and the feasibility of it. I don't see the current system lasting.
- agumonkey 3y ago
  technological progress to that extent is touching ethical issues.. what are we supposed to do if our existence don't depend on us learning / collaborating ?
golergka 3y ago
I kind of flipped out, and was motivated to finish this blog piece, when I saw this: “UK government wants to use computers to cut civil service jobs: Yes, you read that right.” The idea — to have citizen data might be put into and processed by a computer — is hideously toxic and broken; and usefully reveals the kind of thinking that makes morally crippled leaders all across our system love this technology.
carlosjobim 3y ago
Dealing with an AI will be much better than dealing with a civil servant, in most cases. There is a certain kind of person who becomes a civil servant and many of them will not only have a hostile attitude, but also make it their life's mission to try to do as much damage as possible in the pettiest ways possible to the people they "serve". Especially if you're of a sex, ethnicity or age group that they hate. Sometimes the same as theirs. Letting citizens deal with their bureaucratic errands with an online form or portal instead of with a civil servant in their office has been an enormous benefit, in the places that offer this. An AI will fuck things up, being an AI, but it will not necessarily treat people with a hostile attitude and lie to clients to spite them. Unless it's programmed by civil servants, that is.
- rglover 3y ago
  > Especially if you're of a sex, ethnicity or age group that they hate. Sometimes the same as theirs. Well, Gemini just proved that if you're white, you're the ethnicity the AI is told to hate. That's the thing, because it can be programmed by humans means at some point, it will be abused to do something nefarious. And because it only knows what humans tell it about reality, it will always "think" within the context it's been given (never in the abstract).
caseysoftware 3y ago
> The scariest part is that as people start to spend less time developing a skill set, and instead deferring to AI answers, you will cross a point where this problem can't be fixed (because nobody has the skills to fix it and the AI is trained on the outputs of previous generations of humans). The loss of knowledge/skills was a key bit of Foundation which itself was a retelling of the fall of the Roman Empire. As key skills become rarer, the price goes up.. until you can't hire for those skills at any price.
- userulluipeste 3y ago
  But... if the price goes up, isn't this going to attract people to that domain of skills? For skills to vanish, there has to be more factors at play, like restrictions to dissemination of knowledge, no?
ben_w 3y ago
> And this will not just be in government, it will be everywhere. The scariest part is that as people start to spend less time developing a skill set, and instead deferring to AI answers, you will cross a point where this problem can't be fixed (because nobody has the skills to fix it and the AI is trained on the outputs of previous generations of humans). I think that would require AI development to approximately halt at close to the current level for over a lifetime. Conditional on development halting, I'd agree with you. By analogy, there's this single, very useful, very powerful, set of "hidden methods that can be used to win all games, get rich, find love, determine the limits of thought itself!" — mathematics[0]. Do people like learning it? They do not. Calculator much easier. What a calculator does is none of that, calculators are merely arithmetic, but most people can't tell the difference between mathematics and arithmetic. I think LLMs have the same effect on anything that can be expressed in words, and all the various image generator models have this effect on graphical arts. One must be extremely motivated to get past the "but the computer is better than me" hump. However, I don't expect AI development to even approximately halt at anything close to the current level. There's a lot of room for self-play in domains like maths and computing where the proofs can be verified, and probably a lot of room for anything that can be RLHF'd, too. And that's also assuming we don't get any brain uploads; regardless of the question of "is such an upload of a human capable of consciousness", which absolutely matters, it may still be relevant to the economic issues of AI depending on the cost of running one depending on all the details of such an upload that I can't even begin to guess at at this point (last I heard, https://openworm.org https://openworm.org was not actually measuring synaptic weights directly, but rather neural activity? I may be out of date, not my field). Whatever happens, however good it does or doesn't get, I do expect something to go very weird before I reach the current state pension age — close enough that, if that something is "the machines break" or "society breaks", then there will still be plenty who remember the before times. [0] https://www.smbc-comics.com/comic/secrets-2 https://www.smbc-comics.com/comic/secrets-2
- rglover 3y ago
  > I think that would require AI development to approximately halt at close to the current level for over a lifetime. What I'm getting at isn't AI development halting, but human knowledge/creativity halting [1]. Because the AI is and can only be trained on human knowledge, it's knowledge of reality has an upper bound (whereas, theoretically, humans can know anything or make new discoveries that don't exist in our current knowledge set). If you don't tell the AI that strawberries are a thing/reality, it will never conceive of a strawberry on its own. And arguably, it's not fair to call these things "AI" until they can do so. [1] Charlie Munger said "show me the incentives and I'll show you the outcome." Well, in this case, the incentives to use AI > spending the time to learn and develop skills. The outcome here is clear: humans will stop producing new knowledge and by extension, the AI will stop receiving new knowledge to learn.

ildjarn 3y ago

The future was predicted in The Machine Stops and Mockingbird

dgfitz 3y ago
I'd argue more of a mix of Brave New World, 1984, and Atlas Shrugged

mritchie712 3y ago

that'd be WALL-E

temp0826 3y ago
I was thinking Idiocracy. Time to use AI to generate enough blogspam about the benefits of watering plants with gatorade to "poison the well" of datasets used for future training.
- cjbgkagh 3y ago
  It’s your well? That sounds like it’ll just result in Gatorade replacing water for crops even sooner.
- mritchie712 3y ago
  I like to think we'd snap out of it eventually (like in WALL-E)

bluedino 3y ago

On the other hand, in my country we have the stereotypical lazy government employee who is overpaid and doesn't work hard. The government could then save money and provide better service for menial tasks such as "what permit do I need to do such and such"

candiddevmike 3y ago
Isn't that the dream though? To be overpaid and not have to work hard?
- doctor_eval 3y ago
  Yes, but that’s for us tech workers, not the common prole. /s
- stavros 3y ago
  Not if you're the employer, which, if you're the taxpayer, you are.
  - throwanem 3y ago
    I think you meant 'employer'.
    - stavros 3y ago
      I did, thank you!
      - throwanem 3y ago
        Sure thing. You're still wrong on the merits, though. To pick one example, the folks who work at my local MVA ("DMV" in most states) office do not work for me, though they're paid out of my taxes and those of everyone else who earns in Maryland. They don't report to me. Nor should they, because if they did, they would also report to that freak who drives a car plastered with QAnon garbage around Perry Hall. My city councilman and my General Assembly representatives work for me, but the people employed to deliver services managed by the state and city government do not.

willsmith72 3y ago

Doesn't that assume that people will forever be better learners than AI? If the "olds" learnt a skillset at some point, the data they used to learn the skill is presumably available to the AI too. Why can't the AI learn it too? (Not talking about physical labour which clearly has way less potential to be replaced than knowledge work)

rglover 3y ago
> Doesn't that assume that people will forever be better learners than AI? Better creators, not learners. AI can't create, it can only remix what's already been produced by humans. Human progress is created, not learned. The olds who are conditioned to try new things when an existing solution doesn't work still have the capacity to create something new (wholly new, not just remixed new).
- overflow897 3y ago
  AlphaGo and AlphaStar both started out based on human training and then played against versions of themselves to go on and create new strategies in their games. Modern LLMs can't learn/experiment as far as I know in exactly the same way but that may not always be true.
  - rglover 3y ago
    Yeah, but they had a limited set of rules to work within (they were just hyper-efficient at calculating the possible outcomes relative to those rules). Humans, in theory, only have the rules they believe as there technically are no rules (it's all make-believe). For example, what was the "rule" that told people to make a wheel? There wasn't one. The human had to think about it/conceive it, which AI can't (and I'd argue never will be able to) without rules.
  - imtringued 3y ago
    Reinforcement learning is a completely different strategy compared to how most LLMs work.

MyFirstSass 3y ago

Christ this is dark! Imagine a world where everything and everyone will be judged, ranked, evaluated, hired, fired, maybe even as a partner or friend by these AI's. Unfathomably grim even if the alternative is rigid low skill bureaucrats. I find LLM's extremely fascinating but if this is the end game i really hope AI free zones will emerge. You can already see Gen Z being obsessed with face ranking filters, "looksmaxing" from data points and using filters day to day. It's dark.

cjbgkagh 3y ago
I think these gig economy workers and algorithmic punishments already have an element of that. It’s entirely possible to do a good job of it but it appears that no one chooses to do so - I presume the same would be true for LLM based processes.

PheonixPharts 3y ago

Anyone who works in AI, especially very closely with models, will tell you that it's not capable of really replacing any jobs yet. All these jobs being "replaced by AI" are simply being eliminated with the consequences of them being eliminated ignored. Customer service jobs aren't being replaced by AI, companies, like Klarna, are just giving up on customer service and using AI to increase their perceived value rather than reducing it.

rblatz 3y ago
It's not an all or nothing situation you can put an AI agent in front of real agents, if the AI agent is able to answer the customers question you do not need to escalate to a real human. That could allow 10 humans to now do the job of say 20. Did AI replace those 10 jobs, I'd argue yes.
- roland35 3y ago
  Yes, exactly like how a sewing machine helped replace sewers. One person can do the job of 2.
- randomdata 3y ago
  > Did AI replace those 10 jobs, I'd argue yes. Did it, or is it like the automated elevator? The automated elevator didn't reduce the number of elevator operators, it substantially increased the number of elevator operators.
ilamont 3y ago
Amazon and others already claim they use AI to handle customer support and marketing copy (https://www.aboutamazon.com/news/innovation-at-amazon/how-amazon-uses-generative-ai https://www.aboutamazon.com/news/innovation-at-amazon/how-am...). Air Canada got burned when an AI chatbot made a mistake (https://news.ycombinator.com/item?id=39378235 https://news.ycombinator.com/item?id=39378235). Legal firms have been fined for hallucinated precedent citations (https://news.ycombinator.com/item?id=39491510 https://news.ycombinator.com/item?id=39491510). The technology is clearly flawed. Regardless, a lower cost option (AI) is replacing a higher priced option (human labor). Ten years ago these tasks would have been handled by human employees or specialists. There is certainly a history of this in the corporate world, when expensive union labor in Detroit is replaced by lower cost workers in Mississippi, manufacturing experts are forced to train their replacements in lower-cost countries, or entire engineering and customer service departments are shifted overseas.

__loam 3y ago

I'm really worried about the implications of this technology for programming, art, and literacy in general. The skills required to be a good programmer or artist are not what's being sampled by these models, merely the output. There's a real danger here of losing these skills if they can no longer be developed professionally, and that even means no more human training data for newer and better models to be trained on. We'd be stuck with whatever trash current models are putting out.

cjbgkagh 3y ago
The competency pipeline issue predates the threat of AI, I do agree that AI will make it much worse as it’s hard to see a payoff on investment into skills of AI is going to replace you anyway. I feel like a wise old sage knowing things that new generations will never learn and may even be lost to humanity.
netsharc 3y ago
As a programmer, I hope programming remains "sacred", but I can see the flip side: It's a specific way of making a machine do what you want, repeatedly. I can imagine the AI-fuelled world just entering the machine's capabilities/parameters, and asking an LLM in human language to generate code for it. Sure it might not do it in the most elegant or efficient way, but many programmers also use npm.
- pg_1234 3y ago
  In this AI-fuelled world where everything is run by 2nd rate AI-generated code, talented programmers will be able to hack the whole world. Forget working for anyone, just make the machines do what you want.

happytiger 3y ago

lol. This is how you end up with that scene in idiocracy where you find out you get your college degrees at Costco.

lp4vn 3y ago

There is something very wrong with the current dynamic of the world. Work is deeply devalued while capital has been reaping all the benefits of the increased productivity: if you doubt me, ask anyone who has made any kind of investment if they are making more money from the investment or from their regular job. This is creating a ridiculous wealth disparity and deincentivizing a whole generation to get good with a skillset. I already heard from a lot of young people that working is not worth it, hard to disagree with them when even a basic thing like a piece of land or a house looks out of reach for a regular person. But as you put it unsustainable things are not sustainable, society will regress until the equilibrium is found again. But things didn't need to be like this.

rglover 3y ago
You're right, they didn't. > “We can say without exaggeration that the present national ambition of the United States is unemployment. People live for quitting time, for weekends, for vacations, and for retirement; moreover, this ambition seems to be classless, as true in the executive suites as on the assembly lines. One works not because the work is necessary, valuable, useful to a desirable end, or because one loves to do it, but only to be able to quit - a condition that a saner time would regard as infernal, a condemnation.” > - Wendell Berry
- happyjack 3y ago
  This is so spot on. We (assuming you live in the US) live in a country where the purpose is to work for as little as possible, "grind," then throw money into an index fund and have it support us the rest of our lives. Own property, while having the bottom 50% of society continually support us in the gig and service economy. Look, I'm no Bernie Sanders, but you have to be honest about the morality of it, and the feasibility of it. I don't see the current system lasting.
agumonkey 3y ago
technological progress to that extent is touching ethical issues.. what are we supposed to do if our existence don't depend on us learning / collaborating ?

golergka 3y ago

I kind of flipped out, and was motivated to finish this blog piece, when I saw this: “UK government wants to use computers to cut civil service jobs: Yes, you read that right.” The idea — to have citizen data might be put into and processed by a computer — is hideously toxic and broken; and usefully reveals the kind of thinking that makes morally crippled leaders all across our system love this technology.

carlosjobim 3y ago

Dealing with an AI will be much better than dealing with a civil servant, in most cases. There is a certain kind of person who becomes a civil servant and many of them will not only have a hostile attitude, but also make it their life's mission to try to do as much damage as possible in the pettiest ways possible to the people they "serve". Especially if you're of a sex, ethnicity or age group that they hate. Sometimes the same as theirs. Letting citizens deal with their bureaucratic errands with an online form or portal instead of with a civil servant in their office has been an enormous benefit, in the places that offer this. An AI will fuck things up, being an AI, but it will not necessarily treat people with a hostile attitude and lie to clients to spite them. Unless it's programmed by civil servants, that is.

rglover 3y ago
> Especially if you're of a sex, ethnicity or age group that they hate. Sometimes the same as theirs. Well, Gemini just proved that if you're white, you're the ethnicity the AI is told to hate. That's the thing, because it can be programmed by humans means at some point, it will be abused to do something nefarious. And because it only knows what humans tell it about reality, it will always "think" within the context it's been given (never in the abstract).

caseysoftware 3y ago

> The scariest part is that as people start to spend less time developing a skill set, and instead deferring to AI answers, you will cross a point where this problem can't be fixed (because nobody has the skills to fix it and the AI is trained on the outputs of previous generations of humans). The loss of knowledge/skills was a key bit of Foundation which itself was a retelling of the fall of the Roman Empire. As key skills become rarer, the price goes up.. until you can't hire for those skills at any price.

userulluipeste 3y ago
But... if the price goes up, isn't this going to attract people to that domain of skills? For skills to vanish, there has to be more factors at play, like restrictions to dissemination of knowledge, no?

ben_w 3y ago

> And this will not just be in government, it will be everywhere. The scariest part is that as people start to spend less time developing a skill set, and instead deferring to AI answers, you will cross a point where this problem can't be fixed (because nobody has the skills to fix it and the AI is trained on the outputs of previous generations of humans). I think that would require AI development to approximately halt at close to the current level for over a lifetime. Conditional on development halting, I'd agree with you. By analogy, there's this single, very useful, very powerful, set of "hidden methods that can be used to win all games, get rich, find love, determine the limits of thought itself!" — mathematics[0]. Do people like learning it? They do not. Calculator much easier. What a calculator does is none of that, calculators are merely arithmetic, but most people can't tell the difference between mathematics and arithmetic. I think LLMs have the same effect on anything that can be expressed in words, and all the various image generator models have this effect on graphical arts. One must be extremely motivated to get past the "but the computer is better than me" hump. However, I don't expect AI development to even approximately halt at anything close to the current level. There's a lot of room for self-play in domains like maths and computing where the proofs can be verified, and probably a lot of room for anything that can be RLHF'd, too. And that's also assuming we don't get any brain uploads; regardless of the question of "is such an upload of a human capable of consciousness", which absolutely matters, it may still be relevant to the economic issues of AI depending on the cost of running one depending on all the details of such an upload that I can't even begin to guess at at this point (last I heard, https://openworm.org https://openworm.org was not actually measuring synaptic weights directly, but rather neural activity? I may be out of date, not my field). Whatever happens, however good it does or doesn't get, I do expect something to go very weird before I reach the current state pension age — close enough that, if that something is "the machines break" or "society breaks", then there will still be plenty who remember the before times. [0] https://www.smbc-comics.com/comic/secrets-2 https://www.smbc-comics.com/comic/secrets-2

rglover 3y ago
> I think that would require AI development to approximately halt at close to the current level for over a lifetime. What I'm getting at isn't AI development halting, but human knowledge/creativity halting [1]. Because the AI is and can only be trained on human knowledge, it's knowledge of reality has an upper bound (whereas, theoretically, humans can know anything or make new discoveries that don't exist in our current knowledge set). If you don't tell the AI that strawberries are a thing/reality, it will never conceive of a strawberry on its own. And arguably, it's not fair to call these things "AI" until they can do so. [1] Charlie Munger said "show me the incentives and I'll show you the outcome." Well, in this case, the incentives to use AI > spending the time to learn and develop skills. The outcome here is clear: humans will stop producing new knowledge and by extension, the AI will stop receiving new knowledge to learn.