5 ms·

The more I think about it, the more I am convinced developers will be the "first to go" when AGI takes over. Before bloggers and youtubers. Because programming

The more I think about it, the more I am convinced developers will be the "first to go" when AGI takes over. Before bloggers and youtubers. Because programming is an activity that requires the least amount of "grounding to reality" among all human activities. We made sure of this with layers and layers of convenient abstraction. What about developers that code the AI systems? Well.. I am sure AGI will come from other "bootsrapping AIs" just like we see with compilers that compile themselves. When I see Altman and Sutskever talking about AGI being within reach, I feel they are talking about this bootstrapping AI being within reach.

torginus 2y ago
I think there's a lot of evidence out there that supports your theory. - There's the biggest/most high quality training corpus that captures all aspects of dev work (code, changes, discussions about issues, etc.) out there with open source hosting sites like GitHub - Synthetic data is easy to generate and verify, you can just run unit tests/debugger in a loop until you get it right. Try doing that with contracts or tax statements. - Little to no regulatory protections.
raincole 2y ago
When AGI comes, who are the first to go is solely decided by regulation. Nothing else.
kristopolous 2y ago
It's the same thing that happened with mechanical looms. Programming will go from an artisan craft to the sweatshops and software programming will become low paying precarious gig work. This is all about labor and capital. When people toss hundreds of billions at something it almost always is. The social relationship doesn't have to be this way. Technological improvement could help us instead of screw us over. But we'd first have to admit that profit exploitation isn't absolutely the best thing ever and we'll never do that. Soooo here we are.
probably_wrong 2y ago
> Because programming is an activity that requires the least amount of "grounding to reality" among all human activities. Maybe once you're deep into APIs that talk to other APIs, but near the surface where the data is collected there's nothing but "grounding to reality". As my professor of Software Engineering put it: when building a system for counting the number of people inside a room most people would put a turnstile and count the turns. Does this fulfill the requirement? No - people can turn the wheel multiple times, leave through a window, give birth inside the room, etc. Is it good enough? Only your client can say, and only after considering factors like "available technology" and "budget" that have nothing to do with software.
b212 2y ago
With AGI you won’t need most of the human race anymore, developers are just the tip of the iceberg. Luckily ChatGPT and the rest have nothing to do with an AI not to mention AGI.
- Jeff_Brown 2y ago
  It's not AGI yet but has everything to do with it.
  - lttlrck 2y ago
    Hmmm. I'm going to withhold judgement until we get AGI. It might have about as much to do with it as the invention of the wheel and the lunar buggy.
    - maxwell 2y ago
      How will you know when it's sufficiently general?
- maxwell 2y ago
  Won't need most of humanity wasting time gatekeeping and toiling, to instead focus on study and leisure.
  - rayval 2y ago
    That would be nice, to gambol in a lush, sunny meadow, perusing a leather-bound volume that we read in the original Greek or Sanskrit. Unfortunately, I fear we will instead end up sweaty and dirty and bloodied, grappling in the parched terrain, trying to bash members of a neighboring clan with rocks and wooden clubs, while a skyline of crumbling skyscrapers looms in the distance.
  - bbu 2y ago
    Increased productivity has to more, not less toiling in the last 50 or so years. This is not gonna be different with AGI (if it ever happens). Some people will get very rich while the rest is still gonna work just as much as they are now. The jobs just gonna suck even more.
dboreham 2y ago
Sticking to my prediction that lawyers will be first.
- frankdenbow 2y ago
  Agree with that, the startup lawyer I used to use now charges $1,100 / hour which is untenable, would much rather get the basics covered by an AI lawyer.
- vlunkr 2y ago
  There’s a company being fined for exactly this right now. That might be a deterrent.
- choilive 2y ago
  Lawyers have a tricky thing called a license to practice law. Software developers have no such protection.
  - lttlrck 2y ago
    Defending yourself might become a whole lot easier.
  - kylecazar 2y ago
    Robots are already successfully performing surgeries unassisted, despite a license being required in that field. I do think barred lawyers will have a role for quite a while, but it is plausible it shrinks to oversight.
    - gugagore 2y ago
      What robot is performing what surgery unassisted?
      - filoleg 2y ago
        Yeah, plus one on this one, extremely curious to hear as well. I am aware that remote robot surgeries have been a thing for quite a bit of time, but this is the first time ever I am hearing about unassisted robot surgeries being a thing at all. A follow-up question: if an unassisted robot surgery goes wrong, who is liable? I know we have a similar dilemma with self-driving cars, but I was under the impression that things are way more regulated and strict in the realm of healthcare.
kovezd 2y ago
That thinking is driven by limited product imagination. The kind of apps that will be built in the next 5 years,are nowhere near what we have today. Developers will need to update their skillset, though.
- throwawaymanbot 2y ago
  [dead]
- visarga 2y ago
  It's not a fixed lump of work. Widening the road makes the traffic increase, not congestion decrease. Same with coding, if we could do more, we would.
emmanueloga_ 2y ago
Mechanical Turks will be the first to go. More seriously, the output quality of LLMs for code is pretty inconsistent. I think there's an analogy to be made with literature. For instance, a short story generated by an LLM can't really hold a candle to the work of a human author. LLM-generated code can be a good starting point for avoiding tedious aspects of software development, like boilerplate or repetitive tasks. When it works, it saves a lot of time. For example, if I need to generate a bunch of similar functions, an LLM can sometimes act like an ad-hoc code generator, helping to skip the manual labor. I’ve also gotten some helpful suggestions on code style, though mostly for small snippets. It’s especially useful for refreshing things you already know—like quickly recalling "How do I do this with TypeScript?" without needing to search for documentation. Anyway, literature writers and software engineers aren't going to be replaced anytime soon.
- visarga 2y ago
  > Mechanical Turks will be the first to go. On the contrary, human annotation work is stepping up now because we create so many more prompts and want to test them.
- torginus 2y ago
  Those jobs are already outsourced if possible (remember RLHF for ChatGPT outsourced to Kenya, Facebook content moderation to India?) And if they aren't, that's usually for regulatory reasons.
- razodactyl 2y ago
  Yes, but don't dismiss the laws of scaling and the inability of ourselves to predict the future. Wait But Why did a great take on the AI leap. Imagine that current AI is already curating and generating datasets for the next generation. Also consider that what we have now is only possible because hardware capability increased.
  - satvikpendem 2y ago
    Isn't scaling already a big problem in generative AI? Apparently there is not enough data to actually appreciably increase the quality of outputs, and lots of data is also being polluted by AI generations. This is why these companies are now shifting to more ancillary features like in this article (which Claude already has to some extent, as well as Copilot in VSCode and other editors).
juthen 2y ago
Maintenance and long-term development will still require grounding to reality. A super-CEO might build the app themself, but keeping it running 5, 10, 20 years is a completely different deal. I imagine developers will eventually start to act more like librarians, knowing their system very well but not necessarily in charge of making new content.
bamboozled 2y ago
No one has a job when AGI "takes over"? Weird take.

torginus 2y ago

I think there's a lot of evidence out there that supports your theory. - There's the biggest/most high quality training corpus that captures all aspects of dev work (code, changes, discussions about issues, etc.) out there with open source hosting sites like GitHub - Synthetic data is easy to generate and verify, you can just run unit tests/debugger in a loop until you get it right. Try doing that with contracts or tax statements. - Little to no regulatory protections.

raincole 2y ago

When AGI comes, who are the first to go is solely decided by regulation. Nothing else.

kristopolous 2y ago

It's the same thing that happened with mechanical looms. Programming will go from an artisan craft to the sweatshops and software programming will become low paying precarious gig work. This is all about labor and capital. When people toss hundreds of billions at something it almost always is. The social relationship doesn't have to be this way. Technological improvement could help us instead of screw us over. But we'd first have to admit that profit exploitation isn't absolutely the best thing ever and we'll never do that. Soooo here we are.

probably_wrong 2y ago

> Because programming is an activity that requires the least amount of "grounding to reality" among all human activities. Maybe once you're deep into APIs that talk to other APIs, but near the surface where the data is collected there's nothing but "grounding to reality". As my professor of Software Engineering put it: when building a system for counting the number of people inside a room most people would put a turnstile and count the turns. Does this fulfill the requirement? No - people can turn the wheel multiple times, leave through a window, give birth inside the room, etc. Is it good enough? Only your client can say, and only after considering factors like "available technology" and "budget" that have nothing to do with software.

b212 2y ago

With AGI you won’t need most of the human race anymore, developers are just the tip of the iceberg. Luckily ChatGPT and the rest have nothing to do with an AI not to mention AGI.

Jeff_Brown 2y ago
It's not AGI yet but has everything to do with it.
- lttlrck 2y ago
  Hmmm. I'm going to withhold judgement until we get AGI. It might have about as much to do with it as the invention of the wheel and the lunar buggy.
  - maxwell 2y ago
    How will you know when it's sufficiently general?
maxwell 2y ago
Won't need most of humanity wasting time gatekeeping and toiling, to instead focus on study and leisure.
- rayval 2y ago
  That would be nice, to gambol in a lush, sunny meadow, perusing a leather-bound volume that we read in the original Greek or Sanskrit. Unfortunately, I fear we will instead end up sweaty and dirty and bloodied, grappling in the parched terrain, trying to bash members of a neighboring clan with rocks and wooden clubs, while a skyline of crumbling skyscrapers looms in the distance.
- bbu 2y ago
  Increased productivity has to more, not less toiling in the last 50 or so years. This is not gonna be different with AGI (if it ever happens). Some people will get very rich while the rest is still gonna work just as much as they are now. The jobs just gonna suck even more.

dboreham 2y ago

Sticking to my prediction that lawyers will be first.

frankdenbow 2y ago
Agree with that, the startup lawyer I used to use now charges $1,100 / hour which is untenable, would much rather get the basics covered by an AI lawyer.
vlunkr 2y ago
There’s a company being fined for exactly this right now. That might be a deterrent.
choilive 2y ago
Lawyers have a tricky thing called a license to practice law. Software developers have no such protection.
- lttlrck 2y ago
  Defending yourself might become a whole lot easier.
- kylecazar 2y ago
  Robots are already successfully performing surgeries unassisted, despite a license being required in that field. I do think barred lawyers will have a role for quite a while, but it is plausible it shrinks to oversight.
  - gugagore 2y ago
    What robot is performing what surgery unassisted?
    - filoleg 2y ago
      Yeah, plus one on this one, extremely curious to hear as well. I am aware that remote robot surgeries have been a thing for quite a bit of time, but this is the first time ever I am hearing about unassisted robot surgeries being a thing at all. A follow-up question: if an unassisted robot surgery goes wrong, who is liable? I know we have a similar dilemma with self-driving cars, but I was under the impression that things are way more regulated and strict in the realm of healthcare.

kovezd 2y ago

That thinking is driven by limited product imagination. The kind of apps that will be built in the next 5 years,are nowhere near what we have today. Developers will need to update their skillset, though.

throwawaymanbot 2y ago
[dead]
visarga 2y ago
It's not a fixed lump of work. Widening the road makes the traffic increase, not congestion decrease. Same with coding, if we could do more, we would.

emmanueloga_ 2y ago

Mechanical Turks will be the first to go. More seriously, the output quality of LLMs for code is pretty inconsistent. I think there's an analogy to be made with literature. For instance, a short story generated by an LLM can't really hold a candle to the work of a human author. LLM-generated code can be a good starting point for avoiding tedious aspects of software development, like boilerplate or repetitive tasks. When it works, it saves a lot of time. For example, if I need to generate a bunch of similar functions, an LLM can sometimes act like an ad-hoc code generator, helping to skip the manual labor. I’ve also gotten some helpful suggestions on code style, though mostly for small snippets. It’s especially useful for refreshing things you already know—like quickly recalling "How do I do this with TypeScript?" without needing to search for documentation. Anyway, literature writers and software engineers aren't going to be replaced anytime soon.

visarga 2y ago
> Mechanical Turks will be the first to go. On the contrary, human annotation work is stepping up now because we create so many more prompts and want to test them.
torginus 2y ago
Those jobs are already outsourced if possible (remember RLHF for ChatGPT outsourced to Kenya, Facebook content moderation to India?) And if they aren't, that's usually for regulatory reasons.
razodactyl 2y ago
Yes, but don't dismiss the laws of scaling and the inability of ourselves to predict the future. Wait But Why did a great take on the AI leap. Imagine that current AI is already curating and generating datasets for the next generation. Also consider that what we have now is only possible because hardware capability increased.
- satvikpendem 2y ago
  Isn't scaling already a big problem in generative AI? Apparently there is not enough data to actually appreciably increase the quality of outputs, and lots of data is also being polluted by AI generations. This is why these companies are now shifting to more ancillary features like in this article (which Claude already has to some extent, as well as Copilot in VSCode and other editors).

juthen 2y ago

Maintenance and long-term development will still require grounding to reality. A super-CEO might build the app themself, but keeping it running 5, 10, 20 years is a completely different deal. I imagine developers will eventually start to act more like librarians, knowing their system very well but not necessarily in charge of making new content.

bamboozled 2y ago

No one has a job when AGI "takes over"? Weird take.