5 ms·

All the world's smartest minds are racing towards replacing themselves. As programmers, we should take note and see where the wind is blowing. At least don't di

by am17an 1y ago

All the world's smartest minds are racing towards replacing themselves. As programmers, we should take note and see where the wind is blowing. At least don't discard the possibility and rather be prepared for the future. Not to sound like a tin-foil hat but odds of achieving something like this increase by the day. In the long term (post AGI), the only safe white-collar jobs would be those built on data which is not public i.e. extremely proprietary (e.g. Defense, Finance) and even those will rely heavily on customized AIs.

bgwalter 1y ago
The Nobel prize is said to have been created partly out of guilt over having invented dynamite, which was obviously used in a destructive manner. Now we have Geoffrey Hinton getting the prize for contributing to one of the most destructive inventions ever.
- reducesuffering 1y ago
  At least he and Yoshua Bengio are remorseful. Many others haven't even gotten that far...
AstroBen 1y ago
Ultimately this needs to be solved politically Making our work more efficient, or humans redundant should be really exciting. It's not set in stone that we need to leave people middle aged with families and now completely unable to earn enough to provide a good life Hopefully if it happens, it happens to such a huge amount of people that it forces a change
- lyu07282 1y ago
  But that already happened to lots of industries and lots of people, we never cared before about them, now it's us so we care, but nothing is different about us. Just learn to code!
  - AstroBen 1y ago
    The difference is in how many industries AI is threatening. It's not just coding on the chopping block
    - bluefirebrand 1y ago
      No different than how many industries that offshoring wrecked
  - geraneum 1y ago
    > But that already happened to lots of industries and lots of people, we never cared before about them We did. Why do you think labor laws, unions, etc. exist? Why do you think communism was appealing as an idea in the beginning to many? Whether the effects were good or bad or enough or not, that’s a different question. But these changes have demonstrably, grave consequences.
bitpush 1y ago
> All the world's smartest minds are racing towards replacing themselves Isnt every little script, every little automation us programmers do in the same spirit? "I dont like doing this, so I'm going to automate it, so that I can focus on other work". Sure, we're racing towards replacing ourselves, but there would be (and will be) other more interesting work for us to do when we're free to do that. Perhaps, all of us will finally have time to learn surfing, or garden, or something. Some might still write code themselves by hand, just like how some folks like making bread .. but making bread by hand is not how you feed a civilization - even if hundreds of bakers were put out of business.
- AstroBen 1y ago
  > all of us will finally have time to learn surfing, or garden Unless you have a mortgage.. or rent.. or need to eat
wijwp 1y ago
> Not to sound like a tin-foil hat but odds of achieving something like this increase by the day. Where do you get this? The limitations of LLMs are becoming more clear by the day. Improvements are slowing down. Major improvements come from integrations, not major model improvements. AGI likely can't be achieved with LLMs. That wasn't as clear a couple years ago.
- drodgers 1y ago
  I don't know how someone could be following the technical progress in detail and hold this view. The progress is astonishing, and the benchmarks are becoming saturated so fast that it's hard to keep track. Are there plenty of gaps left between here and most definitions of AGI? Absolutely. Nevertheless, how can you be sure that those gaps will remain given how many faculties these models have already been able to excel at (translation, maths, writing, code, chess, algorithm design etc.)? It seems to me like we're down to a relatively sparse list of tasks and skills where the models aren't getting enough training data, or are missing tools and sub-components required to excel. Beyond that, it's just a matter of iterative improvement until 80th percentile coder becomes 99th percentile coder becomes superhuman coder, and ditto for maths, persuasion and everything else. Maybe we hit some hard roadblocks, but room for those challenges to be hiding seems to be dwindling day by day.
  - materiallie 1y ago
    I think benchmark targeting is going to be a serious problem going forward. The recent Nate Silver podcast on poker performance is interesting. Basically, the LLM models still suck at playing poker. Poker tests intelligence. So what gives? One interesting thing is that for whatever reason, poker performance isn't used a benchmark in the LLM showdown between big tech companies. The models have definitely improved in the past few years. I'm skeptical that there's been a "break-through", and I'm growing more skeptical of the exponential growth theory. It looks to me like the big tech companies are just throwing huge compute and engineering budgets at the existing transformer tech, to improve benchmarks one by one. I'm sure if Google allocated 10 engineers a dozen million dollars to improve Gemini's poker performance, it would increase. The idea before AGI and the exponential growth hypothesis is that you don't have to do that because the AI gets smarter in a general sense all on it's own.
BirAdam 1y ago
Nah. As more people are rendered unemployed the buying market and therefore aggregate demand will fall. Fewer sales hurts the bottom line. At some point, revenues across the entire economy fall, and companies cannot afford the massive datacenters and nuclear power plants fueling them. The hardware gets sold cheap, the companies go under, and people get hired again. Eventually, some kind of equilibrium will be found or the world engages in the Butlerian Jihad.
cheema33 1y ago
> All the world's smartest minds are racing towards replacing themselves. I think they are hoping that their future is safe. And it is the average minds that will have to go first. There may be some truth to it. Also, many of these smartest minds are motivated by money, to safeguard their future, from a certain doom that they know might be coming. And AI is a good place to be if you want to accumulate wealth fast.

by am17an 1y ago

bgwalter 1y ago

The Nobel prize is said to have been created partly out of guilt over having invented dynamite, which was obviously used in a destructive manner. Now we have Geoffrey Hinton getting the prize for contributing to one of the most destructive inventions ever.

reducesuffering 1y ago
At least he and Yoshua Bengio are remorseful. Many others haven't even gotten that far...

AstroBen 1y ago

Ultimately this needs to be solved politically Making our work more efficient, or humans redundant should be really exciting. It's not set in stone that we need to leave people middle aged with families and now completely unable to earn enough to provide a good life Hopefully if it happens, it happens to such a huge amount of people that it forces a change

lyu07282 1y ago
But that already happened to lots of industries and lots of people, we never cared before about them, now it's us so we care, but nothing is different about us. Just learn to code!
- AstroBen 1y ago
  The difference is in how many industries AI is threatening. It's not just coding on the chopping block
  - bluefirebrand 1y ago
    No different than how many industries that offshoring wrecked
- geraneum 1y ago
  > But that already happened to lots of industries and lots of people, we never cared before about them We did. Why do you think labor laws, unions, etc. exist? Why do you think communism was appealing as an idea in the beginning to many? Whether the effects were good or bad or enough or not, that’s a different question. But these changes have demonstrably, grave consequences.

bitpush 1y ago

> All the world's smartest minds are racing towards replacing themselves Isnt every little script, every little automation us programmers do in the same spirit? "I dont like doing this, so I'm going to automate it, so that I can focus on other work". Sure, we're racing towards replacing ourselves, but there would be (and will be) other more interesting work for us to do when we're free to do that. Perhaps, all of us will finally have time to learn surfing, or garden, or something. Some might still write code themselves by hand, just like how some folks like making bread .. but making bread by hand is not how you feed a civilization - even if hundreds of bakers were put out of business.

AstroBen 1y ago
> all of us will finally have time to learn surfing, or garden Unless you have a mortgage.. or rent.. or need to eat

wijwp 1y ago

> Not to sound like a tin-foil hat but odds of achieving something like this increase by the day. Where do you get this? The limitations of LLMs are becoming more clear by the day. Improvements are slowing down. Major improvements come from integrations, not major model improvements. AGI likely can't be achieved with LLMs. That wasn't as clear a couple years ago.

drodgers 1y ago
I don't know how someone could be following the technical progress in detail and hold this view. The progress is astonishing, and the benchmarks are becoming saturated so fast that it's hard to keep track. Are there plenty of gaps left between here and most definitions of AGI? Absolutely. Nevertheless, how can you be sure that those gaps will remain given how many faculties these models have already been able to excel at (translation, maths, writing, code, chess, algorithm design etc.)? It seems to me like we're down to a relatively sparse list of tasks and skills where the models aren't getting enough training data, or are missing tools and sub-components required to excel. Beyond that, it's just a matter of iterative improvement until 80th percentile coder becomes 99th percentile coder becomes superhuman coder, and ditto for maths, persuasion and everything else. Maybe we hit some hard roadblocks, but room for those challenges to be hiding seems to be dwindling day by day.
- materiallie 1y ago
  I think benchmark targeting is going to be a serious problem going forward. The recent Nate Silver podcast on poker performance is interesting. Basically, the LLM models still suck at playing poker. Poker tests intelligence. So what gives? One interesting thing is that for whatever reason, poker performance isn't used a benchmark in the LLM showdown between big tech companies. The models have definitely improved in the past few years. I'm skeptical that there's been a "break-through", and I'm growing more skeptical of the exponential growth theory. It looks to me like the big tech companies are just throwing huge compute and engineering budgets at the existing transformer tech, to improve benchmarks one by one. I'm sure if Google allocated 10 engineers a dozen million dollars to improve Gemini's poker performance, it would increase. The idea before AGI and the exponential growth hypothesis is that you don't have to do that because the AI gets smarter in a general sense all on it's own.

BirAdam 1y ago

Nah. As more people are rendered unemployed the buying market and therefore aggregate demand will fall. Fewer sales hurts the bottom line. At some point, revenues across the entire economy fall, and companies cannot afford the massive datacenters and nuclear power plants fueling them. The hardware gets sold cheap, the companies go under, and people get hired again. Eventually, some kind of equilibrium will be found or the world engages in the Butlerian Jihad.

cheema33 1y ago

> All the world's smartest minds are racing towards replacing themselves. I think they are hoping that their future is safe. And it is the average minds that will have to go first. There may be some truth to it. Also, many of these smartest minds are motivated by money, to safeguard their future, from a certain doom that they know might be coming. And AI is a good place to be if you want to accumulate wealth fast.