5 ms·

You are naive if you think you have 10-15 years. GPT-5 will most likely be out by the end of the year. It will be significantly better than GPT-4. I expect it w

by p1esk 3y ago

You are naive if you think you have 10-15 years. GPT-5 will most likely be out by the end of the year. It will be significantly better than GPT-4. I expect it will replace millions of people - not explicitly - companies will gradually have fewer people doing more work, resulting in increasing layoffs and decreasing hiring. This has already started happening: I spoke to several startup founders recently who use GPT-4 instead of hiring marketing people. GPT-5 will be significantly better at coding, to the point where it might no longer make any sense to hire junior developers. And this is just GPT-5, this year. Next year there will be GPT-6, or an equivalent from Google or Anthropic, and at that point I fully expect a lot of people everywhere getting the boot. Sometime next year I expect these powerful models will start effectively controlling robots, and that will start the process of automation of a lot of physical work. So, to summarize, you have at best 2 years left as a software engineer. After that we can hope there will be some new types of professions that we could pivot to, but I’m struggling to think what could people possibly do better than GPT-6, so I’m not optimistic. I’d love for someone to provide a convincing argument why there would be any delay to the timeline I outlined above. p.s. I just looked at the other 20 responses in this thread, and it seems that every single one is based on current (GPT-4) LLM capabilities. Do people seriously not see any progress happening in the nearest future? Why? I’m utterly baffled by this.

findingMeaning 3y ago
> Why? I’m utterly baffled by this. Yes, me too! I am surprised people are unaware of the empirical power-scale law. [1] Currently, Anthropic's new model has proven it by being better than GPT-4 on some tasks. Yet, the old people are saying it is okay. They are saying engineering is not just coding. What they don't seem to grasp is there won't be a necessity for as many engineers. If people are not willing to accept juniors, where will the next generation of seniors come from? No clue about that. Probably, that spot is reserved for the brilliant minds graduating from R1 institutes. What about the average Joe? Not everyone has the fortune to work on very specialized domain skills. People just don't get it easily. Everyone in academia has admitted to productivity boost by 30-40%. And here we are talking about hard research. Imagine, what it looks for regular job which is managing and maintaining codebases. The severity of this situation is truly alarming not for the greybeards but for us, Zoomers. With climate crisis, AI, geopolitical instability, it feels really great to be alive. [1]: https://arxiv.org/abs/2001.08361 https://arxiv.org/abs/2001.08361
- scaredOfAI 3y ago
  Feel you so hard. Sucks to be a Zoomer now, trying to do Boomer things.
jhonof 3y ago
> And this is just GPT-5, this year. Next year there will be GPT-6, or an equivalent from Google or Anthropic, and at that point I fully expect a lot of people everywhere getting the boot. Sometime next year I expect these powerful models will start effectively controlling robots, and that will start the process of automation of a lot of physical work. > So, to summarize, you have at best 2 years left as a software engineer. After that we can hope there will be some new types of professions that we could pivot to, but I’m struggling to think what could people possibly do better than GPT-6, so I’m not optimistic. I’d love for someone to provide a convincing argument why there would be any delay to the timeline I outlined above. This reads to me exactly like people who said learning to be a truck driver in the early 2010s was stupid because we were 2-3 years away from self driving trucks taking their jobs. I have no doubt that the models will get better, but being 90-95% right still implies you need people for the last 5%. I think, like self driving, the corner case 5-10% is going to be really really hard to iron out and it will not be ironed out in 1-2 years like your comment says. We only just barely have self driving taxis now (despite them being 1-2 years away for the past decade and a half), and we have no self driving long haul trucks afaik.
- p1esk 3y ago
  To solve self-driving we need two things: 1. Deep and detailed understanding of how the world works. We are just starting to make real progress there (GPT-4), and more work is needed [1]. 2. Reliability. A model should make significantly fewer mistakes than humans would make in similar scenarios, on average. This includes factual and logical mistakes, as well as hallucinations. I expect the main improvements GPT-5 will bring are improvements in exactly these two areas. The first one is likely to come from training on huge video datasets (next frame prediction objective), and the second one will require high quality data, and some other methods (known and secret), but given that OpenAI has stated many times in the last year that improving reliability is their number one priority, I believe we will see a significant improvement there. Note that simply being better driver than humans is a very low bar, and to be accepted/adopted the self-driving AI must be much better (10x or even 100x better). But I believe that even today’s technology (such as the best models from Waymo or Tesla) could be used today in long haul trucking with similar or better accident rates. And this technology is not even based on large foundational models like GPT-4. Obviously the necessary regulation will delay the automation of self-driving trucks, that’s why I said the automation of physical jobs will come after the automation of routine office jobs like (most types of) software engineering. Other than those two challenges, there’s also an engineering challenge to put a GPT-5 scale model inside every car (needs to run locally). This can be achieved by producing custom built hardware accelerators, but will still be expensive in the near term, so I expect that self driving will become widespread after the cost of a computer inside every car falls below 10% of the cost of the car. Currently I’d imagine we would need an equivalent of an 8x H100 server to run a highly compressed and finetuned for driving version of GPT-5. [1] https://arxiv.org/abs/2403.04732 https://arxiv.org/abs/2403.04732

by p1esk 3y ago

findingMeaning 3y ago

> Why? I’m utterly baffled by this. Yes, me too! I am surprised people are unaware of the empirical power-scale law. [1] Currently, Anthropic's new model has proven it by being better than GPT-4 on some tasks. Yet, the old people are saying it is okay. They are saying engineering is not just coding. What they don't seem to grasp is there won't be a necessity for as many engineers. If people are not willing to accept juniors, where will the next generation of seniors come from? No clue about that. Probably, that spot is reserved for the brilliant minds graduating from R1 institutes. What about the average Joe? Not everyone has the fortune to work on very specialized domain skills. People just don't get it easily. Everyone in academia has admitted to productivity boost by 30-40%. And here we are talking about hard research. Imagine, what it looks for regular job which is managing and maintaining codebases. The severity of this situation is truly alarming not for the greybeards but for us, Zoomers. With climate crisis, AI, geopolitical instability, it feels really great to be alive. [1]: https://arxiv.org/abs/2001.08361 https://arxiv.org/abs/2001.08361

scaredOfAI 3y ago
Feel you so hard. Sucks to be a Zoomer now, trying to do Boomer things.

jhonof 3y ago

> And this is just GPT-5, this year. Next year there will be GPT-6, or an equivalent from Google or Anthropic, and at that point I fully expect a lot of people everywhere getting the boot. Sometime next year I expect these powerful models will start effectively controlling robots, and that will start the process of automation of a lot of physical work. > So, to summarize, you have at best 2 years left as a software engineer. After that we can hope there will be some new types of professions that we could pivot to, but I’m struggling to think what could people possibly do better than GPT-6, so I’m not optimistic. I’d love for someone to provide a convincing argument why there would be any delay to the timeline I outlined above. This reads to me exactly like people who said learning to be a truck driver in the early 2010s was stupid because we were 2-3 years away from self driving trucks taking their jobs. I have no doubt that the models will get better, but being 90-95% right still implies you need people for the last 5%. I think, like self driving, the corner case 5-10% is going to be really really hard to iron out and it will not be ironed out in 1-2 years like your comment says. We only just barely have self driving taxis now (despite them being 1-2 years away for the past decade and a half), and we have no self driving long haul trucks afaik.

p1esk 3y ago
To solve self-driving we need two things: 1. Deep and detailed understanding of how the world works. We are just starting to make real progress there (GPT-4), and more work is needed [1]. 2. Reliability. A model should make significantly fewer mistakes than humans would make in similar scenarios, on average. This includes factual and logical mistakes, as well as hallucinations. I expect the main improvements GPT-5 will bring are improvements in exactly these two areas. The first one is likely to come from training on huge video datasets (next frame prediction objective), and the second one will require high quality data, and some other methods (known and secret), but given that OpenAI has stated many times in the last year that improving reliability is their number one priority, I believe we will see a significant improvement there. Note that simply being better driver than humans is a very low bar, and to be accepted/adopted the self-driving AI must be much better (10x or even 100x better). But I believe that even today’s technology (such as the best models from Waymo or Tesla) could be used today in long haul trucking with similar or better accident rates. And this technology is not even based on large foundational models like GPT-4. Obviously the necessary regulation will delay the automation of self-driving trucks, that’s why I said the automation of physical jobs will come after the automation of routine office jobs like (most types of) software engineering. Other than those two challenges, there’s also an engineering challenge to put a GPT-5 scale model inside every car (needs to run locally). This can be achieved by producing custom built hardware accelerators, but will still be expensive in the near term, so I expect that self driving will become widespread after the cost of a computer inside every car falls below 10% of the cost of the car. Currently I’d imagine we would need an equivalent of an 8x H100 server to run a highly compressed and finetuned for driving version of GPT-5. [1] https://arxiv.org/abs/2403.04732 https://arxiv.org/abs/2403.04732