5 ms·

"Attention is all you need" was published only 5 years ago in 2017. BERT in 2018, GPT-3 in 2020. Now DALL-E, PaLM, LaMDA, etc. The pace of AI progress is fright

"Attention is all you need" was published only 5 years ago in 2017. BERT in 2018, GPT-3 in 2020. Now DALL-E, PaLM, LaMDA, etc. The pace of AI progress is frighteningly fast. I really thin it's plausible that by 2030, AI is writing and debugging most code, and engineers basically become spec writers who manipulate prompts, generate code, and verify. Essentially, project managers whose "team" is a bunch of LLMs. It really doesn't seem we've hit diminishing returns on LLM model size yet, and that's not accounting for the multi-input types where video, text, audio, and more are being fused together.

ummwhat 5y ago
Ya stupid sons of bitches, you automated everyone else out of a job and then you gone done it to yourselves too!
- CryptoPunk 5y ago
  Every higher level language, library and framewok automates coding. But the result is more complex programs, not fewer developers. That's a microcosm of the effect of automation on the economy as a whole.
  - ummwhat 5y ago
    90% of work used to be farming. Today 10% of people are farmers. The next big thing was factory work. At first looms meant more textiles, not less workers. But eventually we hit the transition point where there's actually less factory workers. Now we've moved on to "knowledge work". Sure the effects of automation make more complex knowledge products and jobs for now, but once the automation becomes advanced enough, jobs in the field go down not up. So what comes after knowledge work? We can't have it both ways. We can't pretend like automation will never kill our jobs while simultaneously pursuing the dream of permeant vacation through automation. The very goal of automation contradicts the idea that we would and should always have jobs.
    - CryptoPunk 5y ago
      If we produce enough of everything with less labor, each individual will work fewer hours to get all of their needs met. And that would be a good thing. But we are very very far from that point. We have needs, like extending our life, that current technology cannot meet, and if we had excess time, most of us would trade it in exchange for more technological progress toward that goal. Fundamentally that is what's behind the increase in healthcare spending, and the growth in people employed in healthcare.
      - krapp 5y ago
        >If we produce enough of everything with less labor, each individual will work fewer hours to get all of their needs met. No they won't. If people work fewer hours, they'll simply be paid less, barring legal limitations like a minimum wage. Meanwhile, costs will remain the same.
        CryptoPunk 5y ago
        People are paid in goods/services, fundamentally. As production per hour of labor increases, so does wages, since there are fewer hours chasing more goods.
        pretendscholar 5y ago
        That hasn't been true in 50 years
gfodor 5y ago
Yeah plus these haven’t been trained on multiple epochs and DeepMind just released Chinchilla which implies we don’t need to scale up as much as we have been wrt parameters. Also last month Microsoft published a paper demonstrating deep transformers. We may get to expert level performance on these without any major breakthroughs imo.
dx034 5y ago
You'd still need to translate business requirements into exact requirements. For me, that's 90%+ of programming, and basically the job of a programmer. Technical details can already be abstracted away with libraries. My productivity has definitely been increased by Copilot and there's potential for more increases, but I don't see where that would replace the programmer.

ummwhat 5y ago

Ya stupid sons of bitches, you automated everyone else out of a job and then you gone done it to yourselves too!

CryptoPunk 5y ago
Every higher level language, library and framewok automates coding. But the result is more complex programs, not fewer developers. That's a microcosm of the effect of automation on the economy as a whole.
- ummwhat 5y ago
  90% of work used to be farming. Today 10% of people are farmers. The next big thing was factory work. At first looms meant more textiles, not less workers. But eventually we hit the transition point where there's actually less factory workers. Now we've moved on to "knowledge work". Sure the effects of automation make more complex knowledge products and jobs for now, but once the automation becomes advanced enough, jobs in the field go down not up. So what comes after knowledge work? We can't have it both ways. We can't pretend like automation will never kill our jobs while simultaneously pursuing the dream of permeant vacation through automation. The very goal of automation contradicts the idea that we would and should always have jobs.
  - CryptoPunk 5y ago
    If we produce enough of everything with less labor, each individual will work fewer hours to get all of their needs met. And that would be a good thing. But we are very very far from that point. We have needs, like extending our life, that current technology cannot meet, and if we had excess time, most of us would trade it in exchange for more technological progress toward that goal. Fundamentally that is what's behind the increase in healthcare spending, and the growth in people employed in healthcare.
    - krapp 5y ago
      >If we produce enough of everything with less labor, each individual will work fewer hours to get all of their needs met. No they won't. If people work fewer hours, they'll simply be paid less, barring legal limitations like a minimum wage. Meanwhile, costs will remain the same.
      - CryptoPunk 5y ago
        People are paid in goods/services, fundamentally. As production per hour of labor increases, so does wages, since there are fewer hours chasing more goods.
        pretendscholar 5y ago
        That hasn't been true in 50 years

gfodor 5y ago

Yeah plus these haven’t been trained on multiple epochs and DeepMind just released Chinchilla which implies we don’t need to scale up as much as we have been wrt parameters. Also last month Microsoft published a paper demonstrating deep transformers. We may get to expert level performance on these without any major breakthroughs imo.

dx034 5y ago

You'd still need to translate business requirements into exact requirements. For me, that's 90%+ of programming, and basically the job of a programmer. Technical details can already be abstracted away with libraries. My productivity has definitely been increased by Copilot and there's potential for more increases, but I don't see where that would replace the programmer.