6 ms·

Between this and OpenAI's Github Copilot "programming" will slowly start dying probably. What I mean by that is that sure, you have to learn how to program, but

by hmate9 5y ago

Between this and OpenAI's Github Copilot "programming" will slowly start dying probably. What I mean by that is that sure, you have to learn how to program, but our time will be spent much more on just the design part and writing detailed documentation/specs and then we just have one of these AIs generate the code. It's the next step. Binary code < assembly < C < Python < AlphaCode Historically its always been about abstracting and writing less code to do more.

wittycardio 5y ago
Solving competitive programming problems is essentially solving hard combinatorial optimization problems. Throwing a massive amount of compute and gradient descent at the problem has always been possible. If I'm not mistaken what this does is reduce the representation of the problem to a state where it can run gradient descent and then tune parameters. The real magic is in finding structurally new approaches. If anything I'd say algorithms and math continue to be the core of programming. The particular syntax or level of abstraction don't matter so much.
- chroem- 5y ago
  > Solving competitive programming problems is essentially solving hard combinatorial optimization problems. True, but if you relax your hard requirements of optimality to admit "good enough" solutions, you can use heuristic approaches that are much more tractable. High quality heuristic solutions to NP-hard problems, enabled by ML, are going to be a big topic over the next decade, I think.
  - wittycardio 5y ago
    I should correct myself, this isn't even that. This is just text analysis on codeforces solutions, which makes it even worse than I thought. Very pessimistic about it's generalizability.
- jdlshore 5y ago
  > If anything I'd say algorithms and math continue to be the core of programming. I disagree; I think the core of programming is analyzing things people want and expressing solutions to those wants clearly, unambiguously, and in a way that is easy to change in the future. I'd say algorithms and math are a very small part of this work.
  - wittycardio 5y ago
    That's not programming, that's called being a good employee. Any person in any role should be doing that. Programming is about algorithms and math. Now a good employee who's in a technical role should have both.
    - jdlshore 5y ago
      > Programming is about algorithms and math. You've simply restated your opinion without providing any supporting arguments, and as I already said, I disagree. The vast majority of programming I see (and as a consultant, I see a fairly wide variety) is not about algorithms and math, but instead gluing together systems and expressing domain logic. Now, I suppose you could argue that domain logic is "algorithms and math," but in my experience, it's less about the specific algorithms and more about precisely describing fuzzy human behavior. It's that "precisely describing" and "easy to change in the future" parts that makes what programmers do different than what any good employee does. (I do agree that there is some programming that is focused on algorithms and math, but it's in the minority, in my experience. Perhaps the type of work you do is focused on algorithms and math, but I believe that's a relatively small part of the software development ecosystem.)
      - wittycardio 5y ago
        No I'm not talking about programming that requires calculations or programs written to solve mathematical problems. Programming at its core is about defining precise logical relationships between abstract objects and then writing algorithms to understand and modify these objects. This is a mathematical process and you should use mathematical thinking to do this.It may not always seem like it when the objects and relationships appear to be simple but that is the core of programming.
mhzsh 5y ago
Creating a higher level abstraction is something people have been trying to do for decades with so-called 4th-generation languages. At some point, abstracting away too much makes a tool too cookie-cutter, and suddenly deviating from it causes more difficulty.
- vvilliamperez 5y ago
  Read: Ruby on Rails
- visarga 5y ago
  Maybe it's not more abstraction we need, just automating the drudgery. Abstractions are limited - by definition they abstract things away, they are brittle.
Inufu 5y ago
I agree, I expect programmers will just move up the levels of abstraction. I enjoyed this recent blog post on the topic: https://eli.thegreenplace.net/2022/asimov-programming-and-the-meta-ladder/ https://eli.thegreenplace.net/2022/asimov-programming-and-th...
- hackinthebochs 5y ago
  The "problem" is that as you move up the levels of abstraction, you need fewer people to do the same amount of work. Unless the complexity of the work scales as well. I've always felt that programmers would be the first class of knowledge workers to be put out of work by automation. This may be the beginning of the end for the programming gravy train.
  - bmh100 5y ago
    On the other hand, as the value of an hour of programming increases, the quantity demanded may also increase.
  - hmate9 5y ago
    Or you can do things at a faster pace and increase your productivity.
  - NicoJuicy 5y ago
    There aren't enough developers either way.
  - Inufu 5y ago
    Yes, this is how you increase prosperity (see: agricultural revolution, industrial revolution, etc). You can now create more with the same number of people.
  - paxys 5y ago
    > as you move up the levels of abstraction, you need fewer people to do the same amount of work Yes, but the total amount of work (and surrounding complexity) also increases with it. Just look at the evolution of the software industry over the last few decades.
streetcat1 5y ago
First, If this is correct, if alpha code succeeded, this will bring to its own demise. I.e. as soon as it starts replacing humans, it will not have enough human generated training data, since all of programming will be done by models like himself. Second, alphacode was specifically trained for competitive programming : 1. short programs. 2. Each program has 100's of human generated solutions. However, commercial program are: 1. long. 2. Have no predefined answer or even correct answer. 3. Need to use/reuse a lot of legacy code.
- chroem- 5y ago
  Reinforcement learning and adversarial training can render both of those concerns as non-issues in practice.
  - ialyos 5y ago
    The phrase "in practice" doesn't really work when you're referring to highly finicky strategies like RL and adversarial training
- AnIdiotOnTheNet 5y ago
  > as soon as it starts replacing humans, it will not have enough human generated training data, since all of programming will be done by models like himself. As a natural born pessimist, I can't help but feel that by the time we get to that point we'll just keep blundering forward and adapting our world around the wild nonsense garbage code the model ends up producing in this scenario. After all, that's basically what we've done with the entire web stack.
diehunde 5y ago
My bet would be that it will never happen in a reasonable time frame. And also by that logic, writing that "documentation/spec" would just mean learning a new programming language the AI engine can parse making it as useful as a compiler. Anyone who has been writing and designing software for a while knows the cycle is way more complex than take some input and write code. Let me know when the AI engine is able to do complex refactoring or adding features that keeps backwards compatibility, find a bug in a giant codebase by debugging a test case or write code that's performant but also maintainable.
- fvold 5y ago
  I hear that. Machine Learning also has a long way to go before it can take a long, rambling mess of a meeting and somehow generate a halfway usable spec from it. I mean, the customer says they want X, but X is silly in this context, so we'll give them Y and tell them it's "X-like, but faster". For example, SQL is "Blockchain-like, but faster" for a lot of buzzword use-cases of blockchain.
- Enginerrrd 5y ago
  I agree, from a totally different angle. Let's take something I know better as an example: Structural engineering. Structural engineering should be a "solved problem". It seems, ostensibly, relatively simple compared to a more open ended activity like "programming".(For "technical reasons", it ends up being more similar than you might think.) Still, you are ultimately dealing with the same materials, the same physics, and very similar configurations. And yet, despite the fact that we have programs to help calculate all the things, test code-required load-combinations, even run simulations and size individual components... it turns out that, it doesn't actually save that much work, and you still need an engineer to do most of it. And not just because of regulatory requirements. It's just, that's not the hard part. The hard part is assembling the components and specifications, specifying the correct loads based on location-specific circumstances, coming up with coherent and sensible design ideas, chasing down every possible creative nook and cranny of code to make something that was originally a mistake actually work, and know when the model is just wrong for some reason and the computer isn't simulating load paths accurately. Specifying the inputs and interpreting results is still about as much work as it was before you started with all the fancy tools. Those tools still have advantages mind you, and they do make one slightly more efficient. Substantially so in some cases, but most of the time it still comes out as a slight assist rather than a major automation.
pjmorris 5y ago
I'd note that assembly, C, and Python didn't replace 'programming' but were expected to do so. I'd wager that what you now call 'detailed documentation/specs' will still be called programming in 10 or even 20 years.
- falcor84 5y ago
  If you could change a sentence in the documentation and then run a ~1min compilation to see the resulting software, it would be a very different kind of programming. I suppose it'll give a new meaning to Readme-Driven-Development.
62951413 5y ago
Model-driven development and code generation from UML were once supposed to be the future. It will be interesting to see how much further this approach takes us. Assuming ANNs resemble the way human brain function you'd also expect them to introduce bugs. And so the actual humans beings would partake in debugging too.
bmc7505 5y ago
I disagree that programming is dying -- tools like Copilot will lead to a Renaissance in the art of computer programming by enabling a larger population to design programs and explore the implications of their design choices. I wrote a short essay [1] on the history automated programming and where I think it is heading in the future. [1]: https://breandan.net/public/programming_with_intelligent_machines.html https://breandan.net/public/programming_with_intelligent_mac...
elwell 5y ago
> writing detailed documentation/specs That's what code is.
qualudeheart 5y ago
You also get to specialize harder. You’ll be able to move into more advanced programming styles. I’m thinking of formally verifiable C/C++ programs for safety critical applications, and code using advanced concepts from programming language theory. The programming languages of the future are going to make Rust look like Python. That’ll be in part because you as an individual programmer aren’t weighed down by as much boilerplate as you were pre-copilot, pre-alphacode and pre- the more advanced coding assistants of the future.

by hmate9 5y ago

wittycardio 5y ago

Solving competitive programming problems is essentially solving hard combinatorial optimization problems. Throwing a massive amount of compute and gradient descent at the problem has always been possible. If I'm not mistaken what this does is reduce the representation of the problem to a state where it can run gradient descent and then tune parameters. The real magic is in finding structurally new approaches. If anything I'd say algorithms and math continue to be the core of programming. The particular syntax or level of abstraction don't matter so much.

chroem- 5y ago
> Solving competitive programming problems is essentially solving hard combinatorial optimization problems. True, but if you relax your hard requirements of optimality to admit "good enough" solutions, you can use heuristic approaches that are much more tractable. High quality heuristic solutions to NP-hard problems, enabled by ML, are going to be a big topic over the next decade, I think.
- wittycardio 5y ago
  I should correct myself, this isn't even that. This is just text analysis on codeforces solutions, which makes it even worse than I thought. Very pessimistic about it's generalizability.
jdlshore 5y ago
> If anything I'd say algorithms and math continue to be the core of programming. I disagree; I think the core of programming is analyzing things people want and expressing solutions to those wants clearly, unambiguously, and in a way that is easy to change in the future. I'd say algorithms and math are a very small part of this work.
- wittycardio 5y ago
  That's not programming, that's called being a good employee. Any person in any role should be doing that. Programming is about algorithms and math. Now a good employee who's in a technical role should have both.
  - jdlshore 5y ago
    > Programming is about algorithms and math. You've simply restated your opinion without providing any supporting arguments, and as I already said, I disagree. The vast majority of programming I see (and as a consultant, I see a fairly wide variety) is not about algorithms and math, but instead gluing together systems and expressing domain logic. Now, I suppose you could argue that domain logic is "algorithms and math," but in my experience, it's less about the specific algorithms and more about precisely describing fuzzy human behavior. It's that "precisely describing" and "easy to change in the future" parts that makes what programmers do different than what any good employee does. (I do agree that there is some programming that is focused on algorithms and math, but it's in the minority, in my experience. Perhaps the type of work you do is focused on algorithms and math, but I believe that's a relatively small part of the software development ecosystem.)
    - wittycardio 5y ago
      No I'm not talking about programming that requires calculations or programs written to solve mathematical problems. Programming at its core is about defining precise logical relationships between abstract objects and then writing algorithms to understand and modify these objects. This is a mathematical process and you should use mathematical thinking to do this.It may not always seem like it when the objects and relationships appear to be simple but that is the core of programming.

mhzsh 5y ago

Creating a higher level abstraction is something people have been trying to do for decades with so-called 4th-generation languages. At some point, abstracting away too much makes a tool too cookie-cutter, and suddenly deviating from it causes more difficulty.

vvilliamperez 5y ago
Read: Ruby on Rails
visarga 5y ago
Maybe it's not more abstraction we need, just automating the drudgery. Abstractions are limited - by definition they abstract things away, they are brittle.

Inufu 5y ago

I agree, I expect programmers will just move up the levels of abstraction. I enjoyed this recent blog post on the topic: https://eli.thegreenplace.net/2022/asimov-programming-and-the-meta-ladder/ https://eli.thegreenplace.net/2022/asimov-programming-and-th...

hackinthebochs 5y ago
The "problem" is that as you move up the levels of abstraction, you need fewer people to do the same amount of work. Unless the complexity of the work scales as well. I've always felt that programmers would be the first class of knowledge workers to be put out of work by automation. This may be the beginning of the end for the programming gravy train.
- bmh100 5y ago
  On the other hand, as the value of an hour of programming increases, the quantity demanded may also increase.
- hmate9 5y ago
  Or you can do things at a faster pace and increase your productivity.
- NicoJuicy 5y ago
  There aren't enough developers either way.
- Inufu 5y ago
  Yes, this is how you increase prosperity (see: agricultural revolution, industrial revolution, etc). You can now create more with the same number of people.
- paxys 5y ago
  > as you move up the levels of abstraction, you need fewer people to do the same amount of work Yes, but the total amount of work (and surrounding complexity) also increases with it. Just look at the evolution of the software industry over the last few decades.

streetcat1 5y ago

First, If this is correct, if alpha code succeeded, this will bring to its own demise. I.e. as soon as it starts replacing humans, it will not have enough human generated training data, since all of programming will be done by models like himself. Second, alphacode was specifically trained for competitive programming : 1. short programs. 2. Each program has 100's of human generated solutions. However, commercial program are: 1. long. 2. Have no predefined answer or even correct answer. 3. Need to use/reuse a lot of legacy code.

chroem- 5y ago
Reinforcement learning and adversarial training can render both of those concerns as non-issues in practice.
- ialyos 5y ago
  The phrase "in practice" doesn't really work when you're referring to highly finicky strategies like RL and adversarial training
AnIdiotOnTheNet 5y ago
> as soon as it starts replacing humans, it will not have enough human generated training data, since all of programming will be done by models like himself. As a natural born pessimist, I can't help but feel that by the time we get to that point we'll just keep blundering forward and adapting our world around the wild nonsense garbage code the model ends up producing in this scenario. After all, that's basically what we've done with the entire web stack.

diehunde 5y ago

My bet would be that it will never happen in a reasonable time frame. And also by that logic, writing that "documentation/spec" would just mean learning a new programming language the AI engine can parse making it as useful as a compiler. Anyone who has been writing and designing software for a while knows the cycle is way more complex than take some input and write code. Let me know when the AI engine is able to do complex refactoring or adding features that keeps backwards compatibility, find a bug in a giant codebase by debugging a test case or write code that's performant but also maintainable.

fvold 5y ago
I hear that. Machine Learning also has a long way to go before it can take a long, rambling mess of a meeting and somehow generate a halfway usable spec from it. I mean, the customer says they want X, but X is silly in this context, so we'll give them Y and tell them it's "X-like, but faster". For example, SQL is "Blockchain-like, but faster" for a lot of buzzword use-cases of blockchain.
Enginerrrd 5y ago
I agree, from a totally different angle. Let's take something I know better as an example: Structural engineering. Structural engineering should be a "solved problem". It seems, ostensibly, relatively simple compared to a more open ended activity like "programming".(For "technical reasons", it ends up being more similar than you might think.) Still, you are ultimately dealing with the same materials, the same physics, and very similar configurations. And yet, despite the fact that we have programs to help calculate all the things, test code-required load-combinations, even run simulations and size individual components... it turns out that, it doesn't actually save that much work, and you still need an engineer to do most of it. And not just because of regulatory requirements. It's just, that's not the hard part. The hard part is assembling the components and specifications, specifying the correct loads based on location-specific circumstances, coming up with coherent and sensible design ideas, chasing down every possible creative nook and cranny of code to make something that was originally a mistake actually work, and know when the model is just wrong for some reason and the computer isn't simulating load paths accurately. Specifying the inputs and interpreting results is still about as much work as it was before you started with all the fancy tools. Those tools still have advantages mind you, and they do make one slightly more efficient. Substantially so in some cases, but most of the time it still comes out as a slight assist rather than a major automation.

pjmorris 5y ago

I'd note that assembly, C, and Python didn't replace 'programming' but were expected to do so. I'd wager that what you now call 'detailed documentation/specs' will still be called programming in 10 or even 20 years.

falcor84 5y ago
If you could change a sentence in the documentation and then run a ~1min compilation to see the resulting software, it would be a very different kind of programming. I suppose it'll give a new meaning to Readme-Driven-Development.

62951413 5y ago

Model-driven development and code generation from UML were once supposed to be the future. It will be interesting to see how much further this approach takes us. Assuming ANNs resemble the way human brain function you'd also expect them to introduce bugs. And so the actual humans beings would partake in debugging too.

bmc7505 5y ago

I disagree that programming is dying -- tools like Copilot will lead to a Renaissance in the art of computer programming by enabling a larger population to design programs and explore the implications of their design choices. I wrote a short essay [1] on the history automated programming and where I think it is heading in the future. [1]: https://breandan.net/public/programming_with_intelligent_machines.html https://breandan.net/public/programming_with_intelligent_mac...

elwell 5y ago

> writing detailed documentation/specs That's what code is.

qualudeheart 5y ago

You also get to specialize harder. You’ll be able to move into more advanced programming styles. I’m thinking of formally verifiable C/C++ programs for safety critical applications, and code using advanced concepts from programming language theory. The programming languages of the future are going to make Rust look like Python. That’ll be in part because you as an individual programmer aren’t weighed down by as much boilerplate as you were pre-copilot, pre-alphacode and pre- the more advanced coding assistants of the future.