4 ms·

The point of the article (and your point) seems to be that programming languages are now obsolete, and will be secondary to English prompting of LLMs. So the ch

The point of the article (and your point) seems to be that programming languages are now obsolete, and will be secondary to English prompting of LLMs. So the choice of language no longer matters, GPTs will write it all. And I'm just saying, having hooked up ChatGPT to my python interpreters, my text editor, my IDE - we're definitely not there yet, and I'm not totally convinced there aren't some fundamental limits to the fixed-context next-token-prediction paradigm. You have to be very precise and technical in your prompts, you have to be working on toy isolated problems, and you have to watch it like a hawk and fix a lot of subtle errors (it's a good mimic, which has the unfortunate effect of making its errors harder to spot). And, it doesn't (yet) have a visual interface to see what e.g. a web app is doing interactively, click its buttons, see "soft" bugs or confusing UI aspects, poke around in dev tools, etc. I'm sure that's coming (researchers are working on these multimodal models), but I have my doubts we're going to just turn over trust fully to the next generation of LLMs and let them write everything in brainfuck or whatever. Not trying to poo-poo LLMs or score cheap HN dunks here, this stuff is amazing - but the hype has gotten a little disconnected from reality, so some balance is good I think.

kbourgoin 4y ago
Hey there. My point was close to that, but not quite as drastic as saying languages themselves are obsolete. It's more that less developer friendly languages look more appealing if an LLM can paper over the parts that you don't like. I've been working in Python for a long time because I enjoy writing it. If Copilot starts writing more and more, does that quality matter as much? I've always really disliked Java because it's so verbose. Maybe it's not so bad with copious code generation via LLM. And, yeah, ChatGPT is impressive in lots of ways, but writing code isn't one of them. I look forward to trying GPT4 but I'm not holding my breath. I've found Copilot to be much more impressive and useful. You have to know what you want to write, and double check everything, but it's still a big difference. I'm not switching to brainfuck any time soon, but I'd be more open to languages I used to turn my nose up at. I don't think these things will change overnight, but I really do think the things people value in a language are going to shift over the next few years and that ergonomics/aesthetics (the bits I think of as "the language") just aren't going to matter as much.
- QuadrupleA 4y ago
  They're a good way of delving into less familiar languages, for sure. But if they're creating subtle bugs, the human operator will have to read over and understand the language really well anyway. So a less ergonomic language becomes a problem. Some parallels here to the self driving problem - in order for people to trust their personal safety / codebases to these things, they're going to have to prove a very high degree of reliability. ChatGPT's definitely not there yet. I've heard mixed reports about GPT4, curious to get my hands on it too. And then going from letting it drive your car / write your code, to total hands-free mode where you don't even know how to drive anymore or understand the programming language it's using, that's a big leap.
abraxas 4y ago
No, my point is not that programming languages are obsolete. Rather that they are shifting one layer down. Just like there is room for different virtual machines today (eg. JVM vs V8 etc) there will be room for different computer languages. They just won't be used primarily as a way to write code but rather as an intermediate representation. Just like you rarely need to look at raw Java bytecode or transpiled Javascript you will rarely look at raw Python or Rust code. Those will be human readable but rarely read by humans intermediate representations provided by LLM augmented 5th generation programming tools.
- QuadrupleA 4y ago
  It's an interesting thought - but with the imprecise/random nature of LLMs, I wonder if we could tolerate something so inherently faulty / unreliable / human-like in place of a compiler. If real compilers spit out subtle machine language bugs / mistranslations of code even 1% of the time, we'd be in big trouble.
  - abraxas 4y ago
    This may be at least partially circumvented by lowering the temperature of token generation. Otherwise, a strong set of tests should be generated to minimize the impact. The tooling is going to look very different. Every time we move up an abstraction layer we give up some guarantees from the underlying platform. The move to the cloud forced us to give up on having deterministic hardware. This is going to be a shift in that same direction. The lack of determinism causes problems to be sure but the payback is so great that it's likely to be a worthy tradeoff.

kbourgoin 4y ago

Hey there. My point was close to that, but not quite as drastic as saying languages themselves are obsolete. It's more that less developer friendly languages look more appealing if an LLM can paper over the parts that you don't like. I've been working in Python for a long time because I enjoy writing it. If Copilot starts writing more and more, does that quality matter as much? I've always really disliked Java because it's so verbose. Maybe it's not so bad with copious code generation via LLM. And, yeah, ChatGPT is impressive in lots of ways, but writing code isn't one of them. I look forward to trying GPT4 but I'm not holding my breath. I've found Copilot to be much more impressive and useful. You have to know what you want to write, and double check everything, but it's still a big difference. I'm not switching to brainfuck any time soon, but I'd be more open to languages I used to turn my nose up at. I don't think these things will change overnight, but I really do think the things people value in a language are going to shift over the next few years and that ergonomics/aesthetics (the bits I think of as "the language") just aren't going to matter as much.

QuadrupleA 4y ago
They're a good way of delving into less familiar languages, for sure. But if they're creating subtle bugs, the human operator will have to read over and understand the language really well anyway. So a less ergonomic language becomes a problem. Some parallels here to the self driving problem - in order for people to trust their personal safety / codebases to these things, they're going to have to prove a very high degree of reliability. ChatGPT's definitely not there yet. I've heard mixed reports about GPT4, curious to get my hands on it too. And then going from letting it drive your car / write your code, to total hands-free mode where you don't even know how to drive anymore or understand the programming language it's using, that's a big leap.

abraxas 4y ago

No, my point is not that programming languages are obsolete. Rather that they are shifting one layer down. Just like there is room for different virtual machines today (eg. JVM vs V8 etc) there will be room for different computer languages. They just won't be used primarily as a way to write code but rather as an intermediate representation. Just like you rarely need to look at raw Java bytecode or transpiled Javascript you will rarely look at raw Python or Rust code. Those will be human readable but rarely read by humans intermediate representations provided by LLM augmented 5th generation programming tools.

QuadrupleA 4y ago
It's an interesting thought - but with the imprecise/random nature of LLMs, I wonder if we could tolerate something so inherently faulty / unreliable / human-like in place of a compiler. If real compilers spit out subtle machine language bugs / mistranslations of code even 1% of the time, we'd be in big trouble.
- abraxas 4y ago
  This may be at least partially circumvented by lowering the temperature of token generation. Otherwise, a strong set of tests should be generated to minimize the impact. The tooling is going to look very different. Every time we move up an abstraction layer we give up some guarantees from the underlying platform. The move to the cloud forced us to give up on having deterministic hardware. This is going to be a shift in that same direction. The lack of determinism causes problems to be sure but the payback is so great that it's likely to be a worthy tradeoff.