3 ms·

And brain surgery is just cutting someone's head open with a scalpel. Programming languages are stupidly information dense. I'm highly skeptical an AI will be u

And brain surgery is just cutting someone's head open with a scalpel. Programming languages are stupidly information dense. I'm highly skeptical an AI will be useful for non-trivial boiler plate any time soon, they lack the reasoning powers to get the details right even occasionally for common tasks and basically never for uncommon ones.

dmreedy 4y ago
While I take your meaning here, I would suggest also that natural languages are even more stupidly information-dense. And while the rules of their logic are not as hard and immutable as the semantics of programming languages, they do have plenty.
- packetlost 4y ago
  I don't agree that natural language is more information dense than most programming languages, there's a bunch of shared context, work, and assumptions that must be understood before effective use of both, but natural language is typically pretty loose and relies heavily on context for real meaning, which is still subjective. Conversely, PLs have no subjectivity once executed/compiled. Natural language is very verbose when trying to express the concepts of a PL with the same specificity, just look at how long entry-level tutorials for writing hello world in any PL are. Or try explaining all the details of even a basic function call in C to a CS101 class. You need to say a lot of words to describe the same exact thing. PLs benefit from being targeted at a specific domain dramatically that NL just can't.
koboll 4y ago
Programming languages are stupidly information dense indeed, but if that's the most major roadblock, I don't see why throwing more compute at the problem won't eventually solve it. GPT-4 is already handling a very stupid amount of information density.
- packetlost 4y ago
  My point isn't the information density, it's the fact that every detail matters. LLMs aren't great at details or facts and therefore aren't really suited for writing software right now. It's not a computational limit, it's intrinsic to how LLMs work.

dmreedy 4y ago

While I take your meaning here, I would suggest also that natural languages are even more stupidly information-dense. And while the rules of their logic are not as hard and immutable as the semantics of programming languages, they do have plenty.

packetlost 4y ago
I don't agree that natural language is more information dense than most programming languages, there's a bunch of shared context, work, and assumptions that must be understood before effective use of both, but natural language is typically pretty loose and relies heavily on context for real meaning, which is still subjective. Conversely, PLs have no subjectivity once executed/compiled. Natural language is very verbose when trying to express the concepts of a PL with the same specificity, just look at how long entry-level tutorials for writing hello world in any PL are. Or try explaining all the details of even a basic function call in C to a CS101 class. You need to say a lot of words to describe the same exact thing. PLs benefit from being targeted at a specific domain dramatically that NL just can't.

koboll 4y ago

Programming languages are stupidly information dense indeed, but if that's the most major roadblock, I don't see why throwing more compute at the problem won't eventually solve it. GPT-4 is already handling a very stupid amount of information density.

packetlost 4y ago
My point isn't the information density, it's the fact that every detail matters. LLMs aren't great at details or facts and therefore aren't really suited for writing software right now. It's not a computational limit, it's intrinsic to how LLMs work.