4 ms·

I don't think the halting problem's got anything to do with it, because I certainly also can't prove, in general, whether some code I have written halts or not

I don't think the halting problem's got anything to do with it, because I certainly also can't prove, in general, whether some code I have written halts or not either but that hasn't held me back from churning out vast quantities of text that, at first glance, might also resemble good code. But I agree with your sentiment: to get AI to write the code you want, you will need an engineer to work with it. I do expect that future AI code generation models will learn more about a specific code base over time, partially overcoming domain specific knowledge gaps.

heavyset_go 4y ago
You can reason why or why not you believe your code is correct, and ultimately come to an understanding if your reasoning proves correct or incorrect, but AI is just working off of patterns from its data sets.
- sciolist 4y ago
  LLM's aren't just "matching patters", they demonstrate emergent behavior, allowing them to do much more than what they were explicitly trained on. At the moment AI models might not be able to utilize "train of thought reasoning" like we can, however the distance between these emergent behaviors and "reasoning" is much shorter than we realize (and will continue to converge in the future).
  - dinkumthinkum 4y ago
    So, do you really think these computational AI systems can reason and solve problems equivalently or in a manner superior to humans? If they aren’t matching patterns, which are I think the literature makes clear they are, why does ChatGPT fail at solving logic problems so badly? It is already performing impressive enough at tasks that people think it can replace all knowledge but yet if fails miserably at logic problems and even those that claim it replaces google do not really seems to realize it fails at providing answers to basic facts. I really don’t see any evidence that this type of AI, which is quite advanced for what it is, is type of AI that dissolves technical education / jobs. Another question, if this is on the road to being human intelligence, how is it someone like Ramanujan received no formal schooling and yet could create solutions that seem very far off from the capabilities of these systems which we are claiming will replace humans? Further, if it’s intelligence is so superior, why does it require so many watts of power produce so much thermal by-product when beings that actually do perform in a superior way currently require little more than a bowl of rice every few days to continue solving problems? Is there not sone profound, “emergent,” difference in sophistication?
    - Uehreka 4y ago
      I think what you’re missing in a lot of these comments is that AI doesn’t need to operate at the peak of human intelligence like Ramanujan in order to absolutely gut a lot of job markets. Like, most developers aren’t writing proofs that a series converges to pi or building Apollo modules where everything has to work perfectly. I’m not gonna act like what we do is easy, because it’s not. However I’ll never forget the day I wrote a WebGL shader in GLSL with Copilot’s help, and then clicked over from a GLSL file to a JS file and Copilot immediately wrote the code to create a THREE.ShaderMaterial, import my shader (with the right file path) and filled out the uniform names and types perfectly on the first try. It took under 2 seconds, and at the datacenter maybe a fistful a watts, and it left me fucking speechless. I’ll admit it’s pretty weird that these models can write working code in a multi-language codebase but can’t add two four digit numbers. They’re almost savant-like the way they fail at basic tasks while excelling at more complex ones. But given that they are in fact excelling at complex tasks, I’m worried about what could happen if, say, every enterprise dev shop laid off 60% of their headcount and kept the remaining 40% just to do code review. That would be very bad for everyone in the industry.
      - sciolist 4y ago
        Agreed. The article also speaks about engineers using these LLMs still as a tool - not necessarily replacing humans, but humans wielding them as tools. Copilot is a great example. Dinkum raised a great point about the models spitting out falsities or having other issues. We still will have and need human-in-the-loop engineering so that we can discern which outputs are true (or not)!
      - dinkumthinkum 4y ago
        I’m not missing that. The point about Ramanujan is to show there is something fundamentally different and at a massive scale of difference between human intelligence and these AI systems. It’s not that average developers are math prodigies. It is generating code based on predictions from a large corpus of training data. How well does it do at debugging that code (assuming the bugs / code it finds are not the exact kinds posted to Web sites thousands times)?
- codebje 4y ago
  I am interested in the convergence of AI-generated code and programs-as-proofs: can I work together with AI using a language with a highly expressive type system such that we co-develop a specification in types and properties, then co-develop the implementations that fit the specification. It's not practical to do that sort of thing alone, as there's a lot of tedious nonsense involved. Maybe it will be more practical with an AI assistant. Whether current AIs are more or less somewhat close to the basics of how humans might be approaching code writing is more of a philosophical point, but a "translation room" view of the human mind might on occasion make me feel like I'm just working off patterns from my data sets. :-)

heavyset_go 4y ago

You can reason why or why not you believe your code is correct, and ultimately come to an understanding if your reasoning proves correct or incorrect, but AI is just working off of patterns from its data sets.

sciolist 4y ago
LLM's aren't just "matching patters", they demonstrate emergent behavior, allowing them to do much more than what they were explicitly trained on. At the moment AI models might not be able to utilize "train of thought reasoning" like we can, however the distance between these emergent behaviors and "reasoning" is much shorter than we realize (and will continue to converge in the future).
- dinkumthinkum 4y ago
  So, do you really think these computational AI systems can reason and solve problems equivalently or in a manner superior to humans? If they aren’t matching patterns, which are I think the literature makes clear they are, why does ChatGPT fail at solving logic problems so badly? It is already performing impressive enough at tasks that people think it can replace all knowledge but yet if fails miserably at logic problems and even those that claim it replaces google do not really seems to realize it fails at providing answers to basic facts. I really don’t see any evidence that this type of AI, which is quite advanced for what it is, is type of AI that dissolves technical education / jobs. Another question, if this is on the road to being human intelligence, how is it someone like Ramanujan received no formal schooling and yet could create solutions that seem very far off from the capabilities of these systems which we are claiming will replace humans? Further, if it’s intelligence is so superior, why does it require so many watts of power produce so much thermal by-product when beings that actually do perform in a superior way currently require little more than a bowl of rice every few days to continue solving problems? Is there not sone profound, “emergent,” difference in sophistication?
  - Uehreka 4y ago
    I think what you’re missing in a lot of these comments is that AI doesn’t need to operate at the peak of human intelligence like Ramanujan in order to absolutely gut a lot of job markets. Like, most developers aren’t writing proofs that a series converges to pi or building Apollo modules where everything has to work perfectly. I’m not gonna act like what we do is easy, because it’s not. However I’ll never forget the day I wrote a WebGL shader in GLSL with Copilot’s help, and then clicked over from a GLSL file to a JS file and Copilot immediately wrote the code to create a THREE.ShaderMaterial, import my shader (with the right file path) and filled out the uniform names and types perfectly on the first try. It took under 2 seconds, and at the datacenter maybe a fistful a watts, and it left me fucking speechless. I’ll admit it’s pretty weird that these models can write working code in a multi-language codebase but can’t add two four digit numbers. They’re almost savant-like the way they fail at basic tasks while excelling at more complex ones. But given that they are in fact excelling at complex tasks, I’m worried about what could happen if, say, every enterprise dev shop laid off 60% of their headcount and kept the remaining 40% just to do code review. That would be very bad for everyone in the industry.
    - sciolist 4y ago
      Agreed. The article also speaks about engineers using these LLMs still as a tool - not necessarily replacing humans, but humans wielding them as tools. Copilot is a great example. Dinkum raised a great point about the models spitting out falsities or having other issues. We still will have and need human-in-the-loop engineering so that we can discern which outputs are true (or not)!
    - dinkumthinkum 4y ago
      I’m not missing that. The point about Ramanujan is to show there is something fundamentally different and at a massive scale of difference between human intelligence and these AI systems. It’s not that average developers are math prodigies. It is generating code based on predictions from a large corpus of training data. How well does it do at debugging that code (assuming the bugs / code it finds are not the exact kinds posted to Web sites thousands times)?
codebje 4y ago
I am interested in the convergence of AI-generated code and programs-as-proofs: can I work together with AI using a language with a highly expressive type system such that we co-develop a specification in types and properties, then co-develop the implementations that fit the specification. It's not practical to do that sort of thing alone, as there's a lot of tedious nonsense involved. Maybe it will be more practical with an AI assistant. Whether current AIs are more or less somewhat close to the basics of how humans might be approaching code writing is more of a philosophical point, but a "translation room" view of the human mind might on occasion make me feel like I'm just working off patterns from my data sets. :-)