3 ms·

I'm not sure the author understands how AI learns. At this point, the fundamental "coding" knowledge is baked in, leaving little to "learn" from more code examp

I'm not sure the author understands how AI learns. At this point, the fundamental "coding" knowledge is baked in, leaving little to "learn" from more code examples. The real power that coding models have comes from reasoning, testing code, and having broader integration into hardware to develop it. In other words, we've already zipped past the transfer learning phase of learning how to code.

emdashslop 2mo ago
Perhaps, but IMHO that’s too strong a claim. I agree that better reasoning, testing, and agentic tooling matter, but coding models still benefit from new, high-quality code and real-world coding tasks. Software also keeps changing: new APIs, frameworks, languages, vulnerabilities, and engineering patterns appear constantly. I'm skeptical that “Coding knowledge is already baked in” is supported by the evidence. What is the evidence that the models are already past the "transfer learning phase of learning how to code" more than they are past the "transfer learning phase of learning how to produce high quality animation or music?". The fact that code is produced is not enough evidence that "machines can't learn more from human work". In other words: What evidence shows that code has uniquely reached data saturation? As far as I know, existing controlled studies still show gains from additional and better-targeted code training and I'm not aware of studies that demonstrate the opposite.
- dpforesi 2mo ago
  That's fair. My point is that most of what coding models need to learn to write code proficiently is largely baked in. things like syntax, useful code idioms, data structures and algorithms, important engineering concepts and design patterns etc... are already learned. Code, generally is easier to "bake in" in some ways than creative work because it is highly structured with known and absolute rules, whereas creative work thrives on the surfaces where rules and conventions are violated in some way. I work a lot with Claude Code, not just building, but analyzing how it builds and it is at the point where its failures largely arise from a misunderstanding of my instructions or implied goals, and in some cases a failure to understand basic reality, not on syntax or logic. In other words, it implemented the wrong idea really well (some clean up by humans required assuming any of it is useful). That said, the creative concept needs to come from the human. Case in point. I have a simple python based synthetic data generator. Claude uses it inefficiently because it is a new-ish repo it has not seen so it reads the lengthy README and ~60-70% of the code in the repo to learn it before using it. I had 2 solutions to this. One, create a CLI or MCP interface for models, or two, just create a skill file. I presented only the first idea and it returned only that it was possible if difficult given the complexity of how one creates influences on certain classes in the data. It COULD HAVE just said "It would be a lot easier to make a skill" but it lacked the creative capacity for this leap i guess. I jumped right to the skill idea two paragraphs into the explanation of the core challenges of creating an MCP and did not bother to read on before I suggested a skill. That is the current code barrier, not necessarily understanding syntax etc...

emdashslop 2mo ago

Perhaps, but IMHO that’s too strong a claim. I agree that better reasoning, testing, and agentic tooling matter, but coding models still benefit from new, high-quality code and real-world coding tasks. Software also keeps changing: new APIs, frameworks, languages, vulnerabilities, and engineering patterns appear constantly. I'm skeptical that “Coding knowledge is already baked in” is supported by the evidence. What is the evidence that the models are already past the "transfer learning phase of learning how to code" more than they are past the "transfer learning phase of learning how to produce high quality animation or music?". The fact that code is produced is not enough evidence that "machines can't learn more from human work". In other words: What evidence shows that code has uniquely reached data saturation? As far as I know, existing controlled studies still show gains from additional and better-targeted code training and I'm not aware of studies that demonstrate the opposite.

dpforesi 2mo ago
That's fair. My point is that most of what coding models need to learn to write code proficiently is largely baked in. things like syntax, useful code idioms, data structures and algorithms, important engineering concepts and design patterns etc... are already learned. Code, generally is easier to "bake in" in some ways than creative work because it is highly structured with known and absolute rules, whereas creative work thrives on the surfaces where rules and conventions are violated in some way. I work a lot with Claude Code, not just building, but analyzing how it builds and it is at the point where its failures largely arise from a misunderstanding of my instructions or implied goals, and in some cases a failure to understand basic reality, not on syntax or logic. In other words, it implemented the wrong idea really well (some clean up by humans required assuming any of it is useful). That said, the creative concept needs to come from the human. Case in point. I have a simple python based synthetic data generator. Claude uses it inefficiently because it is a new-ish repo it has not seen so it reads the lengthy README and ~60-70% of the code in the repo to learn it before using it. I had 2 solutions to this. One, create a CLI or MCP interface for models, or two, just create a skill file. I presented only the first idea and it returned only that it was possible if difficult given the complexity of how one creates influences on certain classes in the data. It COULD HAVE just said "It would be a lot easier to make a skill" but it lacked the creative capacity for this leap i guess. I jumped right to the skill idea two paragraphs into the explanation of the core challenges of creating an MCP and did not bother to read on before I suggested a skill. That is the current code barrier, not necessarily understanding syntax etc...