Sure, there are models and variations whose power is not good enough to approximate the problems we are talking about here - most of them trading speed for powe

Sure, there are models and variations whose power is not good enough to approximate the problems we are talking about here - most of them trading speed for power on purpose because of the enormous costs involved. This is actually one of the reasons i think the author's claim is so silly. They don't claim it will never be practical, they claim it will never be. For the purposes of this discussion, I don't claim you can make a practical LLM that understands code or simulates it. Only that you can make one at all :) I actually do think you can make a practical and useful one for a simple reason - while all the edge cases are fun to come up with, and the amount/kinds of code one could write is clearly infinite, code is closer to literary works or musical compositions in how that plays out - daily WTF aside, we don't actually write all possible code, and even to the degree you do, you don't really have to handle it all to be practical or useful. IE The set of music that you can create and books you can write is similarly infinite. However, the set of music that "sounds right" (go with me) and books that "read right" is not anywhere near as huge in practice. This isn't just semantics/syntax. Code is similar - while the possibility is infinite, you don't need to handle any possible thing to be very useful - you just need to be able to handle some part of the spectrum from "worlds shittiest code or most broken code" to "worlds highest quality or most elegant code". To me, this is a much more tractable approximation problem than "handle anything anyone ever throws at you" The same way i think it's a more tractable approximation problem to say "make something that sounds good to the average pop music listener" vs "make an original and unique composition of music that sounds good to everyone who hears it" or whatever. (I'm not being super precise, but i expect you get the point :P)