4 ms·
The opening of the article derives from (or at least relates to) Peter Naur's classic 1985 essay "Programming as Theory Building". (That's Naur of Algol and BNF
by dang 2y ago
The opening of the article derives from (or at least relates to) Peter Naur's classic 1985 essay "Programming as Theory Building". (That's Naur of Algol and BNF btw.)
Naur argued that complex software is a shared mental construct that lives in the minds of the people who originally build it. Source code and documentation are lossy representations of the program—lossy because the real program (the 'theory' behind the code) can never be fully reconstructed from them.
Legacy code here would mean code where you still have the artifacts (source code and documentation), but have lost the theory, because the original builders have left the team. That means you've lost access to the original program, and can only make patchwork changes to the software rather than "deep improvements" (to quote the OP). Naur gives some vivid examples of this in his essay.
What this means in the context of LLMs seems to me an open question. In Naur's terms, do LLMs necessarily lack the theory of a program? It seems to me there are other possibilities:
* LLMs may already have something like a 'theory' when generating code, even if it isn't obvious to us
* perhaps LLMs can build such a theory from existing codebases, or will be able to in the future
* perhaps LLMs don't need such a theory in the way that human teams do
* if a program is AI-generated, then maybe the AI has the theory and we don't!
* or maybe there is still a theory, in Naur's sense, shared by the people who write the prompts, not the code.
There was an interesting recent article and thread about this:
Naur's "Programming as Theory Building" and LLMs replacing human programmers - https://news.ycombinator.com/item?id=43818169 https://news.ycombinator.com/item?id=43818169 - April 2025 (129 comments)
- krrishd 2y agoTIL / unconscious reference on my part, super interesting! Link for the curious: https://pages.cs.wisc.edu/~remzi/Naur.pdf https://pages.cs.wisc.edu/~remzi/Naur.pdf
- dang 2y agoWith lots of HN threads over the years! Naur's "Programming as Theory Building" and LLMs replacing human programmers - https://news.ycombinator.com/item?id=43818169 https://news.ycombinator.com/item?id=43818169 - April 2025 (129 comments) Programming as Theory Building (1985) [pdf] - https://news.ycombinator.com/item?id=42592543 https://news.ycombinator.com/item?id=42592543 - Jan 2025 (44 comments) Programming as Theory Building (1985) - https://news.ycombinator.com/item?id=38907366 https://news.ycombinator.com/item?id=38907366 - Jan 2024 (12 comments) Programming as Theory Building (1985) [pdf] - https://news.ycombinator.com/item?id=37263121 https://news.ycombinator.com/item?id=37263121 - Aug 2023 (36 comments) Programming as Theory Building (1985) [pdf] - https://news.ycombinator.com/item?id=33659795 https://news.ycombinator.com/item?id=33659795 - Nov 2022 (1 comment) Naur on Programming as Theory Building (1985) [pdf] - https://news.ycombinator.com/item?id=31500174 https://news.ycombinator.com/item?id=31500174 - May 2022 (4 comments) Naur on Programming as Theory Building (1985) [pdf] - https://news.ycombinator.com/item?id=30861573 https://news.ycombinator.com/item?id=30861573 - March 2022 (3 comments) Programming as Theory Building (1985) - https://news.ycombinator.com/item?id=23375193 https://news.ycombinator.com/item?id=23375193 - June 2020 (35 comments) Programming as Theory Building (1985) [pdf] - https://news.ycombinator.com/item?id=20736145 https://news.ycombinator.com/item?id=20736145 - Aug 2019 (11 comments) Peter Naur – Programming as Theory Building (1985) [pdf] - https://news.ycombinator.com/item?id=10833278 https://news.ycombinator.com/item?id=10833278 - Jan 2016 (15 comments) Naur’s “Programming as Theory Building” (2011) - https://news.ycombinator.com/item?id=7491661 https://news.ycombinator.com/item?id=7491661 - March 2014 (14 comments) Programming as Theory Building (by Naur of BNF) - https://news.ycombinator.com/item?id=121291 https://news.ycombinator.com/item?id=121291 - Feb 2008 (2 comments)
- akoboldfrying 2y ago> * perhaps LLMs can build such a theory from existing codebases, or will be able to in the future If the source code is a lossy representation of the theory, doesn't that answer this question with a conclusive "No"? (If there is anything -- LLM or not -- that can do this, then the source code is not lossy after all.) EDIT: Clarify last sentence.
- dang 2y agoI guess that's right, but by "such a theory" I didn't mean the original theory - I meant maybe they can figure a different one out that works well enough, at least for the LLM. This is all handwavey, of course.