6 ms·

Python is locally readable. Reasoning about larger systems in Python is where things get really hard, because you have to describe how many small individually r

Python is locally readable. Reasoning about larger systems in Python is where things get really hard, because you have to describe how many small individually readable things interact with each other in a very limited vocabulary.

ant6n 5mo ago
That’s true. Once you have APIs and want to use classes to create larger structures, the language is full of warts.
- cturner 5mo ago
  I have built large systems on python that use classes, for more than ten years. I came to it from Java, ten years. As a rule, I avoid implementation inheritance. Occasionally I need to facade a library that assumes implementation inheritance to avoid it spreading into my codebase. When the codebase hits a certain size, I hand-roll some decorators to create functionality like java interfaces. With that done, and a suite of acceptance tests, I find it scales up well.
bryanrasmussen 5mo ago
hmm, yeah given LLM's ability to churn out lots of code quickly and be overly verbose in that code that is a potential downside. That it could in a quick one time edit create so much intellectual overhead that Python might be the wrong language to understand what is going on. What language do you feel is easier to reason about in the large?
- hiAndrewQuinn 5mo ago
  Haskell would be my vote, and Rust too, actually, both because of their very strong type systems. The type system lets you very quickly figure out what something is before you figure out what something does, and it turns out that separating those two concerns as hard as those two languages do often results in doing the whole one-two punch faster.
  - lukan 5mo ago
    Haskell does not qualify for a large training set, though. (Nor for readability in my opinion) I think I have never seen haskell software made wih LLM's but well, aside from university, I have not seen Haskell code at all. (Also Haskell purists I would associate with people who avoid LLM's) I would rather go with Rust given these choices. But I have good results with typescript (or javascript for simpler things). Really large set of examples. Tools optimized for it, agents debugging in the browser works allmost out of the box. And well, a elaborate typesystem.
    - yakshaving_jgt 5mo ago
      [dead]
    - klodolph 5mo ago
      I used Claude to generate Haskell and it works really well. Claude struggles sometimes with respecting abstraction boundaries, but Haskell enforces parts of those boundaries in its type system better than a lot of other languages (if a module can’t do IO, for example). Works well, in my experience. Sometimes the agent does weird stuff that you have to rewrite, but I get the sense that this happens in any language. Maybe Haskell’s training set is not large enough, but it seems to work despite the smaller training set.
    - co_dh 5mo ago
      I used Claude that created a terminal based table viewer from rust first, to lean , and finally to Haskell. https://github.com/co-dh/tv-hask/tree/main https://github.com/co-dh/tv-hask/tree/main I give up rust because it’s not functional enough. There aren’t many things Claude can prove about a table viewer, and Haskell fits very well, and have enough libraries. Claude is pretty good at Haskell. I barely write Haskell before but I do know monad.
    - mightybyte 5mo ago
      How much code do you think is necessary for LLMs to be good enough?
  - jorvi 5mo ago
- harperlee 5mo ago
  I'd say Java, because it has a massive footprint amenable for training, and a strong type system (does not have sum types though and those are trendy). You'd have to steer the LLM to use the style you want, and not massively overarchitect things though, but that's going to be an issue nonetheless.
  - mands 5mo ago
    Java has sum types - they are fairly recent, called sealed records, and can be exhaustively pattern matched on. (I do agree however, Java is a great target for LLMs)
- jimmaswell 5mo ago
  C# is as close to an ideal language as you can get for most things IMO. I find AI does a great job with it.
  - pjerem 5mo ago
    I do agree. C# is an hidden gem for IA. There are not that much different ways to get somewhere so the model have probably been trained on the framework and libraries everybody uses (the Microsoft ones). Compared to most languages, including Java, C# will have a hard time letting you compile incoherent code. You barely need any dependencies other than aspnetcore and efcore for most applications and your AI knows them well. It’s easy to do TDD with it so it’s easy to keep your IA from hallucinating.
    - kuboble 5mo ago
      I definitely agree with the sentiment. However this part. > There are not that much different ways to get somewhere This is far from true. C# is a language where you can operate on the raw pointers through unsafe keyword. On the other end of the spectrum, you can have duck-typing in dynamic blocks. For operating on collections you can use old style loops, or chain of lambdas or sql like syntax. I have been coding in C# old school way for most of my life at this point, and I feel like I'm in a foreign land reading code from some other C# projects.
  - HumblyTossed 5mo ago
    I like C#, it's how I make a living, but it's way too large today. I can program in valid C# and it looks like C or I can program in C# and it looks like a functional language or I can program in C# and it's looks all angle-brakety like C++. The problem with that is everyone has an opinion on what good C# looks like. For personal projects, I'll take a much simpler language any day.
    - bonesss 5mo ago
      C# has recreated the C++ dialect conundrum. For some it’s effectively an idempotent functional language with unfortunate failings of exhaustiveness, for others it’s Java ca 2009, for others it’s C++ but not quite. Discipline, effort, linters, reviews, more discipline, more effort, retraining, discipline… and foot guns everywhere because so much of the adaptation has been a 95% solution. Personally I got everything C# promises even now when F# was dropped years ago and have found the interim pretty annoying.
- barkingcat 5mo ago
  get LLM to write ADA and have it use SPARK for verification.
bazoom42 5mo ago
For larger systems you create your own modules and abstractions, so comprehensibility at higher level does not depend so much on the language.
- sundarurfriend 5mo ago
  The tools the language gives you to create those abstractions make a lot of difference, however.
  - mbreese 5mo ago
    But every abstraction that an LLM has to write is a choice. Your way of writing Python may not match that choice. The next run of the agent might not choose the same way. Because the language gives you many different tools, an LLM generated codebase can get inconsistent and overly complicated quickly. The flexibility of Python is a downside when you’re having an LLM generate the code. If you’re working in an existing codebase, it’s great - those choices were already made and it can match your style. When an LLM has to derive its own style is when things can devolve into a jumbled mess.
    - andyferris 5mo ago
      To me applying LLMs to a python (or similarly dynamic) code base where it’s currently spaghetti and monkey patched, it can miss things just like I can. But… I have to admit Opus 4.7 has been very pragmatic in detecting root causes and proposing sensible fixes to bugs in this situation (ie bugs encountered in production not compile time). It’s also fine at matching current styles and conventions (which is great if they are good styles and conventions). In terms of new code, rust would have been near impossible to write with such a high degree of non-local reasoning, so I’m assuming these bugs wouldn’t be present.
      - gbro3n 5mo ago
        The larger models really are more reliable at following instruction and reasoning their way to solutions. I haven't found that the harness makes that much difference. CoPilot, Claude, Pi, all see similar results for me. What really does make a difference is clean task separation and a clear plan / todo / implement workflow. I've consolidated a lot of the way I work with agents in https://www.agentkanban.io https://www.agentkanban.io - the task board keeps the tasks discrete and minimal. I built in plan todo implement into the agent instruction that binds the board task to the chat.
scared_together 5mo ago
I’m curious about the design space of languages & frameworks which are lower level than LLM prompts but higher level than Python, Ruby and Common Lisp. Do you have any recommendations for systems where reasoning about large systems is easier than in python?
- skydhash 5mo ago
  You have to go into live programming, code in a system, and saving images. Readability is no longer a factor, what you want is easy access to documentation, quick navigation, and a playground.
  - splitstud 5mo ago
    [dead]
- simonask 5mo ago
  Anything with a good, static type system will be an improvement, in my opinion. Types exist to encode invariants in an enforceable way, after all. Rust is the gold standard among imperative languages, but it’s standard fare among functional languages such as Haskell, OCaml, F#. You can also get really far in C++ if you have the stomach for it.
  - scared_together 5mo ago
    I have used Rust, Java and TypeScript before, so I understand that static typing is a major help. But I don’t think types are really sufficient to solve the problem you identified earlier of understanding how “many small individually readable things interact with each other”. Maybe you meant that phrase in a different sense than I read it, but it seems to me that there are still a lot of small individually readable things to keep track of in Rust.
bootsabota 5mo ago
This is why good design documents will always be necessary. When I work with AI I always have it keep an up-to-date architectural document committed to the repository. Also, we need to be able to understand what is happening under the hood somewhat, so I very much agree the readability is crucial. And frankly, rust is not up there in the readability realm. I think all the previous language designs still hold for their respective use case. AI written or otherwise. Why? Because performance acceptability is domain specific, and also the algorithms complexity generally determines overall performance. For example, move the performance critical stuff into a Python C extension like Torch etc…
neuronexmachina 5mo ago
Although it's not part of core Python, tach is pretty handy for specifying and enforcing those larger-scale interactions: https://github.com/tach-org/tach https://github.com/tach-org/tach
- nrub 5mo ago
  Yeah, that's cool, but it would be almost completely unnecessary if python just had actual private methods/classes/properties. It's a lot like pydantic, which is completely unnecessary if you had strong typing.
nostrademons 5mo ago
Locally readable is what I want for LLM-generated code, though. If I need to change the whole architecture, I re-prompt the LLM and have it rewrite the code for me. The changes that I'd need the code to be human-readable for are quick fixes where the LLM got something simple wrong and it'd take longer to explain to the LLM where it went off-track than to just fix it myself.

ant6n 5mo ago

That’s true. Once you have APIs and want to use classes to create larger structures, the language is full of warts.

cturner 5mo ago
I have built large systems on python that use classes, for more than ten years. I came to it from Java, ten years. As a rule, I avoid implementation inheritance. Occasionally I need to facade a library that assumes implementation inheritance to avoid it spreading into my codebase. When the codebase hits a certain size, I hand-roll some decorators to create functionality like java interfaces. With that done, and a suite of acceptance tests, I find it scales up well.

bryanrasmussen 5mo ago

hmm, yeah given LLM's ability to churn out lots of code quickly and be overly verbose in that code that is a potential downside. That it could in a quick one time edit create so much intellectual overhead that Python might be the wrong language to understand what is going on. What language do you feel is easier to reason about in the large?

hiAndrewQuinn 5mo ago
Haskell would be my vote, and Rust too, actually, both because of their very strong type systems. The type system lets you very quickly figure out what something is before you figure out what something does, and it turns out that separating those two concerns as hard as those two languages do often results in doing the whole one-two punch faster.
- lukan 5mo ago
  Haskell does not qualify for a large training set, though. (Nor for readability in my opinion) I think I have never seen haskell software made wih LLM's but well, aside from university, I have not seen Haskell code at all. (Also Haskell purists I would associate with people who avoid LLM's) I would rather go with Rust given these choices. But I have good results with typescript (or javascript for simpler things). Really large set of examples. Tools optimized for it, agents debugging in the browser works allmost out of the box. And well, a elaborate typesystem.
  - yakshaving_jgt 5mo ago
    [dead]
  - klodolph 5mo ago
    I used Claude to generate Haskell and it works really well. Claude struggles sometimes with respecting abstraction boundaries, but Haskell enforces parts of those boundaries in its type system better than a lot of other languages (if a module can’t do IO, for example). Works well, in my experience. Sometimes the agent does weird stuff that you have to rewrite, but I get the sense that this happens in any language. Maybe Haskell’s training set is not large enough, but it seems to work despite the smaller training set.
  - co_dh 5mo ago
    I used Claude that created a terminal based table viewer from rust first, to lean , and finally to Haskell. https://github.com/co-dh/tv-hask/tree/main https://github.com/co-dh/tv-hask/tree/main I give up rust because it’s not functional enough. There aren’t many things Claude can prove about a table viewer, and Haskell fits very well, and have enough libraries. Claude is pretty good at Haskell. I barely write Haskell before but I do know monad.
  - mightybyte 5mo ago
    How much code do you think is necessary for LLMs to be good enough?
- jorvi 5mo ago
harperlee 5mo ago
I'd say Java, because it has a massive footprint amenable for training, and a strong type system (does not have sum types though and those are trendy). You'd have to steer the LLM to use the style you want, and not massively overarchitect things though, but that's going to be an issue nonetheless.
- mands 5mo ago
  Java has sum types - they are fairly recent, called sealed records, and can be exhaustively pattern matched on. (I do agree however, Java is a great target for LLMs)
jimmaswell 5mo ago
C# is as close to an ideal language as you can get for most things IMO. I find AI does a great job with it.
- pjerem 5mo ago
  I do agree. C# is an hidden gem for IA. There are not that much different ways to get somewhere so the model have probably been trained on the framework and libraries everybody uses (the Microsoft ones). Compared to most languages, including Java, C# will have a hard time letting you compile incoherent code. You barely need any dependencies other than aspnetcore and efcore for most applications and your AI knows them well. It’s easy to do TDD with it so it’s easy to keep your IA from hallucinating.
  - kuboble 5mo ago
    I definitely agree with the sentiment. However this part. > There are not that much different ways to get somewhere This is far from true. C# is a language where you can operate on the raw pointers through unsafe keyword. On the other end of the spectrum, you can have duck-typing in dynamic blocks. For operating on collections you can use old style loops, or chain of lambdas or sql like syntax. I have been coding in C# old school way for most of my life at this point, and I feel like I'm in a foreign land reading code from some other C# projects.
- HumblyTossed 5mo ago
  I like C#, it's how I make a living, but it's way too large today. I can program in valid C# and it looks like C or I can program in C# and it looks like a functional language or I can program in C# and it's looks all angle-brakety like C++. The problem with that is everyone has an opinion on what good C# looks like. For personal projects, I'll take a much simpler language any day.
  - bonesss 5mo ago
    C# has recreated the C++ dialect conundrum. For some it’s effectively an idempotent functional language with unfortunate failings of exhaustiveness, for others it’s Java ca 2009, for others it’s C++ but not quite. Discipline, effort, linters, reviews, more discipline, more effort, retraining, discipline… and foot guns everywhere because so much of the adaptation has been a 95% solution. Personally I got everything C# promises even now when F# was dropped years ago and have found the interim pretty annoying.
barkingcat 5mo ago
get LLM to write ADA and have it use SPARK for verification.

bazoom42 5mo ago

For larger systems you create your own modules and abstractions, so comprehensibility at higher level does not depend so much on the language.

sundarurfriend 5mo ago
The tools the language gives you to create those abstractions make a lot of difference, however.
- mbreese 5mo ago
  But every abstraction that an LLM has to write is a choice. Your way of writing Python may not match that choice. The next run of the agent might not choose the same way. Because the language gives you many different tools, an LLM generated codebase can get inconsistent and overly complicated quickly. The flexibility of Python is a downside when you’re having an LLM generate the code. If you’re working in an existing codebase, it’s great - those choices were already made and it can match your style. When an LLM has to derive its own style is when things can devolve into a jumbled mess.
  - andyferris 5mo ago
    To me applying LLMs to a python (or similarly dynamic) code base where it’s currently spaghetti and monkey patched, it can miss things just like I can. But… I have to admit Opus 4.7 has been very pragmatic in detecting root causes and proposing sensible fixes to bugs in this situation (ie bugs encountered in production not compile time). It’s also fine at matching current styles and conventions (which is great if they are good styles and conventions). In terms of new code, rust would have been near impossible to write with such a high degree of non-local reasoning, so I’m assuming these bugs wouldn’t be present.
    - gbro3n 5mo ago
      The larger models really are more reliable at following instruction and reasoning their way to solutions. I haven't found that the harness makes that much difference. CoPilot, Claude, Pi, all see similar results for me. What really does make a difference is clean task separation and a clear plan / todo / implement workflow. I've consolidated a lot of the way I work with agents in https://www.agentkanban.io https://www.agentkanban.io - the task board keeps the tasks discrete and minimal. I built in plan todo implement into the agent instruction that binds the board task to the chat.

scared_together 5mo ago

I’m curious about the design space of languages & frameworks which are lower level than LLM prompts but higher level than Python, Ruby and Common Lisp. Do you have any recommendations for systems where reasoning about large systems is easier than in python?

skydhash 5mo ago
You have to go into live programming, code in a system, and saving images. Readability is no longer a factor, what you want is easy access to documentation, quick navigation, and a playground.
- splitstud 5mo ago
  [dead]
simonask 5mo ago
Anything with a good, static type system will be an improvement, in my opinion. Types exist to encode invariants in an enforceable way, after all. Rust is the gold standard among imperative languages, but it’s standard fare among functional languages such as Haskell, OCaml, F#. You can also get really far in C++ if you have the stomach for it.
- scared_together 5mo ago
  I have used Rust, Java and TypeScript before, so I understand that static typing is a major help. But I don’t think types are really sufficient to solve the problem you identified earlier of understanding how “many small individually readable things interact with each other”. Maybe you meant that phrase in a different sense than I read it, but it seems to me that there are still a lot of small individually readable things to keep track of in Rust.

bootsabota 5mo ago

This is why good design documents will always be necessary. When I work with AI I always have it keep an up-to-date architectural document committed to the repository. Also, we need to be able to understand what is happening under the hood somewhat, so I very much agree the readability is crucial. And frankly, rust is not up there in the readability realm. I think all the previous language designs still hold for their respective use case. AI written or otherwise. Why? Because performance acceptability is domain specific, and also the algorithms complexity generally determines overall performance. For example, move the performance critical stuff into a Python C extension like Torch etc…

neuronexmachina 5mo ago

Although it's not part of core Python, tach is pretty handy for specifying and enforcing those larger-scale interactions: https://github.com/tach-org/tach https://github.com/tach-org/tach

nrub 5mo ago
Yeah, that's cool, but it would be almost completely unnecessary if python just had actual private methods/classes/properties. It's a lot like pydantic, which is completely unnecessary if you had strong typing.

nostrademons 5mo ago

Locally readable is what I want for LLM-generated code, though. If I need to change the whole architecture, I re-prompt the LLM and have it rewrite the code for me. The changes that I'd need the code to be human-readable for are quick fixes where the LLM got something simple wrong and it'd take longer to explain to the LLM where it went off-track than to just fix it myself.