6 ms·

The tools the language gives you to create those abstractions make a lot of difference, however.

mbreese 5mo ago
But every abstraction that an LLM has to write is a choice. Your way of writing Python may not match that choice. The next run of the agent might not choose the same way. Because the language gives you many different tools, an LLM generated codebase can get inconsistent and overly complicated quickly. The flexibility of Python is a downside when you’re having an LLM generate the code. If you’re working in an existing codebase, it’s great - those choices were already made and it can match your style. When an LLM has to derive its own style is when things can devolve into a jumbled mess.
- andyferris 5mo ago
  To me applying LLMs to a python (or similarly dynamic) code base where it’s currently spaghetti and monkey patched, it can miss things just like I can. But… I have to admit Opus 4.7 has been very pragmatic in detecting root causes and proposing sensible fixes to bugs in this situation (ie bugs encountered in production not compile time). It’s also fine at matching current styles and conventions (which is great if they are good styles and conventions). In terms of new code, rust would have been near impossible to write with such a high degree of non-local reasoning, so I’m assuming these bugs wouldn’t be present.
  - gbro3n 5mo ago
    The larger models really are more reliable at following instruction and reasoning their way to solutions. I haven't found that the harness makes that much difference. CoPilot, Claude, Pi, all see similar results for me. What really does make a difference is clean task separation and a clear plan / todo / implement workflow. I've consolidated a lot of the way I work with agents in https://www.agentkanban.io https://www.agentkanban.io - the task board keeps the tasks discrete and minimal. I built in plan todo implement into the agent instruction that binds the board task to the chat.
- brookst 5mo ago
  That’s why proper using LLMs on large python codebases establish coding standards docs and tests. Turning the LLM loose is chaos, but having clear arch and naming and other standards can get pretty consistent results.
rglullis 5mo ago
Name one of those abstractions that is missing in Python.
- nrub 5mo ago
  You joking? - strong typing - real concurrency (heaven forbid you want a background task without having to spool up an external message queue and worker) - immutability - limitations in error handling (sort of just typing really) - limitations in nullability (also typing) - memory layout is usually hidden or abstracted away - no actual private methods or classes That's far from a complete list, but maybe you're taking for granted the typical pythonic conventions that many practice. It requires a ton of work to design and architect python systems of any non-trivial size for maintainability and understanding. No language is perfect, but there are plenty of languages that make supporting complex systems easier than python.
  - rglullis 5mo ago
    So, the abstractions are there, you just happen to think that the implementations are flawed or limited. This is not the same as claiming they are lacking.
    - nrub 5mo ago
      No, they're not there. Did you read my comment? It's not just flawed or limited implementation, those are things that python just doesn't have.
      - rglullis 5mo ago
        It absolutely does. - Typing annotations + mypy can completely help you build and understand a complex system. WIth pyright you can even analyze code that is not annotated. The tooling that enables developers to design and conceptualize their application around the type abstraction is there. You make it sound like people can write`x = "2" + 20` in python like in Javascript or PHP4. - Concurrency: take your pick of multithreading, multiprocessing or asyncio. The abstraction of a thread model is there. The abstraction for an event loop is there. Would it be nice to have something like the Actor model as well? Sure, but to go from that to "python does not have it" is a completely wild take. - "No actual private methods or classes": I mean, really? Obviously classes are supported. You can create different classes by composition, you can create a hierachical structure. You can use Protocol to define types that must implement interfaces. You can define functions that are overloaded and you can have method dispatching. All of these ABSTRACTIONS are provided. It's not because they are not forced on you that they don't exist.

But every abstraction that an LLM has to write is a choice. Your way of writing Python may not match that choice. The next run of the agent might not choose the same way. Because the language gives you many different tools, an LLM generated codebase can get inconsistent and overly complicated quickly. The flexibility of Python is a downside when you’re having an LLM generate the code. If you’re working in an existing codebase, it’s great - those choices were already made and it can match your style. When an LLM has to derive its own style is when things can devolve into a jumbled mess.

andyferris 5mo ago
To me applying LLMs to a python (or similarly dynamic) code base where it’s currently spaghetti and monkey patched, it can miss things just like I can. But… I have to admit Opus 4.7 has been very pragmatic in detecting root causes and proposing sensible fixes to bugs in this situation (ie bugs encountered in production not compile time). It’s also fine at matching current styles and conventions (which is great if they are good styles and conventions). In terms of new code, rust would have been near impossible to write with such a high degree of non-local reasoning, so I’m assuming these bugs wouldn’t be present.
- gbro3n 5mo ago
  The larger models really are more reliable at following instruction and reasoning their way to solutions. I haven't found that the harness makes that much difference. CoPilot, Claude, Pi, all see similar results for me. What really does make a difference is clean task separation and a clear plan / todo / implement workflow. I've consolidated a lot of the way I work with agents in https://www.agentkanban.io https://www.agentkanban.io - the task board keeps the tasks discrete and minimal. I built in plan todo implement into the agent instruction that binds the board task to the chat.
brookst 5mo ago
That’s why proper using LLMs on large python codebases establish coding standards docs and tests. Turning the LLM loose is chaos, but having clear arch and naming and other standards can get pretty consistent results.

rglullis 5mo ago

Name one of those abstractions that is missing in Python.

nrub 5mo ago
You joking? - strong typing - real concurrency (heaven forbid you want a background task without having to spool up an external message queue and worker) - immutability - limitations in error handling (sort of just typing really) - limitations in nullability (also typing) - memory layout is usually hidden or abstracted away - no actual private methods or classes That's far from a complete list, but maybe you're taking for granted the typical pythonic conventions that many practice. It requires a ton of work to design and architect python systems of any non-trivial size for maintainability and understanding. No language is perfect, but there are plenty of languages that make supporting complex systems easier than python.
- rglullis 5mo ago
  So, the abstractions are there, you just happen to think that the implementations are flawed or limited. This is not the same as claiming they are lacking.
  - nrub 5mo ago
    No, they're not there. Did you read my comment? It's not just flawed or limited implementation, those are things that python just doesn't have.
    - rglullis 5mo ago
      It absolutely does. - Typing annotations + mypy can completely help you build and understand a complex system. WIth pyright you can even analyze code that is not annotated. The tooling that enables developers to design and conceptualize their application around the type abstraction is there. You make it sound like people can write`x = "2" + 20` in python like in Javascript or PHP4. - Concurrency: take your pick of multithreading, multiprocessing or asyncio. The abstraction of a thread model is there. The abstraction for an event loop is there. Would it be nice to have something like the Actor model as well? Sure, but to go from that to "python does not have it" is a completely wild take. - "No actual private methods or classes": I mean, really? Obviously classes are supported. You can create different classes by composition, you can create a hierachical structure. You can use Protocol to define types that must implement interfaces. You can define functions that are overloaded and you can have method dispatching. All of these ABSTRACTIONS are provided. It's not because they are not forced on you that they don't exist.