4 ms·

I'm only three weeks into using Claude Code but I'm now seeing impressive results using a structured, "role" or "persona" based approach in a large (500K+ SLOC)

I'm only three weeks into using Claude Code but I'm now seeing impressive results using a structured, "role" or "persona" based approach in a large (500K+ SLOC) Elixir / Phoenix codebase. I'm using the $200 Max plan - so my inference costs are fixed. For certain, the results are better when I use it to build new features into our platform - as opposed to making complicated refactors or other deep changes to existing parts of the system. But even in the latter case, if we have good technical documentation capturing the design and how parts of the system work (which we don't in many places), Claude Code can make good progress. At first I was seeing a fair amount of what I would consider "bad code" - implementation and code that either didn't follow accepted coding style and patterns or that simply wasn't structured for reusability, maintainability. But after strengthening the CLAUDE.md file and adding an "elixir-code-reviewer" subagent which the "developer" persona had to use - the quality of code improved significantly. Our platform is open source, you can see our current Claude commands and subagents here: https://github.com/Simon-Initiative/oli-torus/tree/master/.claude https://github.com/Simon-Initiative/oli-torus/tree/master/.c...

iandanforth 1y ago
Thanks for sharing that extensive documentation!
fny 1y ago
I didn't realize just how bad LLMs are with unpopular languages until now: "Elixir lists do not support index based access via the access syntax" "Never use else if or elseif in Elixir, always use cond or case for multiple conditionals."
- LiamPowell 1y ago
  Interestingly I've found that Claude is very good at writing valid Ada, it just writes complete garbage that doesn't follow the specification at all. As an example I asked it to change the handling of command line arguments in a program so that duplicates would cause an error rather than being ignored. If I recall correctly it took 6 course corrections to get to what I asked for, but each time it wrote valid code that just didn't complete the task. One I remember was arbitrarily limiting the length of an argument to 4096 characters and then only accepting arguments that were exactly 4096 characters. Here is the relevant change, it didn't have any sort of hidden complexity: https://github.com/Prunt3D/prunt/commit/b4d7f5e35be6017846b889a692a36c5a3e8a0bfc https://github.com/Prunt3D/prunt/commit/b4d7f5e35be6017846b8...
  - fny 1y ago
    I'm pretty convinced the only developers who think we're on the cusp of AGI code exclusively in Python or JavaScript.
    - Leynos 1y ago
      GPT-5 is a dab hand at Rust
      - evolve2k 1y ago
        I was hearing that it was good with writing Ruby on Rails as the rails community is so structured already in where things go and sorta the Rails way of writing the code. Anyone have experience with this?
    - 3uler 1y ago
      The problem is that there is a lot of bad python and typescript/javascript out there, and I similarly find my self having to define my coding style in context files in order to get decent results in newer code bases without a lot of examples to follow. And even then you need to say do it like @example.py all the time. Maybe the future is fine-tuned models on specific coding styles?
    - touristtam 1y ago
      I think those are cult followers which _leaders_ have no understanding of programming language in the first place. The Python/JS bias might be because of the training dataset ingested.

iandanforth 1y ago

Thanks for sharing that extensive documentation!

fny 1y ago

I didn't realize just how bad LLMs are with unpopular languages until now: "Elixir lists do not support index based access via the access syntax" "Never use else if or elseif in Elixir, always use cond or case for multiple conditionals."

LiamPowell 1y ago
Interestingly I've found that Claude is very good at writing valid Ada, it just writes complete garbage that doesn't follow the specification at all. As an example I asked it to change the handling of command line arguments in a program so that duplicates would cause an error rather than being ignored. If I recall correctly it took 6 course corrections to get to what I asked for, but each time it wrote valid code that just didn't complete the task. One I remember was arbitrarily limiting the length of an argument to 4096 characters and then only accepting arguments that were exactly 4096 characters. Here is the relevant change, it didn't have any sort of hidden complexity: https://github.com/Prunt3D/prunt/commit/b4d7f5e35be6017846b889a692a36c5a3e8a0bfc https://github.com/Prunt3D/prunt/commit/b4d7f5e35be6017846b8...
- fny 1y ago
  I'm pretty convinced the only developers who think we're on the cusp of AGI code exclusively in Python or JavaScript.
  - Leynos 1y ago
    GPT-5 is a dab hand at Rust
    - evolve2k 1y ago
      I was hearing that it was good with writing Ruby on Rails as the rails community is so structured already in where things go and sorta the Rails way of writing the code. Anyone have experience with this?
  - 3uler 1y ago
    The problem is that there is a lot of bad python and typescript/javascript out there, and I similarly find my self having to define my coding style in context files in order to get decent results in newer code bases without a lot of examples to follow. And even then you need to say do it like @example.py all the time. Maybe the future is fine-tuned models on specific coding styles?
  - touristtam 1y ago
    I think those are cult followers which _leaders_ have no understanding of programming language in the first place. The Python/JS bias might be because of the training dataset ingested.