4 ms·

Any sufficiently complicated LLM generated program contains an ad hoc, informally-specified, bug-ridden, slow implementation of half of an open source project.

• kakacik 8mo ago
  We had an effort recently where one much more experienced dev from our company ran Claude on our oldish codebase for one system, with the goal of transforming it into newer structure, newer libraries etc. while preserving various built in functionalities. Not the first time this guy did such a thing and he is supposed to be an expert. I took a look at the result and its maybe half of stuff missing completely, rest is cryptic. I know that codebase by heart since I created it. From my 20+ years of experience correcting all this would take way more effort than manual rewrite from scratch by a senior. Suffice to say thats not what upper management wants to hear, llm adoption often became one of their yearly targets to be evaluated against. So we have a hammer and looking for nails to bend and crook. Suffice to say this effort led nowhere since we have other high priority goals, for now. Smaller things here & there, why not. Bigger efforts, so far sawed-off 2-barrel shotgun loaded with buckshot right into both feet.
  • kjksf 8mo ago
    Not to take away from your experience but to offer a counterpoint. I used claude code to port rust pdb parsing library to typescript. My SumatraPDF is a large C++ app and I wanted visibility into where does the size of functions / data go, layout of classes. So I wanted to build a tool to dump info out of a PDB. But I have been diagnosed with extreme case of Rustophobiatis so I just can't touch rust code. Hence, the port to typescript. With my assistance it did the work in an afternoon and did it well. The code worked. I ran it against large PDB from SumatraPDF and it matched the output of other tools. In a way porting from one language to another is extreme case of refactoring and Claude did it very well. I think that in general (your experience notwithstanding) Claude Caude is excellent at refactorings. Here are 3 refactorings from SumatraPDF where I asked claude code to simplify code written by a human: https://github.com/sumatrapdfreader/sumatrapdf/commit/a472d32048da750b45a9b39ba5da3e649d6165c5 https://github.com/sumatrapdfreader/sumatrapdf/commit/a472d3... https://github.com/sumatrapdfreader/sumatrapdf/commit/5624aa73729b0a5aaea8bc4c5d95328d5f7ec6ff https://github.com/sumatrapdfreader/sumatrapdf/commit/5624aa... https://github.com/sumatrapdfreader/sumatrapdf/commit/a40bc91e4da2338bf8cc8748277416db058ee58d https://github.com/sumatrapdfreader/sumatrapdf/commit/a40bc9... I hope you agree the code written by Claude is better than the code written by a human. Granted, those are small changes but I think it generalizes into bigger changes. I have few refactorings in mind I wanted to do for a long time and maybe with Claude they will finally be feasible (they were not feasible before only because I don't have infinite amount of time to do everything I want to do).
    • pja 8mo ago
      “I want this thing, but in a different language” seems to be something that the current generation of cutting edge LLMs are pretty good at. Translating a vibe is something the Ur-LLMS (GPT3 etc) were very good at so it’s not entirely surprising that the current state of the art is to be found in things of a “translate thing X that already exists into context Y” nature.
• cactusplant7374 8mo ago
  All software before LLMs had a copious number of bugs, many of which were never fixed.