4 ms·

And (rigorously) reviewing code is easily 10x harder than writing it. But this won't stop it from happening.

tptacek 1y ago
This is just obviously not true. I had a full-time job of reviewing code for roughly 15 years and it was never true, but it's also just intuitively not true that engineers spend 10 hours reviewing their peers code to every 1 they spend writing it. What you mean to claim here is that verification is 10x harder than authorship. That's true, but unhelpful to skeptics, because LLMs are extremely useful for verification.
- JohnKemeny 1y ago
  I once graded over 100 exams in an introductory programming course (Python). The main exercise was to implement a simple game (without using a runtime). Some answers were trivial to grade—either obviously correct or clearly wrong. The rest were painful and exhausting to evaluate. Checking whether the code was correct and tracing it step by step in my head was so draining that I swore never to grade programming again.
  - tptacek 1y ago
    Right, sure. So: this doesn't generally happen with LLM outputs, but if it does, you simply kill the PR. A lot of people seem to be hung up on the idea that LLM agents don't have a 100% hit rate, let alone a 100% one-shot hit rate. A huge part of the idea is that it does not matter if an agents output is immediately workable. Just take the PRs where the code is straightforwardly reviewable.
    - JohnKemeny 1y ago
      But your reply was to "reviewing code is easily 10x harder than writing it". Of course that's not true if you just kill all PRs that are difficult to review. Sometimes, code is hard to review. It's not very helpful if the reviewer just kills it because it's hard.
      - tptacek 1y ago
        I understand everything about this comment except the words "but" and "it's not very helpful if".
      - girvo 1y ago
        > It's not very helpful if the reviewer just kills it because it's hard. I am absolutely still an AI skeptic, but like: we do this at work. If a dev has produced some absolutely nonsense overcomplicated impossible to understand PR, it gets rejected and sent back to the drawing board (and then I work with them to find out what happened, because thats a leadership failure more than a developer one IMO)
    - fzeroracer 1y ago
      You're massively burying the lede here with your statement of 'just take the PRs where the code is straightforwardly reviewable'. It's honestly such an odious statement that it makes me doubt your expertise in reviewing code and PRs. A lot of code can not and will not be straightforwardly reviewable because it all depends on context. Using an LLM adds an additional layer of abstraction between you and the context, because now you have to untangle whether or not it accomplished the context you gave it.
      - enraged_camel 1y ago
        I have no idea what you mean. Tptacek is correct. LLM does not add an additional layer because at the end of the day code is code. You read it and you can tell whether it does what you want because you were the person who gave the instructions. It is no different than reviewing the code written by a junior (who also does not add an additional layer of abstraction).
        fzeroracer 1y ago
        That's exactly what an additional layer is! If I outsource coding to someone else whether it's a junior engineer, an outside firm or an LLM that is an additional layer of context you need to understand. You need to figure out if the junior engineer grasped the problem set, if the firm understood the requirements or if the LLM actually generated decent code.
- girvo 1y ago
  While I'm still a skeptic (despite my Cursor usage at work), I still absolutely agree. Careful review has never been 10x more difficult than writing the code, I'm not sure where this comes from. And I've had about the same experience as you. Also, and know this doesn't matter, but it's so weird to see this downvoted. That's not an "I disagree" button...
  - tptacek 1y ago
    Thanks! On HN, it actually is a "disagree" button though; totally cromulent to downvote disagreement here. One of the site's big idiosyncrasies!
    - whyever 1y ago
      If it is a disagree button, then why is it disabled for new users?
      - tptacek 1y ago
        I think the karma thresholds here are kind of silly given how low they are, but Dan and Tom actually run this place and they know things I don't know about what chaos would be unleashed if we eliminated them.
    - girvo 1y ago
      Weird, I always read it as more than that due to the karma threshold! The more you know :)
meowface 1y ago
I really don't think that's true at all. Anyone here can read and sign off on 1000 lines of code faster than they can write the same 1000 lines of code, pretty much no matter what the code is.
- theshrike79 1y ago
  It also depends heavily on the language. I can review 100x more Go code in a set amount of time than I can, say React. With Go there are repetitive structures (if err == nil) and not that many keywords, it's easy to sniff out the suspicious bits and focus on them. With Javascript and all of the useWhatevers and cyclic dependencies and functions returning functions that call functions, it's a lot harder to figure out what the code does just by reading it.
  - meowface 1y ago
    I definitely get your point, but it's also pretty annoying to write weird indirect or meta-functional code, not just to read it. It's still almost always faster to read than to write. I can think of edge cases where a certain section of code is easier and faster to write than to read, but in general - in our practical day-to-day experiences - reading a lot of code is faster than writing a lot of code. Not just faster but less mentally and physically taxing. Still mentally taxing, yes, but less.
sothatsit 1y ago
You have misunderstood this "common wisdom". The idea is that people will spend 10x more time reading your code in all future time stacked together. Not that reading and understanding your code once takes 10x the effort of writing it, which is obviously untrue. Here is the quote from Clean Code, where this idea seems to originate from: > Indeed, the ratio of time spent reading versus writing code is well over 10 to 1.

This is just obviously not true. I had a full-time job of reviewing code for roughly 15 years and it was never true, but it's also just intuitively not true that engineers spend 10 hours reviewing their peers code to every 1 they spend writing it. What you mean to claim here is that verification is 10x harder than authorship. That's true, but unhelpful to skeptics, because LLMs are extremely useful for verification.

JohnKemeny 1y ago
I once graded over 100 exams in an introductory programming course (Python). The main exercise was to implement a simple game (without using a runtime). Some answers were trivial to grade—either obviously correct or clearly wrong. The rest were painful and exhausting to evaluate. Checking whether the code was correct and tracing it step by step in my head was so draining that I swore never to grade programming again.
- tptacek 1y ago
  Right, sure. So: this doesn't generally happen with LLM outputs, but if it does, you simply kill the PR. A lot of people seem to be hung up on the idea that LLM agents don't have a 100% hit rate, let alone a 100% one-shot hit rate. A huge part of the idea is that it does not matter if an agents output is immediately workable. Just take the PRs where the code is straightforwardly reviewable.
  - JohnKemeny 1y ago
    But your reply was to "reviewing code is easily 10x harder than writing it". Of course that's not true if you just kill all PRs that are difficult to review. Sometimes, code is hard to review. It's not very helpful if the reviewer just kills it because it's hard.
    - tptacek 1y ago
      I understand everything about this comment except the words "but" and "it's not very helpful if".
    - girvo 1y ago
      > It's not very helpful if the reviewer just kills it because it's hard. I am absolutely still an AI skeptic, but like: we do this at work. If a dev has produced some absolutely nonsense overcomplicated impossible to understand PR, it gets rejected and sent back to the drawing board (and then I work with them to find out what happened, because thats a leadership failure more than a developer one IMO)
  - fzeroracer 1y ago
    You're massively burying the lede here with your statement of 'just take the PRs where the code is straightforwardly reviewable'. It's honestly such an odious statement that it makes me doubt your expertise in reviewing code and PRs. A lot of code can not and will not be straightforwardly reviewable because it all depends on context. Using an LLM adds an additional layer of abstraction between you and the context, because now you have to untangle whether or not it accomplished the context you gave it.
    - enraged_camel 1y ago
      I have no idea what you mean. Tptacek is correct. LLM does not add an additional layer because at the end of the day code is code. You read it and you can tell whether it does what you want because you were the person who gave the instructions. It is no different than reviewing the code written by a junior (who also does not add an additional layer of abstraction).
      - fzeroracer 1y ago
        That's exactly what an additional layer is! If I outsource coding to someone else whether it's a junior engineer, an outside firm or an LLM that is an additional layer of context you need to understand. You need to figure out if the junior engineer grasped the problem set, if the firm understood the requirements or if the LLM actually generated decent code.
girvo 1y ago
While I'm still a skeptic (despite my Cursor usage at work), I still absolutely agree. Careful review has never been 10x more difficult than writing the code, I'm not sure where this comes from. And I've had about the same experience as you. Also, and know this doesn't matter, but it's so weird to see this downvoted. That's not an "I disagree" button...
- tptacek 1y ago
  Thanks! On HN, it actually is a "disagree" button though; totally cromulent to downvote disagreement here. One of the site's big idiosyncrasies!
  - whyever 1y ago
    If it is a disagree button, then why is it disabled for new users?
    - tptacek 1y ago
      I think the karma thresholds here are kind of silly given how low they are, but Dan and Tom actually run this place and they know things I don't know about what chaos would be unleashed if we eliminated them.
  - girvo 1y ago
    Weird, I always read it as more than that due to the karma threshold! The more you know :)

meowface 1y ago

I really don't think that's true at all. Anyone here can read and sign off on 1000 lines of code faster than they can write the same 1000 lines of code, pretty much no matter what the code is.

theshrike79 1y ago
It also depends heavily on the language. I can review 100x more Go code in a set amount of time than I can, say React. With Go there are repetitive structures (if err == nil) and not that many keywords, it's easy to sniff out the suspicious bits and focus on them. With Javascript and all of the useWhatevers and cyclic dependencies and functions returning functions that call functions, it's a lot harder to figure out what the code does just by reading it.
- meowface 1y ago
  I definitely get your point, but it's also pretty annoying to write weird indirect or meta-functional code, not just to read it. It's still almost always faster to read than to write. I can think of edge cases where a certain section of code is easier and faster to write than to read, but in general - in our practical day-to-day experiences - reading a lot of code is faster than writing a lot of code. Not just faster but less mentally and physically taxing. Still mentally taxing, yes, but less.

sothatsit 1y ago

You have misunderstood this "common wisdom". The idea is that people will spend 10x more time reading your code in all future time stacked together. Not that reading and understanding your code once takes 10x the effort of writing it, which is obviously untrue. Here is the quote from Clean Code, where this idea seems to originate from: > Indeed, the ratio of time spent reading versus writing code is well over 10 to 1.