5 ms·

It's actually 1.2x productivity. Writing code does not take most of the day. If GPT can be just as good at debugging as it is at writing code, maybe the speedup

It's actually 1.2x productivity. Writing code does not take most of the day. If GPT can be just as good at debugging as it is at writing code, maybe the speedup would increase a bit. The ultimate AI speed == human reading + thinking speed, not AI generation speed.

blowski 3y ago
I’ve found it’s very useful for understanding snippets of code, like a 200-line function. But systems are more than “lots of 200-line functions” - there’s a lot of context hidden in flags, conditional blocks, data, Git histories. Maybe one day, we’ll be able to run it over all the Git histories, Jira tickets, Confluence documentation, Slack conversations, emails, meeting transcripts, presentations, etc. Until then, the humans will need to stitch it all together as best they can.
- darkerside 3y ago
  I can't think of a reason you couldn't specifically train an AI on your own large code base. After all, current LLMs are trained on effectively the entire internet.
  - zappchance 3y ago
    Unless your documentation is 20~100x the size of your codebase and written in a conversational tone, the LLM won't be able to be asked any questions about it using English. If your only aim is to use it like Copilot, sure, it's useful.
    - sandos 3y ago
      So the first step is to let the LLM write the documentation.... :)
      - still_grokking 3y ago
        Sure. Because it understands the code so well.
    - oceanplexian 3y ago
      You might be able fine tune a model on pull requests if they have really high quality descriptions, high quality commit messages, and the code is well documented and organized.
      - still_grokking 3y ago
        I'm really not sure this is irony or a serious comment.
  - blowski 3y ago
    I haven't yet seen anything that can scan an entire codebase and build, say, a data lineage to understand how a value in the UI was calculated. I'm sure it's coming, though.
    - still_grokking 3y ago
      It's coming, I'm sure. Just right after we have invented AGI.
walthamstow 3y ago
1.2x is a good rough number. Some things it saves me hours (regex, writing unfamiliar languages), some thing I never even bother asking it about
fragmede 3y ago
but not all human thinking is worthwhile. I had it do a simple chrome extension to let me reply inline on HN, and it coughed out a manifest.json that worked first try. I didn't have to poke around the Internet to find a reference and then debug that via stack overflows. Easily saved me half an hour and gave me more mental bandwidth for the futzing with the DOM that I did need to do. (to your point tho, I didn't try feeding the html to it to see if it could do that part for me.) so it's somewhere between 1.2x and 10x for me, depending on what I'm doing that day. Maybe 3x on a good day?
- jjnoakes 3y ago
  I don't mean to pick on you specifically, but this kind of approach doesn't fit the way I like to work. For example, just because the manifest.json worked doesn't mean it is correct - is it free of issues (security or otherwise)? I would argue that every system in production today seemed to "just work" when it was built and initially tested, and yet how many serious issues are in the wild today (security or otherwise)? I prefer to take a little more time solving problems, gaining an understanding of WHY things are done certain ways, and some insight into potential problems that may arise in the future even if something seems to work at first glance. Now I get that you are just talking about a small chrome extension that maybe you are only using for yourself... but scaling that up to anything beyond that seems like a ticking time bomb to me.
  - 0xedd 3y ago
    Most managers don't care for people like you. Companies sell their product. Another successful fiscal year. Irregardless of the absolute shit code base, wasteful architecture and gaping security vulnerabilities.
    - jjnoakes 3y ago
      Maybe, but I've always had lucrative jobs and my work has always been appreciated. Maybe you just have to find the right employer. I think longer-term, employers that value high quality work will have the upper-hand. And to be honest, I don't care for managers like that, so the feeling is mutual.
pfsalter 3y ago
> Writing code does not take most of the day. Definitely this. I spend about 10-15% of my time writing code so a 20% increase really doesn't save me a lot of time. Also AI generated code requires more reading of code, which is harder and more cognitively expensive than writing code.
- darkerside 3y ago
  Depends on the code. If AI can quickly write even mediocre quality automated tests, that's a tremendous speedup, and, if I'm being totally honest, morale boost.
  - anotherpaulg 3y ago
    It can. I use aider/GPT-4 for this all the time. It’s super valuable and very low effort.
  - still_grokking 3y ago
    If AI can write your tests, you're testing likely implementation details. This has "negative value"! https://www.youtube.com/watch?v=EZ05e7EMOLM https://www.youtube.com/watch?v=EZ05e7EMOLM But an AI can't write higher level tests as it would need to understand large chunks of code (sometimes whole systems), which it can't.
    - darkerside 3y ago
      I disagree about negative value. When it's this easy to throw away and rewrite tests, it's helpful to test the implementation details of your code.
      - still_grokking 3y ago
        Throwing away and rewriting tests is work (== negative value). Testing implementation details is always contra productive. Have you watched the video? (I'm not recommending videos often, as I think writings have more value per time-unit, but this talk is a kind of classic on that topic.)

blowski 3y ago

I’ve found it’s very useful for understanding snippets of code, like a 200-line function. But systems are more than “lots of 200-line functions” - there’s a lot of context hidden in flags, conditional blocks, data, Git histories. Maybe one day, we’ll be able to run it over all the Git histories, Jira tickets, Confluence documentation, Slack conversations, emails, meeting transcripts, presentations, etc. Until then, the humans will need to stitch it all together as best they can.

darkerside 3y ago
I can't think of a reason you couldn't specifically train an AI on your own large code base. After all, current LLMs are trained on effectively the entire internet.
- zappchance 3y ago
  Unless your documentation is 20~100x the size of your codebase and written in a conversational tone, the LLM won't be able to be asked any questions about it using English. If your only aim is to use it like Copilot, sure, it's useful.
  - sandos 3y ago
    So the first step is to let the LLM write the documentation.... :)
    - still_grokking 3y ago
      Sure. Because it understands the code so well.
  - oceanplexian 3y ago
    You might be able fine tune a model on pull requests if they have really high quality descriptions, high quality commit messages, and the code is well documented and organized.
    - still_grokking 3y ago
      I'm really not sure this is irony or a serious comment.
- blowski 3y ago
  I haven't yet seen anything that can scan an entire codebase and build, say, a data lineage to understand how a value in the UI was calculated. I'm sure it's coming, though.
  - still_grokking 3y ago
    It's coming, I'm sure. Just right after we have invented AGI.

walthamstow 3y ago

1.2x is a good rough number. Some things it saves me hours (regex, writing unfamiliar languages), some thing I never even bother asking it about

fragmede 3y ago

but not all human thinking is worthwhile. I had it do a simple chrome extension to let me reply inline on HN, and it coughed out a manifest.json that worked first try. I didn't have to poke around the Internet to find a reference and then debug that via stack overflows. Easily saved me half an hour and gave me more mental bandwidth for the futzing with the DOM that I did need to do. (to your point tho, I didn't try feeding the html to it to see if it could do that part for me.) so it's somewhere between 1.2x and 10x for me, depending on what I'm doing that day. Maybe 3x on a good day?

jjnoakes 3y ago
I don't mean to pick on you specifically, but this kind of approach doesn't fit the way I like to work. For example, just because the manifest.json worked doesn't mean it is correct - is it free of issues (security or otherwise)? I would argue that every system in production today seemed to "just work" when it was built and initially tested, and yet how many serious issues are in the wild today (security or otherwise)? I prefer to take a little more time solving problems, gaining an understanding of WHY things are done certain ways, and some insight into potential problems that may arise in the future even if something seems to work at first glance. Now I get that you are just talking about a small chrome extension that maybe you are only using for yourself... but scaling that up to anything beyond that seems like a ticking time bomb to me.
- 0xedd 3y ago
  Most managers don't care for people like you. Companies sell their product. Another successful fiscal year. Irregardless of the absolute shit code base, wasteful architecture and gaping security vulnerabilities.
  - jjnoakes 3y ago
    Maybe, but I've always had lucrative jobs and my work has always been appreciated. Maybe you just have to find the right employer. I think longer-term, employers that value high quality work will have the upper-hand. And to be honest, I don't care for managers like that, so the feeling is mutual.

pfsalter 3y ago

> Writing code does not take most of the day. Definitely this. I spend about 10-15% of my time writing code so a 20% increase really doesn't save me a lot of time. Also AI generated code requires more reading of code, which is harder and more cognitively expensive than writing code.

darkerside 3y ago
Depends on the code. If AI can quickly write even mediocre quality automated tests, that's a tremendous speedup, and, if I'm being totally honest, morale boost.
- anotherpaulg 3y ago
  It can. I use aider/GPT-4 for this all the time. It’s super valuable and very low effort.
- still_grokking 3y ago
  If AI can write your tests, you're testing likely implementation details. This has "negative value"! https://www.youtube.com/watch?v=EZ05e7EMOLM https://www.youtube.com/watch?v=EZ05e7EMOLM But an AI can't write higher level tests as it would need to understand large chunks of code (sometimes whole systems), which it can't.
  - darkerside 3y ago
    I disagree about negative value. When it's this easy to throw away and rewrite tests, it's helpful to test the implementation details of your code.
    - still_grokking 3y ago
      Throwing away and rewriting tests is work (== negative value). Testing implementation details is always contra productive. Have you watched the video? (I'm not recommending videos often, as I think writings have more value per time-unit, but this talk is a kind of classic on that topic.)