4 ms·

Ok, if I notice other devs suddenly getting 5x as productive I'll give it a try, but so far no such effect has been demonstrated. It seems like a pretty straigh

Ok, if I notice other devs suddenly getting 5x as productive I'll give it a try, but so far no such effect has been demonstrated. It seems like a pretty straightforward research question, and you'd think if there was any demonstrable effect the companies selling these things would use such research to market their products. So where is it?

fragmede 2y ago
There's a dead body lying there but you won't believe they're actually dead until someone creates a video reenactment of the killer stabbing them and shoves that video in your face? Speaking of videos, https://youtu.be/opi1s_5Dm-c https://youtu.be/opi1s_5Dm-c is probably below your level and it's kinda long.
- jcgrillo 2y ago
  I don't get it. What is the dead body thing about?
  - fragmede 2y ago
    There are people building things and making money using these new techniques, but because they haven't stopped to produce research studies and white papers about their work, you want to presume it doesn't exist.
    - jcgrillo 2y ago
      Yes, without evidence there's no effect. It's telling that nowhere on cursor.ai's website do they appear to claim using their tool will actually make you, or crucially your team, more efficient.
      - fragmede 2y ago
        evidencthe absence of evidence doesn't mean there's no effect
        jcgrillo 2y ago
        There is no trustworthy evidence that I'm aware of to support the claim that "using an LLM IDE makes developers 5x more effective". Do you disagree? Testimonials don't cut it, because individuals are prone to hysteria, delusions, and just making stuff up. All I'm asking for is a few trustworthy case studies showing unambiguously that it's worth my time and money to pay attention to this genai stuff. That shouldn't be hard for a trillion dollar industry, right? Like show me this isn't just hysterical hype. Don't tell me, show me.
        fragmede 2y ago
        What does it even mean to be 5x more effective? Because that's the thing. Testimonials is where we're at because how do you measure "more effective"? I'm not making any claim on a number, 5x 10x? .2x because you have to go back and fix the shitty code it wrote? We don't have a good way to measure programmer productivity. We all know lines of code is ridiculous. And how do you account for how much brain space it takes up. If the LLM lets someone bang out useful code in between meetings all day, vs needing some truly quiet time for some deep deep work to properly get into the right zone... how do you metricize that and show, what, lines of code and interruptions per hour on a graph to have a line that goes up and to the right? It turns out that it's actually a really hard problem to measure, so, even for a trillion dollar industry, it is hard. What did you think of that video?

fragmede 2y ago

There's a dead body lying there but you won't believe they're actually dead until someone creates a video reenactment of the killer stabbing them and shoves that video in your face? Speaking of videos, https://youtu.be/opi1s_5Dm-c https://youtu.be/opi1s_5Dm-c is probably below your level and it's kinda long.

jcgrillo 2y ago
I don't get it. What is the dead body thing about?
- fragmede 2y ago
  There are people building things and making money using these new techniques, but because they haven't stopped to produce research studies and white papers about their work, you want to presume it doesn't exist.
  - jcgrillo 2y ago
    Yes, without evidence there's no effect. It's telling that nowhere on cursor.ai's website do they appear to claim using their tool will actually make you, or crucially your team, more efficient.
    - fragmede 2y ago
      evidencthe absence of evidence doesn't mean there's no effect
      - jcgrillo 2y ago
        There is no trustworthy evidence that I'm aware of to support the claim that "using an LLM IDE makes developers 5x more effective". Do you disagree? Testimonials don't cut it, because individuals are prone to hysteria, delusions, and just making stuff up. All I'm asking for is a few trustworthy case studies showing unambiguously that it's worth my time and money to pay attention to this genai stuff. That shouldn't be hard for a trillion dollar industry, right? Like show me this isn't just hysterical hype. Don't tell me, show me.
        fragmede 2y ago
        What does it even mean to be 5x more effective? Because that's the thing. Testimonials is where we're at because how do you measure "more effective"? I'm not making any claim on a number, 5x 10x? .2x because you have to go back and fix the shitty code it wrote? We don't have a good way to measure programmer productivity. We all know lines of code is ridiculous. And how do you account for how much brain space it takes up. If the LLM lets someone bang out useful code in between meetings all day, vs needing some truly quiet time for some deep deep work to properly get into the right zone... how do you metricize that and show, what, lines of code and interruptions per hour on a graph to have a line that goes up and to the right? It turns out that it's actually a really hard problem to measure, so, even for a trillion dollar industry, it is hard. What did you think of that video?