3 ms·

Great essay by Yegge as always. I have worked in the search space for a long time. And cherry picked examples of obvious wins, some NLP thing being smart, are

Great essay by Yegge as always. I have worked in the search space for a long time. And cherry picked examples of obvious wins, some NLP thing being smart, are _rife_. You get worked up about that one time something spooky/incredible happened and think the world is changing/ending. What I would like to see is an actual, independent, reproducible study of productivity gains on specific tasks. I've not seen this yet. I'm curious if anything is out there?

solsane 4y ago
Have you played with ChatGPT or GitHub copilot? 5 years ago, I'd agree with you, but nowadays, 'cherry picked wins' happen far more often than not in my experience.
- softwaredoug 4y ago
  Yes, quite extensively. It was quite good. I'm very enthusiastic about it. I just want some data other than my n=1 experience.
- ModernMech 4y ago
  The thing about it is the distribution of wins is going to be different depending on the prompter, and that will make all the difference. As experienced devs, we can prompt the AI, and take the wins when they come, but deftly avert the nonsense when we see it, because we know what to look for. For experienced devs like you and I, this is a huge win and will result in productivity gains. I worry for the junior dev. They can take the wins and feel good, but they’re going to fall for the nonsense every time, because they don’t k ow how to spot it. Junior devs are fed lies and they take them to be truths. Hopefully they are dispelled immediately when they are tested in code, but other lies will persist and be sold as truths. I am worried this is a bigger problem than people might think, cause it’s a feedback loop that can lead to a vicious cycle. We see what vicious cycles have done to social media, and social spheres, imagine what one will do to software systems. Next year we’ll be graduating students who started college during the pandemic. I would say their programming skills are quite atrophied compared to previous generations. Now I’m worried in four years we’ll be graduating students who started high school in covid times and started colleges using AI chat bots. Who knows if these graduates will even be able to do anything close to what students even 2 years ago could do. They’ll all just blankly stare and reach for their iPhones like they do today, but exponentially worse because they won’t even be able to formulate the search prompt. They’ll need AI for that too.
callahad 4y ago
GitHub did some research around productivity with Copilot. They discuss the results and link out to other research at https://github.blog/2022-09-07-research-quantifying-github-copilots-impact-on-developer-productivity-and-happiness/ https://github.blog/2022-09-07-research-quantifying-github-c...

solsane 4y ago

Have you played with ChatGPT or GitHub copilot? 5 years ago, I'd agree with you, but nowadays, 'cherry picked wins' happen far more often than not in my experience.

softwaredoug 4y ago
Yes, quite extensively. It was quite good. I'm very enthusiastic about it. I just want some data other than my n=1 experience.
ModernMech 4y ago
The thing about it is the distribution of wins is going to be different depending on the prompter, and that will make all the difference. As experienced devs, we can prompt the AI, and take the wins when they come, but deftly avert the nonsense when we see it, because we know what to look for. For experienced devs like you and I, this is a huge win and will result in productivity gains. I worry for the junior dev. They can take the wins and feel good, but they’re going to fall for the nonsense every time, because they don’t k ow how to spot it. Junior devs are fed lies and they take them to be truths. Hopefully they are dispelled immediately when they are tested in code, but other lies will persist and be sold as truths. I am worried this is a bigger problem than people might think, cause it’s a feedback loop that can lead to a vicious cycle. We see what vicious cycles have done to social media, and social spheres, imagine what one will do to software systems. Next year we’ll be graduating students who started college during the pandemic. I would say their programming skills are quite atrophied compared to previous generations. Now I’m worried in four years we’ll be graduating students who started high school in covid times and started colleges using AI chat bots. Who knows if these graduates will even be able to do anything close to what students even 2 years ago could do. They’ll all just blankly stare and reach for their iPhones like they do today, but exponentially worse because they won’t even be able to formulate the search prompt. They’ll need AI for that too.

callahad 4y ago

GitHub did some research around productivity with Copilot. They discuss the results and link out to other research at https://github.blog/2022-09-07-research-quantifying-github-copilots-impact-on-developer-productivity-and-happiness/ https://github.blog/2022-09-07-research-quantifying-github-c...