2 ms·

> He said it's "fantastic" that developers can now write software faster with these tools, sometimes while "barely looking at the code." It there _any_ empiric

> He said it's "fantastic" that developers can now write software faster with these tools, sometimes while "barely looking at the code." It there _any_ empirical evidence for this, at all? Like, beyond the _perception_ of the users.

shafyy 1y ago
No
spacecadet 1y ago
Yes none. However, one cannot discredit lived experience... and for many the lived experience of vibe coding is transformative and democratizing. Dare I say thats only fare given all the unwarranted "democratizing" the tech industry thought was helpful over the last 30 years...
logicchains 1y ago
>It there _any_ empirical evidence for this, at all? Like, beyond the _perception_ of the users. Someone managed to write a standards-compliant HTTP 2.0 server in just two weeks, 40k+ lines of code (including tests) with almost all of the coding done by Gemini Pro: https://outervationai.substack.com/p/building-a-100-llm-written-standards https://outervationai.substack.com/p/building-a-100-llm-writ... . No unassisted programmer could do that so fast (except maybe Fabrice Bellard).
Energiekomin 1y ago
[dead]
fragmede 1y ago
How would that evidence exist? Aider writes how much code was generated in each release, and it's in the 80%s, but what you're asking for is for someone to do a timed trial on writing a bit of code unassisted, then for that same person to write the same bit of code with LLM assistance and also with forgetting the first version they wrote. Are we so caught up in proving they're not idk that we're stuck with the parachute engineering/science problem? Because, scientifically, the only way we can know parachutes are any good is to test the hypothesis by throwing the control group out of the plane without parachutes and comparing them to a group of people who jumped out of the plane with parachutes.

2 ms·

> He said it's "fantastic" that developers can now write software faster with these tools, sometimes while "barely looking at the code." It there _any_ empiric

by rsynnott 1y ago

shafyy 1y ago
No
spacecadet 1y ago
Yes none. However, one cannot discredit lived experience... and for many the lived experience of vibe coding is transformative and democratizing. Dare I say thats only fare given all the unwarranted "democratizing" the tech industry thought was helpful over the last 30 years...
logicchains 1y ago
>It there _any_ empirical evidence for this, at all? Like, beyond the _perception_ of the users. Someone managed to write a standards-compliant HTTP 2.0 server in just two weeks, 40k+ lines of code (including tests) with almost all of the coding done by Gemini Pro: https://outervationai.substack.com/p/building-a-100-llm-written-standards https://outervationai.substack.com/p/building-a-100-llm-writ... . No unassisted programmer could do that so fast (except maybe Fabrice Bellard).
Energiekomin 1y ago
[dead]
fragmede 1y ago
How would that evidence exist? Aider writes how much code was generated in each release, and it's in the 80%s, but what you're asking for is for someone to do a timed trial on writing a bit of code unassisted, then for that same person to write the same bit of code with LLM assistance and also with forgetting the first version they wrote. Are we so caught up in proving they're not idk that we're stuck with the parachute engineering/science problem? Because, scientifically, the only way we can know parachutes are any good is to test the hypothesis by throwing the control group out of the plane without parachutes and comparing them to a group of people who jumped out of the plane with parachutes.