5 ms·

It is 80/20 again - it gets you 80% of the way in 20% of the time and then you spend 80% of the time to get the rest of the 20% done. And since it always feels

It is 80/20 again - it gets you 80% of the way in 20% of the time and then you spend 80% of the time to get the rest of the 20% done. And since it always feels like it is almost there, sunk-cost fallacy comes into play as well and you just don't want to give up. I think an approach that I tried recently is to use it as a friction remover instead of a solution provider. I do the programming but use it to remove pebbles such as that small bit of syntax I forgot, basically to keep up the velocity. However, I don't look at the wholesale code it offers. I think keeping the active thinking cap on results in code I actually understand while avoiding skill atrophy.

wmeredith 1y ago
> and then you spend 80% of the time to get the rest of the 20% done This was my pr-AI experience anyway, so getting that first chunk of time back is helpful. Related: One of the better takes I've seen on AI from an experienced developer was, "90% of my skills just became worthless, and the other 10% just became 1,000 times more valuable." There's some hyperbole there, I but I like the gist.
- skydhash 1y ago
  It’s not funny when you find yourself redoing the first 80%, as the only way to complete the second 80%.
- bluefirebrand 1y ago
  Let us know if that dev you're talking about winds up working 90% less for the same amount, or earning 1000x more Otherwise he can shut the fuck up about being 1000x more valuable imo
emodendroket 1y ago
I think it’s most useful when you basically need Stack Overflow on steroids: I basically know what I want to do but I’m not sure how to achieve it using this environment. It can also be helpful for debugging and rubber ducking generally.
- threetonesun 1y ago
  Absolutely this. For a while I was working with a language I was only partially familiar with, and I'd say "here's how I would do this in [primary language], rewrite it in [new language]" and I'd get a decent piece of code back. A little searching in the project to make sure it was stylistically correct and then done.
  - emodendroket 1y ago
    Those kind of tasks are good for it, yeah. “Here’s some JSON. Please generate a Java class I can deserialize it into” is similar.
- some-guy 1y ago
  All those things are true, but it's such a small part of my workflow at this point that the savings, while nice, aren't nearly as life-changing to my job as my CEO is forcing us to think it is. Once AI can actually untangle our 14 year old codebase full of hosh-posh code, read every commit message, JIRA ticket, and Slack conversation related to the changes in full context, it's not going to solve a lot of the hard problems at my job.
  - emodendroket 1y ago
    Some of the “explain what it does” functionality is better than you might think but to be honest I find myself called on to work with unfamiliar tools all the time so I find plenty of value.
- skydhash 1y ago
  The issue is that it is slow and verbose, at least in its default configuration. The amount of reading is non trivial. There’s a reason most references are dense.
  - lukan 1y ago
    Those issues you can partly solve by changing the prompt to tell it to be concise and don't explain its code. But nothing will make them stick to the one API version I use.
eknkc 1y ago
It works great on adding stuff to an already established codebase. Things like “we have these search parameters, also add foo”. Remove anything related to x…
- antonvs 1y ago
  Exactly. If you can give it a contract and a context, essentially, and it doesn't need to write a large amount of code to fulfill it, it can be great. I just used it to write about 80 lines of new code like that, and there's no question it saves time.
reverendsteveii 1y ago
well we used to have a sort of inverse pareto where 80% of the work took 80% of the effort and the remaining 20% of the work also took 80% of the effort. I do think you're onto something with getting pebbles out of the road inasmuch as once I know what I need to do AI coding makes the doing much faster. Just yesterday I was playing around with removing things from a List object using the Java streams API and I kept running into ConcurrentOperationsExceptions, which happen when multiple threads are mutating the list object at the same time because no thread can guarantee it has the latest copy of the list unaltered by other threads. I spent about an hour trying to write a method that deep copies the list, makes the change and then returns the copy and running into all sorts of problems til I asked AI to build me a thread-safe list mutation method and it was like "Sure, this is how I'd do it but also the API you're working with already has a method that just....does this." Cases like this are where AI is supremely useful - intricate but well-defined problems.
- cwmoore 1y ago
  Code reuse at scale: 80 + 80 = 160% ~ phi...coincidence? I think this may become a long horizon harvest for the rigorous OOP strategy, may Bill Joy be disproved. Gray goo may not [taste] like steel-cut oatmeal.
  - Sharlin 1y ago
    It's often said that π is the factor by which one should multiply all estimates – reducing it to ɸ would be a significant improvement in estimation accuracy!
  - visarga 1y ago
    1.6x multiplier is low, we usually need to apply 5x
    - deleted 1y ago
      [deleted]
- jahnu 1y ago
  > once I know what I need to do AI coding makes the doing much faster Most commenters on this paper seem to not respond to the strongest result from it. That is, the developers wrongly thought and felt that using AI had sped up their work. So we need to be super cautious about what we think we know.
01100011 1y ago
As an old dev this is really all I want: a sort of autocorrect for my syntactical errors to save me a couple compile-edit cycles.
- pferde 1y ago
  What I want is not autocorrect, because that won't teach me anything. I want it to yell at me loudly and point to the syntactical error. Autocorrect is a scourge of humanity.
causal 1y ago
Agreed and +1 on "always feels like it is almost there" leading to time sink. AI is especially good at making you feel like it's doing something useful; it takes a lot of skill to discern the truth.
i_love_retros 1y ago
The problem is I then have to also figure out the code it wrote to be able to complete the final 20%. I have no momentum and am starting from almost scratch mentally.
qingcharles 1y ago
This is just not true in my experience. Not with the latest models. I routinely manage to 1-shot a whole "thing." e.g. yesterday I needed a Wordpress plugin for a single-time use to clean up a friend's site. I described exactly what I needed, it produced the code, it ran perfect first time and the UI looked like a million dollars. It got me 100% of the way in 0% of the time. I'm the biggest skeptic, but more and more I'm seeing it get me the bulk of the way with very little back-and-forth. If it was even more heavily integrated in my dev environment, it would save me even more time.
curious_cat_163 1y ago
100% agreed. It is all about removing friction for me. Case in point: I would not have touched React in my previous career without the assist that LLMs now provide. The barrier to entry just _felt_ to be too large and one always has the instinct to stick with what one knows. However, it is _fun_ to go over the barrier if it is chatting with a model to get a quick tutorial and produce working code for a prototype (for your specific needs) where the understanding that you just developed is applied. The alternative (without LLMs) is to first do the ground work of learning via tutorials in text/video form and then do the cognitive mapping of applying the learning to one's prototype. I would make a lot of mistakes that expert/intermediate React developers don't make on this path. One could argue that it shortcuts some learning and perhaps the old way results in better retention. But, our field changes so fast... and when it remains static for too long, projects die. I think of all this as accelerant for progress in adoption of new ways of thinking about software and diffusing that more quickly across the developer population globally. Code is always fungible, anyway. The job is about all the other things that one needs to do besides coding.
RainyDayTmrw 1y ago
More sinister is it then takes 80% of the credit.

wmeredith 1y ago

> and then you spend 80% of the time to get the rest of the 20% done This was my pr-AI experience anyway, so getting that first chunk of time back is helpful. Related: One of the better takes I've seen on AI from an experienced developer was, "90% of my skills just became worthless, and the other 10% just became 1,000 times more valuable." There's some hyperbole there, I but I like the gist.

skydhash 1y ago
It’s not funny when you find yourself redoing the first 80%, as the only way to complete the second 80%.
bluefirebrand 1y ago
Let us know if that dev you're talking about winds up working 90% less for the same amount, or earning 1000x more Otherwise he can shut the fuck up about being 1000x more valuable imo

emodendroket 1y ago

I think it’s most useful when you basically need Stack Overflow on steroids: I basically know what I want to do but I’m not sure how to achieve it using this environment. It can also be helpful for debugging and rubber ducking generally.

threetonesun 1y ago
Absolutely this. For a while I was working with a language I was only partially familiar with, and I'd say "here's how I would do this in [primary language], rewrite it in [new language]" and I'd get a decent piece of code back. A little searching in the project to make sure it was stylistically correct and then done.
- emodendroket 1y ago
  Those kind of tasks are good for it, yeah. “Here’s some JSON. Please generate a Java class I can deserialize it into” is similar.
some-guy 1y ago
All those things are true, but it's such a small part of my workflow at this point that the savings, while nice, aren't nearly as life-changing to my job as my CEO is forcing us to think it is. Once AI can actually untangle our 14 year old codebase full of hosh-posh code, read every commit message, JIRA ticket, and Slack conversation related to the changes in full context, it's not going to solve a lot of the hard problems at my job.
- emodendroket 1y ago
  Some of the “explain what it does” functionality is better than you might think but to be honest I find myself called on to work with unfamiliar tools all the time so I find plenty of value.
skydhash 1y ago
The issue is that it is slow and verbose, at least in its default configuration. The amount of reading is non trivial. There’s a reason most references are dense.
- lukan 1y ago
  Those issues you can partly solve by changing the prompt to tell it to be concise and don't explain its code. But nothing will make them stick to the one API version I use.

eknkc 1y ago

It works great on adding stuff to an already established codebase. Things like “we have these search parameters, also add foo”. Remove anything related to x…

antonvs 1y ago
Exactly. If you can give it a contract and a context, essentially, and it doesn't need to write a large amount of code to fulfill it, it can be great. I just used it to write about 80 lines of new code like that, and there's no question it saves time.

reverendsteveii 1y ago

well we used to have a sort of inverse pareto where 80% of the work took 80% of the effort and the remaining 20% of the work also took 80% of the effort. I do think you're onto something with getting pebbles out of the road inasmuch as once I know what I need to do AI coding makes the doing much faster. Just yesterday I was playing around with removing things from a List object using the Java streams API and I kept running into ConcurrentOperationsExceptions, which happen when multiple threads are mutating the list object at the same time because no thread can guarantee it has the latest copy of the list unaltered by other threads. I spent about an hour trying to write a method that deep copies the list, makes the change and then returns the copy and running into all sorts of problems til I asked AI to build me a thread-safe list mutation method and it was like "Sure, this is how I'd do it but also the API you're working with already has a method that just....does this." Cases like this are where AI is supremely useful - intricate but well-defined problems.

cwmoore 1y ago
Code reuse at scale: 80 + 80 = 160% ~ phi...coincidence? I think this may become a long horizon harvest for the rigorous OOP strategy, may Bill Joy be disproved. Gray goo may not [taste] like steel-cut oatmeal.
- Sharlin 1y ago
  It's often said that π is the factor by which one should multiply all estimates – reducing it to ɸ would be a significant improvement in estimation accuracy!
- visarga 1y ago
  1.6x multiplier is low, we usually need to apply 5x
  - deleted 1y ago
    [deleted]
jahnu 1y ago
> once I know what I need to do AI coding makes the doing much faster Most commenters on this paper seem to not respond to the strongest result from it. That is, the developers wrongly thought and felt that using AI had sped up their work. So we need to be super cautious about what we think we know.

01100011 1y ago

As an old dev this is really all I want: a sort of autocorrect for my syntactical errors to save me a couple compile-edit cycles.

pferde 1y ago
What I want is not autocorrect, because that won't teach me anything. I want it to yell at me loudly and point to the syntactical error. Autocorrect is a scourge of humanity.

causal 1y ago

Agreed and +1 on "always feels like it is almost there" leading to time sink. AI is especially good at making you feel like it's doing something useful; it takes a lot of skill to discern the truth.

i_love_retros 1y ago

The problem is I then have to also figure out the code it wrote to be able to complete the final 20%. I have no momentum and am starting from almost scratch mentally.

qingcharles 1y ago

This is just not true in my experience. Not with the latest models. I routinely manage to 1-shot a whole "thing." e.g. yesterday I needed a Wordpress plugin for a single-time use to clean up a friend's site. I described exactly what I needed, it produced the code, it ran perfect first time and the UI looked like a million dollars. It got me 100% of the way in 0% of the time. I'm the biggest skeptic, but more and more I'm seeing it get me the bulk of the way with very little back-and-forth. If it was even more heavily integrated in my dev environment, it would save me even more time.

curious_cat_163 1y ago

100% agreed. It is all about removing friction for me. Case in point: I would not have touched React in my previous career without the assist that LLMs now provide. The barrier to entry just _felt_ to be too large and one always has the instinct to stick with what one knows. However, it is _fun_ to go over the barrier if it is chatting with a model to get a quick tutorial and produce working code for a prototype (for your specific needs) where the understanding that you just developed is applied. The alternative (without LLMs) is to first do the ground work of learning via tutorials in text/video form and then do the cognitive mapping of applying the learning to one's prototype. I would make a lot of mistakes that expert/intermediate React developers don't make on this path. One could argue that it shortcuts some learning and perhaps the old way results in better retention. But, our field changes so fast... and when it remains static for too long, projects die. I think of all this as accelerant for progress in adoption of new ways of thinking about software and diffusing that more quickly across the developer population globally. Code is always fungible, anyway. The job is about all the other things that one needs to do besides coding.

RainyDayTmrw 1y ago

More sinister is it then takes 80% of the credit.