3 ms·

Hm, are other AI Assistants much more than just a bit fancier autocomplete? We're using CoPilot at work. When we were evaluating this, the question we asked ou

by tetha 2y ago

Hm, are other AI Assistants much more than just a bit fancier autocomplete? We're using CoPilot at work. When we were evaluating this, the question we asked our test group: How much time does it save you per week? And most people arrived at estimations of like 1-4 hours saved a week, especially when banging out new code in a code base with patterns and architecture. This was a good enough tradeoff to buy it. Like, I recently got a terraform provider going for one of our systems. Copilot was useful to generate the boilerplate code for resources so I just had to fill in the blanks of actual logic. Or you can just hand it sample JSON and it creates go structs for those to use in an API client, and generates the bulk of methods accessing APIs with these. Or it's decent at generating test cases for edge cases. It doesn't enable me to do things I could not do before, but it saves me a lot of typing. Well maybe I wouldn't test my functions with all of these edge cases because that's a lot of stuff to do, heh.

register 2y ago
My experience with Copilot's productivity estimates differs significantly from these figures. As someone who rarely consults Stack Overflow, I've found their estimates to be exaggerated. During a recent 2-hour coding session (yesterday), I found Copilot's contributions to be minimal. The only substantial benefit was a single autocomplete suggestion for a unit test, which saved me approximately 5 minutes. While this may seem insignificant, it represents a mere 4% time savings in a focused coding session.Interestingly, I rejected at least 70% of Copilot's suggestions during this session. Most were either obvious or didn't align with my coding style. Your estimates appear to be based on scenarios where developers frequently consult external resources like Stack Overflow. Given that I don't typically seek help from such platforms, these estimates do not reflect my experience at all.
- vbezhenar 2y ago
  For me Copilot saves a sizeable chunk of coding time. That said, it might be because of my style. I often prefer writing repetitive dumb code, so Copilot fits perfectly, auto-completing obvious code which still must be written. Before Copilot I often used things like vim macroses, regex search&replace, multiple cursors and so on, but of course they take more time that hitting "<Tab>". At the same time Copilot is very smart at getting patterns and in my experience absolutely reliable, it never makes simple things wrong. Also I can note, that some languages are just naturally fit for Copilot, one of them is Go with its constant `if err != nil` incantations.

by tetha 2y ago

My experience with Copilot's productivity estimates differs significantly from these figures. As someone who rarely consults Stack Overflow, I've found their estimates to be exaggerated. During a recent 2-hour coding session (yesterday), I found Copilot's contributions to be minimal. The only substantial benefit was a single autocomplete suggestion for a unit test, which saved me approximately 5 minutes. While this may seem insignificant, it represents a mere 4% time savings in a focused coding session.Interestingly, I rejected at least 70% of Copilot's suggestions during this session. Most were either obvious or didn't align with my coding style. Your estimates appear to be based on scenarios where developers frequently consult external resources like Stack Overflow. Given that I don't typically seek help from such platforms, these estimates do not reflect my experience at all.

vbezhenar 2y ago
For me Copilot saves a sizeable chunk of coding time. That said, it might be because of my style. I often prefer writing repetitive dumb code, so Copilot fits perfectly, auto-completing obvious code which still must be written. Before Copilot I often used things like vim macroses, regex search&replace, multiple cursors and so on, but of course they take more time that hitting "<Tab>". At the same time Copilot is very smart at getting patterns and in my experience absolutely reliable, it never makes simple things wrong. Also I can note, that some languages are just naturally fit for Copilot, one of them is Go with its constant `if err != nil` incantations.