5 ms·
There have been many exaggerated claims surfacing across Twitter and the blogosphere. What I found to resonate best with my own experience is that Copilot is ok
by jmaker 2y ago
There have been many exaggerated claims surfacing across Twitter and the blogosphere. What I found to resonate best with my own experience is that Copilot is okay on simple tasks, but actually misguides, confuses, and in fact breaks your flow of thought on anything beyond that. I like that sort of smart autocompletion and snippet retrieval for boilerplate, templates code, but for the business logic proper it’s just awful. It either doesn’t get it at all but wants to insert some crap and the IDE isn’t always helpful to recognize that it’s distracting you, or it hallucinates something that seems to fit at first sight but once you accept the suggestion moments later you have to undo it because you realize how far off that is, how it misrepresents the domain entities or logic, as if a junior dev suddenly injected that code into your shared file buffer in a pair programming session, while you were focusing. So while Copilot autocomplete is on, you always have to analyze what a gung-ho quasi junior dev knee-jerks into your file buffer.
As many have witnesses, the Copilot Chat has been just terrible. I give up on my hopes for the current wave of the AI evolution.
What is uniform across all LLM is the lack of nuance. Even when they manage to generate some domain specific characters that do make sense, there’s always lack of detail and nuance to the domain, regardless how you try to trick your prompts to retrieve something from a very specific context. I’m impressed by how much easier it’s become for me to get a digest of longer texts, but at the same time I’m disappointed by the quality of the results. It is very rare that I get what I actually ask for. It’s like talking to a mid level consultant who pretends to know everything but the output is rather questionable, and you just give up and seek to end the meeting.
- wslh 2y ago> ...okay on simple tasks, but actually misguides, confuses, and in fact breaks your flow of thought on anything beyond that. Beyond the LLMs themselves, don't you think this is related to the data used for training? I mean, this echoes what you find as a developer on the wild, you can find answers to "all" simple things, much less discrete answers to complex stuff, and almost nothing for very complex stuff and/or specific programming domains.
- meowfly 2y agoI think half the problem is the form factor of using it as an auto complete. I might instinctively know what to write, hit tab, and miss what copilot actually wrote isn't what I wanted to write. For example, it might add when you meant to subtract from a list but my quick scan didn't catch it. So you originally miss it, bump into a bug when writing tests, and spend more time fixing it than if you had just typed it out to begin with.
- geepytee 2y ago>As many have witnesses, the Copilot Chat has been just terrible. I give up on my hopes for the current wave of the AI evolution. Hey, I'd love to hear more about why you think Copilot Chat performs poorly. I personally had a very similar experience and eventually decided to build a solution for it while in YC (shameless plug but I wrote about the Copilot bugs that annoyed me here: https://docs.double.bot/copilot https://docs.double.bot/copilot)