5 ms·

The killer feature about LLMs with programming in my opinion is autocomplete (the simple copilot feature). I can probably be 2-3x more productive as I'm not typ

The killer feature about LLMs with programming in my opinion is autocomplete (the simple copilot feature). I can probably be 2-3x more productive as I'm not typing (or thinking much). It does a fairly good job pulling in nearby context to help it. And that's even without a language server. Using it to generate blocks of code in a chat like manner in my opinion just never works well enough in the domains I use it on. I'll try to get it to generate something and then realize when I get some functional result I could've done it faster and more effectively. Funny enough, other commenters here hate autocomplete but love chat.

m3kw9 2y ago
The autocomplete is mostly a nusance and maybe low percentage of the time it does right.
- ahoka 2y ago
  The industry standard was 40% accepted the last time I checked. Correct could be a bit lower, so maybe 1/3? It’s like having to delete the auto-closed parenthesis more often than not.
- tptacek 2y ago
  Yeah, I don't like it either. I think it speaks to the mindset difference Crawshaw is talking about here. When I'm writing code, I don't want things getting in my way. I have a plan. I'm actually pretty Zen about all the typing. It's part of my flow-state. But when I'm exploring code in a dialog with a chatbot, I'm happy for the help.
  - switchbak 2y ago
    I think we're going to be considered dinosaurs pretty soon. Much like how it's getting harder to buy a manual transmission, programming 'the old way' will probably just fade away over time.
- LVB 2y ago
  The biggest nuisance aspect for me is when it is trying to do things that the LSP can do 100% correctly. Almost surely it is my tooling setup and the LLM is squashing LSP stuff. Seeing Copilot (or even Cursor) suggesting methods or parameters that don't exist is really annoying. Just stand down and let the LSP answer those basic questions, TYVM.
  - throwup238 2y ago
    Cursor ostensibly has a config setting to run a “shadow” workspace [1], aka a headless copy of the window you’re working in to get feedback from linters and LSPs but they’ve been iterating so fast I’m not sure it’s still working (or ever did much, really). It really feels like we’re at the ARPANET stage where there’s so much obvious hanging fruit, it’s just going to take companies a while to perfect it. [1] https://www.cursor.com/blog/shadow-workspace https://www.cursor.com/blog/shadow-workspace
- jghn 2y ago
  I thought so too. Until I worked with a client who doesn't allow the use of LLM tools, and I had to turn my Copilot off. That's when I realized how much I'd grown to rely on it despite the headaches.
LeftHandPath 2y ago
I’ve never used it, simply because I hate autocomplete in emails. Gmail autocomplete saves me maybe 2-5s per email: the recipients name, a comma, and a sign off. Maybe a quarter or half sentence here or there, but never exactly what I would’ve typed. In code bases, I’ve never seen the appeal. It’s only reliably good at stuff that I can easily find on Google. The savings are inconsequential at best, and negative at worst when it introduces hard-to-pinpoint bugs. LLMS are incredible technology, but when applied to code, they act more like non-deterministic macros.
- deleted 2y ago
  [deleted]
- switchbak 2y ago
  "negative at worst when it introduces hard-to-pinpoint bugs" - this is actually very true. I've had it recreate patterns _partially_, and paste in the wrong thing in a place that was very hard to discern. It probably saved me 40 mins, then proceeded to waste 2 hours of me hunting for that issue. I'm probably at the break-even on the whole. The ultimate promise is very compelling, but my current use isn't particularly amazing. I do use a niche language though, so I'm outside the global optima.
  - LeftHandPath 2y ago
    Exactly! I expect that some are able to put it to good use. I am not one of those people. My experiences with ChatGPT and Gemini have included lots of confident but wrong answers, eg “What castle was built at the highest altitude”. Thats what gives me pause. Gemini spits out a great 2D A* implementation no problem. That is awesome. Actually, contrary to my original comment, I probably will use AI for that sort of thing going forward. Despite that, I don’t want it in my IDE. Maybe I’m just a bit of a Luddite.
imhoguy 2y ago
Both autocomplete and chat are half-way UX solutions. Really what I need is some kind of mix of in-place chat with completion. For context, very often I have to put some comment before the line for completion to set an expectation context. Instead editor should allow me to influence completion with some kind of in-place suggestion input available under keyboard shortcut. Then I could type what I want into such input and when I hit Enter or Tab the completion proposal appears. Even better if it would let me undo/modify such input, and have shortcuts like "show me different option", "go back to previous".
switchbak 2y ago
I had to turn autocomplete off. I value it when I want it, but otherwise it's such a distraction that it both slows me down, and actively irritates me. Perhaps I'm just an old man telling the LLM to get off my lawn, but I find it does bad things to my ability to concentrate on hard things. Having a good sense of when it would be useful, and invoking it on demand seems to be a decent enough middle ground for me. Much of it boils down to UX - if it could be present but not actively distracting, I'd probably be ok with it.
- alexxys 2y ago
  I absolutely love autocomplete in VSCode but hate it in Visual Studio while using the same LLM. The big difference for me is the function AcceptNextWord which Visual Studio doesn't have. A long autocomplete suggestion is rarely completely correct and becomes an annoying distraction but one or several words at the beginning of the suggestion are often correct. So, I usually accept only one or few words in VSCode with a hotkey, then type a bit more, then accept few words more etc. That works great for me. Also, I developed intuition in which pieces of code an LLM suggestion would be most probably useless, so I just ignore any suggestions there to avoid unnecessary distraction. My guess is that many devs who don't like LLM autocomplete, are just unlucky to use a suboptimal UI. As an example, I personally don't understand how some people could like autocomplete in Visual Studio. As you said, it's just too distracting and irritating. BTW, I use Codeium, not Copilot. But I guess they should have the same autocomplete UI which depends more on IDE than LLM.
- gregh30 2y ago
  This mirrors my experience with it as well. Autocomplete is just way too distracting for me, and I could end up writing the code faster myself without having to proofread every suggestion it throws at me. I ended up turning off autocomplete as well and instead use the copilot chat feature when I need it.

m3kw9 2y ago

The autocomplete is mostly a nusance and maybe low percentage of the time it does right.

ahoka 2y ago
The industry standard was 40% accepted the last time I checked. Correct could be a bit lower, so maybe 1/3? It’s like having to delete the auto-closed parenthesis more often than not.
tptacek 2y ago
Yeah, I don't like it either. I think it speaks to the mindset difference Crawshaw is talking about here. When I'm writing code, I don't want things getting in my way. I have a plan. I'm actually pretty Zen about all the typing. It's part of my flow-state. But when I'm exploring code in a dialog with a chatbot, I'm happy for the help.
- switchbak 2y ago
  I think we're going to be considered dinosaurs pretty soon. Much like how it's getting harder to buy a manual transmission, programming 'the old way' will probably just fade away over time.
LVB 2y ago
The biggest nuisance aspect for me is when it is trying to do things that the LSP can do 100% correctly. Almost surely it is my tooling setup and the LLM is squashing LSP stuff. Seeing Copilot (or even Cursor) suggesting methods or parameters that don't exist is really annoying. Just stand down and let the LSP answer those basic questions, TYVM.
- throwup238 2y ago
  Cursor ostensibly has a config setting to run a “shadow” workspace [1], aka a headless copy of the window you’re working in to get feedback from linters and LSPs but they’ve been iterating so fast I’m not sure it’s still working (or ever did much, really). It really feels like we’re at the ARPANET stage where there’s so much obvious hanging fruit, it’s just going to take companies a while to perfect it. [1] https://www.cursor.com/blog/shadow-workspace https://www.cursor.com/blog/shadow-workspace
jghn 2y ago
I thought so too. Until I worked with a client who doesn't allow the use of LLM tools, and I had to turn my Copilot off. That's when I realized how much I'd grown to rely on it despite the headaches.

LeftHandPath 2y ago

I’ve never used it, simply because I hate autocomplete in emails. Gmail autocomplete saves me maybe 2-5s per email: the recipients name, a comma, and a sign off. Maybe a quarter or half sentence here or there, but never exactly what I would’ve typed. In code bases, I’ve never seen the appeal. It’s only reliably good at stuff that I can easily find on Google. The savings are inconsequential at best, and negative at worst when it introduces hard-to-pinpoint bugs. LLMS are incredible technology, but when applied to code, they act more like non-deterministic macros.

deleted 2y ago
[deleted]
switchbak 2y ago
"negative at worst when it introduces hard-to-pinpoint bugs" - this is actually very true. I've had it recreate patterns _partially_, and paste in the wrong thing in a place that was very hard to discern. It probably saved me 40 mins, then proceeded to waste 2 hours of me hunting for that issue. I'm probably at the break-even on the whole. The ultimate promise is very compelling, but my current use isn't particularly amazing. I do use a niche language though, so I'm outside the global optima.
- LeftHandPath 2y ago
  Exactly! I expect that some are able to put it to good use. I am not one of those people. My experiences with ChatGPT and Gemini have included lots of confident but wrong answers, eg “What castle was built at the highest altitude”. Thats what gives me pause. Gemini spits out a great 2D A* implementation no problem. That is awesome. Actually, contrary to my original comment, I probably will use AI for that sort of thing going forward. Despite that, I don’t want it in my IDE. Maybe I’m just a bit of a Luddite.

imhoguy 2y ago

Both autocomplete and chat are half-way UX solutions. Really what I need is some kind of mix of in-place chat with completion. For context, very often I have to put some comment before the line for completion to set an expectation context. Instead editor should allow me to influence completion with some kind of in-place suggestion input available under keyboard shortcut. Then I could type what I want into such input and when I hit Enter or Tab the completion proposal appears. Even better if it would let me undo/modify such input, and have shortcuts like "show me different option", "go back to previous".

switchbak 2y ago

I had to turn autocomplete off. I value it when I want it, but otherwise it's such a distraction that it both slows me down, and actively irritates me. Perhaps I'm just an old man telling the LLM to get off my lawn, but I find it does bad things to my ability to concentrate on hard things. Having a good sense of when it would be useful, and invoking it on demand seems to be a decent enough middle ground for me. Much of it boils down to UX - if it could be present but not actively distracting, I'd probably be ok with it.

alexxys 2y ago
I absolutely love autocomplete in VSCode but hate it in Visual Studio while using the same LLM. The big difference for me is the function AcceptNextWord which Visual Studio doesn't have. A long autocomplete suggestion is rarely completely correct and becomes an annoying distraction but one or several words at the beginning of the suggestion are often correct. So, I usually accept only one or few words in VSCode with a hotkey, then type a bit more, then accept few words more etc. That works great for me. Also, I developed intuition in which pieces of code an LLM suggestion would be most probably useless, so I just ignore any suggestions there to avoid unnecessary distraction. My guess is that many devs who don't like LLM autocomplete, are just unlucky to use a suboptimal UI. As an example, I personally don't understand how some people could like autocomplete in Visual Studio. As you said, it's just too distracting and irritating. BTW, I use Codeium, not Copilot. But I guess they should have the same autocomplete UI which depends more on IDE than LLM.
gregh30 2y ago
This mirrors my experience with it as well. Autocomplete is just way too distracting for me, and I could end up writing the code faster myself without having to proofread every suggestion it throws at me. I ended up turning off autocomplete as well and instead use the copilot chat feature when I need it.