3 ms·

I tried using the pycharm copilot and that lasted a couple days. I think the UI side of picking between copilot and regular autocomplete needs to be ironed out

I tried using the pycharm copilot and that lasted a couple days. I think the UI side of picking between copilot and regular autocomplete needs to be ironed out a bit better, but I think those are short term problems. What actually stopped me was the suggestions for comments. When the grey text appeared as I was typing I realized how much it was guiding what I should think about and how hard it was to express what I wanted. I would try to type my thoughts but be derailed as I saw an alternate wording appear like an unwanted whisper that is just ahead of your thoughts. I think the same was happening in code to a large extent as well but the comments really hammered it home. The simple code is just that, simple. Taking care of the simple code may free you to think about the bigger things, but the copilot grey text whisper in my face as I am typing crushes my bigger thinking and removes all the gains from taking away the small stuff. I don't actually need to be faster at the small stuff. I need to think clearer about the big things. I will check back on this in a bit but for now my reaction is decidedly 'meh'.

mordae 2y ago
One of my first coworkers, my senior at that time, told me that software development is 50% debugging, 40% reading and thinking and only 10% coding. Optimizing those 10% is not that important.
olejorgenb 2y ago
I haven't used the new assistants that much yet, but my impression so far is that automatic suggestions is a bad idea (at least for me). I'd rather trigger it explicitly. That way one can afford a bit more compute per suggestion as well. I think you would get a feel for when the suggestions will be useful and when they won't. One feature I think could be useful is to ask it to correct very basic errors in a section of code, but this need to be well-integrated in the editor to be worth it. This could also be automatic (the detection) - like an slightly more intelligent jetbrains inspection. I've also tried some "whole program" synthesis with mixed results. It usually can get something up and running, but almost every time it has had subtle bugs in it - fixing these (with or without the AI) often got me close or higher to what it would have taken me to do it from scratch. The advantage(?) is I might never have bothered beginning from scratch - getting the first rough draft get you immediately into iteration mode. As an example - I asked GPT-4 to write a python program reading in a json file conforming to a jsonchema[1] and transforming this into a GPX file. It ended up generating code trying to get `properties` keys from the json *file*. So it clearly did not understand the schema. Maybe jsonschema is to obscure? [1] (I provided the schema and a link to the schema standard (though I'm not sure it actually read the link))
- jmward01 2y ago
  I really like the concept of triggering it explicitly. I hope something like that becomes part of the standard UI for these things. Getting the UI right is a major part of how useful these will be.
  - olejorgenb 2y ago
    Cody has that option (I've only tried Gitlab Duo other than that)
joshstrange 2y ago
Interesting. Writing comments is a great use-case of LLMs for me. It gets it right about 50-75% of the time which means way less typing for me. I’ll let the LLM explain what the code is doing and then I’ll add in context specific to this codebase/my company. If it gets it wrong I just keep typing until it guesses the rest or I finish writing the comment.

mordae 2y ago

One of my first coworkers, my senior at that time, told me that software development is 50% debugging, 40% reading and thinking and only 10% coding. Optimizing those 10% is not that important.

olejorgenb 2y ago

I haven't used the new assistants that much yet, but my impression so far is that automatic suggestions is a bad idea (at least for me). I'd rather trigger it explicitly. That way one can afford a bit more compute per suggestion as well. I think you would get a feel for when the suggestions will be useful and when they won't. One feature I think could be useful is to ask it to correct very basic errors in a section of code, but this need to be well-integrated in the editor to be worth it. This could also be automatic (the detection) - like an slightly more intelligent jetbrains inspection. I've also tried some "whole program" synthesis with mixed results. It usually can get something up and running, but almost every time it has had subtle bugs in it - fixing these (with or without the AI) often got me close or higher to what it would have taken me to do it from scratch. The advantage(?) is I might never have bothered beginning from scratch - getting the first rough draft get you immediately into iteration mode. As an example - I asked GPT-4 to write a python program reading in a json file conforming to a jsonchema[1] and transforming this into a GPX file. It ended up generating code trying to get `properties` keys from the json *file*. So it clearly did not understand the schema. Maybe jsonschema is to obscure? [1] (I provided the schema and a link to the schema standard (though I'm not sure it actually read the link))

jmward01 2y ago
I really like the concept of triggering it explicitly. I hope something like that becomes part of the standard UI for these things. Getting the UI right is a major part of how useful these will be.
- olejorgenb 2y ago
  Cody has that option (I've only tried Gitlab Duo other than that)

joshstrange 2y ago

Interesting. Writing comments is a great use-case of LLMs for me. It gets it right about 50-75% of the time which means way less typing for me. I’ll let the LLM explain what the code is doing and then I’ll add in context specific to this codebase/my company. If it gets it wrong I just keep typing until it guesses the rest or I finish writing the comment.