18 ms·

GitHub touts a 55% improvement in coding speed based on a study conducted in-house that tested the participants’ ability to paste a pre-written prompt and then

• 12907835202 2y ago
  This last sentence is the most important. You can describe your dB schema in words and get laravel migration, models, controllers and if you wanted policies, form requests etc all near perfect. You can get a V1 in 10-20 minutes then go about handling all the actual logic
  • rm_-rf_slash 2y ago
    Yeah the copilot doesn’t need to be integrated into every keystroke, just able to analyze the context and kickstart the code. Getting up to speed with new libraries is so much easier with AI instead of the bad old days of trial-and-error-and-marked-as-duplicate
    • gtirloni 2y ago
      Can Copilot analyze my whole codebase these days or just the file I've opened? Honest question as I've stopped to use it a long time ago.
      • selcuka 2y ago
        It seems to be hit and miss, at least with the current PyCharm integration. In some cases it can infer information using other files, in some cases it can't (even if they are open in other tabs).
        • dagw 2y ago
          As someone who switches between PyCharm and VSCode, I find that Copilot seems to work better in VSCode for some reason. Nothing major, but the suggestions I get just seem slightly more relevant to my code base, and are more often what I wanted. Although I could be hallucinating the whole thing.
      • muglug 2y ago
        Whole-codebase understanding is beyond current coding assistants’ capabilities. To do that you need to add in traditional static analysis. But I’m sure people are working on it!
      • theshrike79 2y ago
        With VSCode at least you can say @workspace to Copilot and it'll take it into account. No idea if it feeds all of the code to the LLM or just parts, but it's pretty good at interpreting what the code does
        • dathinab 2y ago
          It guaranteed can't have all of your code in the LLM context, or even all of your file (in case of longer, through not even quite long files). Through it could do stuff like go through your repository(ies) and generate embedding for sections of it and then have a vector database + retrieval argumented generation (RAG) system.
          • theshrike79 2y ago
            Yea, that's the way they're going based on the low-memory models they're building. Basically all the LLM needs to do is translate human writing to some format that a "normal" service can use. That can then leverage the existing spotlight system that's pretty decent at searching stuff on the phone anyway. Then it'll report it back to the LLM which translates whatever format back to something humans can process. Basically "less shit Siri"
    • erhaetherth 2y ago
      I especially like when I'm looking for a particular method on a class I've never used before and it just makes one up for me as though it exists.
      • what-the-grump 2y ago
        It’s adding features that should have existed for you! It can also make up the documentation for the method while you are at it.
  • mlinhares 2y ago
    So rails scaffold from 20 years ago but with a chance of hallucinations?
    • ben_jones 2y ago
      Shhh we’re raising trillions for this technology remember?
    • dudus 2y ago
      It's very easy to find multiple solutions to one problem. The power of LLM is that it's one solution to many problems.
      • marcus_holmes 2y ago
        But is it actually a solution to any of them? The "chance of hallucinations" is the tricky bit - if I have to manually check everything it does in case it's hallucinating, then it's not actually a solution. It's not saving me time (as TFA says).
        • perfmode 2y ago
          while you’re in doubt, i’ll be speeding out
          • marcus_holmes 2y ago
            Speed on, I'll enjoy the slo-mo crash footage later :)
      • LtWorf 2y ago
        What about the problem of needless pollution?
        • dudus 2y ago
          I thought we had moved on from cryptocurrencies
• yreg 2y ago
  My personal feeling is that utilising LLM assistance often isn't faster, but it can take less "stamina", tiring me less.
  • heyoni 2y ago
    Especially when renaming variables that are “immune” to normal refactoring. Copilot handles that pretty well and I don’t have to spend all that focus on such a menial task.
    • eru 2y ago
      Yes, I often find that copilot is pretty good at picking up on the pattern of refactorings that I am doing. Especially those that are a bit tedious and almost mechanical, but not quite mechanical enough to do with a simple search-and-replace.
      • skydhash 2y ago
        Vim and text motions should do the trick.
        • dartos 2y ago
          Sure, but doing the refactor the first time, without thinking about how to record a vim macro, then just hitting tab to have copilot do the same change over and over is a lower friction experience
        • eru 2y ago
          Totally depends on what kind of refactoring you want to do; and how well vim's commands map to your language's syntax, too. (Or whether your vim has special support for your language's syntax.)
    • Semaphor 2y ago
      Sidebar, but what does "“immune” to normal refactoring" mean?
• noobermin 2y ago
  So I am a scientist and I've generally resisted using llms for my work. My general critique reading others' experience is this sounds like what is needed is better abstractions if all it helps people with is doing boilerplate that needs to be checked. Also no need for "better abstractions" to be some ethereal thing, this just means better standardised libraries and frameworks. May be if the word "abstractions" sounds too bespoke may be what is needed is better tools and tooling. The standardised bit seems more difficult than it should be as it's essentially a social problem with development in general, but that also is a problen AI does not address, it's merely a bandaid over the problem.
  • thomashop 2y ago
    It takes literally 10 minutes to try it out. I'd suggest trying it instead of recycling other people's opinions.
• KaiserPro 2y ago
  I have a "top class" LLM based autocomplete provided by work. At first it was a massive pain because I didn't realise it wasn't a "proper" autocomplete(intellisense is probably the king in that regard), and get hit with a large number of hallucinated functions. This was really hard for me, as I'm slightly dyslexic, which means spotting plausible but bullshit completions is very hard. (I suspect its hard for everyone else too). Worse still, at the time the linter/type inspector was/is very slow so only ran on save/execution. However its both improved in the last year significantly, and I have got used to it. For me there are a few techniques that help me: 1) It has a recency bias. Which is great for when you're jumping about in code making changes 2) It rewards proper variable names 3) Your comments should say what, and why your doing what you are doing. 2 & 3 should be obvious and you should be doing it anyway. But it really re-enforces that. However I would really like some UI changes so that I can make _better_ use of the LLM plugin. 1) a completely different colour to indicate that its an LLM suggestion (bonus points for giving a confidence as well) 2) a different keystroke to accept the suggestion. (bonus for partial selection)
• skywhopper 2y ago
  The last entity I would trust for trustworthy data on how much Copilot helps programmers is GitHub. They obviously skew the scenarios and metrics to find the best possible number to report. I think the truth as revealed in your comment and many others is that it all depends on the context. For repetitive code or boilerplate, or starting new projects in well-known frameworks and languages, it probably does increase velocity. The other context factor is the programmer themselves. Their familiarity with the problem domain, the language, and the framework all matter, as does the personality and coding style of the individual.
  • claytongulick 2y ago
    It also depends on what tools a developer is already using as a productivity booster. For example, I use vanilla web components which have some boilerplate for new components. I already have a simple vscode snippet that does the job well, with no hallucinations [1]. I've experimented with llms doing the same thing with not great results. It took me longer to explain what I wanted than it did for me to just write that snippet. Doing it repeatedly and waiting for the results definitely didn't increase my speed, though I was impressed that it was eventually able to figure it out (vanilla web components with lit-html renderer isn't a super common technique). Also, I prefer the pythonic approach of snake-case local variable names. Getting the llm to do that in a js project where it's not super common was another whole iteration. We've had code generation tools around for decades to deal with repetitive boilerplate tasks. Maybe they aren't quite as capable as when the llm "gets it right" - but I wonder with these productivity claims how they are measured. Are they starting from zero and a new developer? Or comparing against an experienced developer with proficiency with lots of workflow enhancers like templates, snippets, vim macros, etc... [1] https://gist.github.com/claytongulick/e4251c1b27b22c25fb68f31761b0f293 https://gist.github.com/claytongulick/e4251c1b27b22c25fb68f3...
• scaryclam 2y ago
  Coding speed really is a horrible metric. I can code really quickly, but it doesn't mean I'm doing anything productive or correct. I'd rather slower coding speed, properly written as it provides higher overall velocity. And velocity should take into account the refactoring that happens months or even years later. Crappy code can look really fancy, and even be bug free, but if it's overengineered and hard to change, it can create long change times or even a full stop in development later in the products lifecycle. And that's on top of developers losing the understanding of how something actually works. If AI helps create the code that would have been written without AI, then great, but I don't observe that happening, and the code has never been a better idea than then dev could have done without it.