24 ms·

I wholeheartedly agree with your analysis, but feel like it’s ignoring the elephant in the room: writing code is not the bottleneck in need of optimization. Co

• paskozdilar 4y ago
  Can Copilot write tests? That way it could test its own code and tweak it until it works. Of course, one would then ask how to verify tests. I suppose Copilot could write meta-tests - tests that verify other tests. That way it could test its own tests and tweak them until they work. Of course, one would then ask how to verify meta-tests. I suppose Copilot could write meta-meta-tests - tests that verify meta-tests. That way it could test its own meta-tests and tweak them until they work. Of course, one would then ask how to verify meta-meta-tests...
  • DeathArrow 4y ago
    >Can Copilot write tests? That way it could test its own code and tweak it until it works. Sure it can. But you can't rely on them being good. You have to read the tests carefully.
  • prepend 4y ago
    You need an adversarial co-pilot to write tests for those tests so you would put the two AIs against each other to try to properly test.
  • datatrashfire 4y ago
    I can almost envision a future where human devs write tests, code generating frameworks build code from a spec.
    • paskozdilar 4y ago
      I think that's the underlying idea of Logic Programming.
      • goatlover 4y ago
        And UML diagrams.
  • Hnrobert42 4y ago
    Isn’t this how they managed to stop the Borg?
    • frereubu 4y ago
      Given my vintage, I'm thinking more about the noughts-and-crosses game in War Games from 1983.
  • Hamcha 4y ago
    I've used Copilot to help with writing verbose unit tests. It can do it as long as you keep an eye over it (basically like an autocomplete), it definitely cannot produce robust test cases on its own though. If you try to do that, it won't take "meta-tests" to figure out they don't look right.
• vosper 4y ago
  > I wholeheartedly agree with your analysis, but feel like it’s ignoring the elephant in the room: writing code is not the bottleneck in need of optimization. Conceiving the solution is. I dunno about this. I know the received wisdom is that "writing the code isn't the hard part", but I think reality is more like "writing the code is only one of the hard parts". There's an awful lot of badly-written code, or code which is only partly correct, or only correct under some circumstances. The only way to make writing code not one of the hard parts is to specify 100% of the functionality, every corner case, and all test scenarios, before any code is written. And then you still have to verify that it was translated correctly into code, which I think we can all agree is another one of the hard parts! Conceiving the solution is hard, thinking of edge cases, what-ifs, and failure scenarios is hard, creating effective tests is hard, and writing the actual code understandably and correctly is also hard!
  • omginternets 4y ago
    Yes, I think you’ve more precisely articulated what I had in mind. The point stands, though: codepilot does not help with the hard part of the job. It solves a problem that only exists for people who aren’t exercising care.
  • jgwil2 4y ago
    Writing the code is the hard part mainly inasmuch as it forces you to concretize and clarify previously vague notions. In that sense, it's hard to separate "conceiving the solution" from "writing the code," unless you're perhaps one of those rare geniuses who are simply able to dictate ideas fully formed in their head (I'm thinking of the scene in Amadeus when Salieri examines some clean, uncorrected sheet music and is then shocked to discover that Mozart doesn't make copies and he's holding originals).
  • marcosdumay 4y ago
    > I know the received wisdom is that "writing the code isn't the hard part", but I think reality is more like "writing the code is only one of the hard parts". Writing the code isn't the bottleneck. And there is no point in optimizing some part of a process that isn't a bottleneck. Anyway, have you noticed that "understandably and correctly" isn't included on the OP's definition of "writing code"? That's for a reason, and it's the most adequate definition to use on this context.
• bryanrasmussen 4y ago
  >I can’t imagine how Copilot would save anything but a negligible amount of effort for someone who is actually thinking about what they’re writing. since I have a right arm swelled up to twice normal size right now and it hurts to type for more than ten minutes (hopefully ok in a few days) I can imagine an advanced autocomplete being really useful for some disabilities.
  • omginternets 4y ago
    More so than, say, classical snippets, auto-complete, and speech-to-text? And pray tell, how much typing is required to go back and fix the incorrect code produced by copilot? P.S.: wishing you a speedy recovery!
    • bryanrasmussen 4y ago
      the person using co-pilot on all their hobby projects described it as working best as an advanced auto-complete, so I guess you should ask them that. I figure advanced auto-complete should not produce big blocks of code that are more likely to have logical errors in them, since the grandfather comment here suggested that problems show up when you generate larger blocks of code.
      • BrandonJung 4y ago
        this is consistent with the feedback from many of the users we have talked to as well (transparently I am with Tabnine). Long blocks of code are difficult to digest while short quick ones can be very quick AND easy to validate the logic.
• teawrecks 4y ago
  If you have a sufficiently well defined solution to a problem, then you have the code. The next step is just to compile it into something a machine understands. In other words, the code IS the solution, there is no difference between the two.
  • danachow 4y ago
    Only for the most trivial problems. Having seen the same problem implemented both with a spaghetti ball of shit vs something well organized that can be easily read and maintained I’m going to hard disagree on this sentiment.
• 3pt14159 4y ago
  Right on the money. What I want is a copilot that finds errors ala spellcheck-esque. Did I miss an early return? For example in the code below def some_worker if disabled_via_feature_flag logger.info("skipping some_worker") some_potentially_hazardous_method_call() Right after the logger call I missed a return. A copilot could easily catch this. Invert the relationship. I don't need some boilerplate generator, I need a nitpicker that's smarter than a linter. I'm the smart thinker with a biological brain that is inattentive at times. Why is the computer trying to code and leaving mistake catching to me? It's backwards.
  • O_H_E 4y ago
    > I need a nitpicker that's smarter than a linter. I'm the smart thinker with a biological brain that is inattentive at times. Why is the computer trying to code and leaving mistake catching to me? It's backwards. Hmmmm, that is actually a good observation.
  • tsbertalan 4y ago
    After a few more leading attempts, I managed to give this prompt at https://beta.openai.com/playground https://beta.openai.com/playground: What problem does the following pseudocode have? def some_worker if disabled_via_feature_flag logger.info("skipping some_worker") some_potentially_hazardous_method_call() And receive this response: The problem with this pseudocode is that there is no "end" keyword to close off the "if" statement. This means that the code after " some_potentially_hazardous_method_call()" will always be executed, even if the "disabled_via_feature_flag" condition is true. And that's with a GPT3 without any special fine tuning. Of course, the name `some_potentially_hazardous_method_call` is pretty leading in itself. I rewrote the prompt slightly more realistically, as: What problem does the following code have? def optionally_do_work(): if disabling_flag: logger.info("skipping the work due to flag") do_work() and received: The problem is that the code will still try to do the work, even if the flag is set. This does seem like a pretty trivial easier-than-fizzbuzz question to be asking, though, since it's so encapsulated.
  • rasz 4y ago
• zamfi 4y ago
  I also agree that I’d never assume copilot is right when it blurts out code, and that “writing code” is not the hard part — but I’d note three things I found from using copilot pretty intensively over the past year or so: 1. It has shifted some of the code-writing I do from generation to curation. Most of the time, I have to make some small change to one of the first options I get. Sometimes I don’t. Sometimes I get some cool idiomatic way of doing something that’s still wrong, but inspires me to write something different than I originally planned. All of these are useful outcomes — and unrelated to whether someone is “actually thinking about what they’re writing”. 2. It has changed my tolerance for writing redundant code, for the better. Like many programmers, I tend to optimize my code for readability first, and then other things later when I have more information. Sometimes, my desire for readability conflicts with my desire for code that avoids redundancy (e.g., “oh but if I put these three cases into an array I can just use a for loop and don’t have to write out as much code” etc. etc.) — and my old bias was avoiding redundancy more often than not. But copilot is really great at generating code that has redundancy, which has often helped me write more readable code in quite a few cases. 3. I refactor code way more now. In part this is because, given code that already works but is not ideal (e.g., needs to be broken into more functions, or needs extra context, or some critical piece needs to be abstracted), copilot does a fantastic job at rewriting that code to fit new function prototypes or templates. IDEs can help with this task, for a few common types of refactoring, but copilot is way more flexible and I find myself much more willing to rewrite code because of it. Copilot is not what many people want it to be, in much the same way that Tesla’s Autopilot is not what many people want it to be. But both do have their uses, and in general those uses fall into the category of “I, as human, get to watch and correct some things instead of having to generate all things.” This can be very useful. (FWIW, it takes some time to adapt to this; I teach and mentor a lot and I found myself relying on those skills a ton when working with copilot.) We shouldn’t discount this usefulness just because these systems don’t also have other usefulness that we also want!
• TillE 4y ago
  Part of the pitch is that it helps you learn new languages, which I do sort of buy. But yeah, the hard part of writing nontrivial software isn't typing code, it's the software architecture and design.
• jan_Inkepa 4y ago
  I swap between programming languages a lot and copilot saves me a lot of "what's the syntax for for loops in language X again?" style friction, stuff with suggesting correct API usage patterns . It just saves on the friction of writing random scripts.
• culi 4y ago
  I don't think you've actually used it tbh. It's much quicker to read code than to write it. In addition, 95% of Copilots suggestions are a single line and they're almost always right (and also totally optional). Here's how it actually works in practice 1. Start a line to do an obvious piece of code that is slightly tedious to write 2. Type 2 characters and Copilot usually guesses what you want based on context. Perhaps this time it's off. 3. No matter, just type another 3 characters or so and Copilot catches up and gives a different suggestion. I just hit "tab" and the line is complete It really shines in writing boiler plate. I admit that I'm paranoid every time it suggests more than 2 lines so I usually avoid it. But in ~year of using it I've run into Copilot induced headaches twice. Once was in the first week or so of using it. I sweared off of using it for anything more than a line then. Eventually I started to ease up since it was accurate so often and then I learned my second lesson with another mistake. Other than that it's done nothing but save me time. It's also been magnificent for learning new languages. I'll usually look up it's suggestions to understand better but even knowing what to look up is a huge service it provides
  • deleted 4y ago
    [deleted]
• eurasiantiger 4y ago
  You haven’t tried Copilot, have you?
  • Winsaucerer 4y ago
    I have, and it was producing impressive results. However, I was trying to learn Rust and to do so I needed to do the hard yards myself. Moreover, in my brief time using it, it was switching me to a code reviewing frame of mind rather than thinking through the problem. I thought it might be more useful to me for a language I’m already good at, or one I’m not trying to master but just need to get a task done for.
  • deleted 4y ago
    [deleted]