11 ms·

I've been using Copilot non-stop on every hobby project I have ever since they've let me in (2021/07/13) and I am honestly flabbergasted they think it's worth 1

• baby 4y ago
  I'd be willing to pay more than 10$/month for it personally. It has a greater impact than Netflix on my day-to-day. Also, I would expect my work to expense it.
• w4ffl35 4y ago
  I've also been using this for months, and would not pay for it. I think i might be getting it for free actually as I haven't been asked to pay yet. I came to the same conclusion as you, you can see comments I made elsewhere in this thread. I'm not thrilled with it.
  • nacs 4y ago
    It has been free but now they're making it a "free 60 day trial" followed by $10 a month. I've tried it with a few projects with different languages and it's not worth anything close to that $10/m fee personally. It's OK at filling in a line here and there if it's boilerplate-type code but otherwise, it's like a beginner programmer at best.
• jerf 4y ago
  "how to put a sentence together describing the rules, it absolutely doesn't actually understand how "Chii" melds work" The more experience I get with GPT-3 type technologies, the more I would never let them near my code. It wasn't an intent of the technology per se, but it has proved to be very good at producing superficially appealing output that can stand up not only to a quick scan, but to a moderately deep reading, but still falls apart on a more careful reading. At least when that's in my prose it isn't cheerfully and plausibly charging the wrong customer or cheerfully and plausibly dereferencing a null pointer. Or to put it another way, it's an uncanny valley type effect. All props and kudos to the technologists who developed it, it's a legitimate step forward in technology, but at the same time it's almost the most dangerous possible iteration of it, where it's good enough to fool a human functioning at anything other than the highest level of attentiveness but not good enough to be correct all the time. See also, the dangers of almost self-driving cars; either be self-driving or don't but don't expect halfway in between to work well.
  • omginternets 4y ago
    I wholeheartedly agree with your analysis, but feel like it’s ignoring the elephant in the room: writing code is not the bottleneck in need of optimization. Conceiving the solution is. Any time “saved” through Copilot and it’s ilk is immediately nullified by having to check it’s correctness. From there, the problem is worsened by the Frankensteinesque stitching together of disparate parts that you describe. I can’t imagine how Copilot would save anything but a negligible amount of effort for someone who is actually thinking about what they’re writing.
    • paskozdilar 4y ago
      Can Copilot write tests? That way it could test its own code and tweak it until it works. Of course, one would then ask how to verify tests. I suppose Copilot could write meta-tests - tests that verify other tests. That way it could test its own tests and tweak them until they work. Of course, one would then ask how to verify meta-tests. I suppose Copilot could write meta-meta-tests - tests that verify meta-tests. That way it could test its own meta-tests and tweak them until they work. Of course, one would then ask how to verify meta-meta-tests...
• cjauvin 4y ago
  It's interesting to draw parallels between the way you describe it and the way more general large language models (LLMs, of which Copilot is in a sense, a specialized instance, applied to code, instead of general language) operate: they also always "know" how to answer any specific question, or how to complete any prompt, without any exception. A model which would be able to "show restraint", and "know when it doesn't know", would be a really impressive improvement to this technology in my opinion.
  • visarga 4y ago
    There are language models that have an internal search engine, they can copy/verify the facts they generate from the source. They are also easier to update, just refresh the search engine. Now you have to provide a collection of "true facts".
    • YeGoblynQueenne 4y ago
      Could you please link to some examples of such systems? Thanks in advance!
• threatofrain 4y ago
  Github Copilot is on the fence for me between yes/no at $100 per year. I agree that you should rarely if ever allow Copilot to write multiple lines, as your double-checking or debugging time is going to exceed your time savings — the probability of good-looking but bad code is just that high right now. In order to experience a net time-win you'll likely want to be an intermediate at whatever you're doing. If it were $60 yearly it'd be an auto-yes for me.
  • rexreed 4y ago
    Your decision making delta is really $40 a year? How much is your time worth?
    • threatofrain 4y ago
      The time savings win is really that marginal. I'm not sure I can save more than 3 hours per year with Copilot. And this isn't saving 3 hours in a single week, this is saving a few seconds here and there accumulated over a year. Saving time with Copilot is itself a learning process and a probabilistic affair. Copilot can win you a few seconds at a time, but can easily set you back minutes if you aren't careful or experienced. It's the probability of a downward spike in time-win that makes it such a gamble. Such complex deals just turns on the cautious side of my brain.
      • tnorthcutt 4y ago
        Just to clarify: are you saying you'd pay $20 for an hour saved, but not $33.33 for an hour saved?
        • threatofrain 4y ago
          I'm saying that at $100 yearly I'm on the fence of maybe yes or no. At $60 yearly I'm auto-yes without having to think in rational terms. I guess I'm just not at that place in life where $100 is the tier in which I think emotionally. Also, if I magically knew that I could save you 3 hours yearly, but it were spread out over the course of a year, and that your savings would occasionally spike down into negative and then slowly climb up, I just wouldn't entertain such a complex offer at such low numbers. People pay insurance just to avoid such incidental downward spikes. Copilot's biggest limitation right now is that you can't dare to allow minutes of savings per day without inviting the risk of a severe spike in debugging time, the kind that wipes out all your savings. This means you cannot spike up.
• rexreed 4y ago
  Of course GPT-3 doesn't "understand" what you are doing. All it's doing is generating high probability text based on a huge training corpus. It's guessing what text will come next. That doesn't mean it understands jack squat. It's basically a parrot with a huge database. Polly want a program?
  • toyg 4y ago
    The fact that such a program is so hyped, is actually an indicator of how much boilerplate and wheel-reinvention goes on among programmers every day. The state of the sector is somewhat embarrassing. We have armies of monkeys well-paid to bang out the same Java/Javascript/C#/Python over, and over, and over...
    • rexreed 4y ago
      Microsoft spent a Billion dollars for an exclusive license of GPT-3, and now they want their return. Expect to see GPT-3 hyped on every platform (including Github).
  • eurasiantiger 4y ago
    That is also how your brain works.
• staticassertion 4y ago
  To me it's easily worth twice that, so I'm happy to pay 100/year.
  • titaniczero 4y ago
    Jetbrains costs $150/y. IMO, an autocomplete is not worth $100/y at all, let alone $200/y.
    • staticassertion 4y ago
      I also happily pay for Jetbrains.
• jmkni 4y ago
  > I've been using Copilot non-stop > I am honestly flabbergasted they think it's worth 10$/mo These two statments seem contradictory to me. Why are you using it 'non-stop' if it isn't even worth $10/month?
  • soraki_soladead 4y ago
    My interpretation is that it's fun to use so they use it a lot but not altogether useful (eta: and/or necessary): they didn't miss it on work projects.
  • foerbert 4y ago
    I don't think they are contradictory at all. Aside from the basic "it has value, just less than $10/month" option, they may also just be interested in the tech and are evaluating it in actual use, etc.
  • raunak 4y ago
    Not OP - I use it non-stop for boiler plate filling as well. I wouldn't use it for anything other than that, so I would say it's worth honestly at max $1/month.
    • tedunangst 4y ago
      $1 is worth what, one minute of your time? If you're using a tool that doesn't even save you one minute, why bother?
      • presentation 4y ago
        Most people don't make purchasing decisions based on the value they create but rather based on some ingrained assumptions about how expensive software is supposed to be. VSCode and many other complex pieces of software are free, autocomplete is built into my OS, and those subscription consumer software that does have a price usually are priced very low—so relative to those, $10/month feels like a lot (even though I hope that practically anything anyone makes the effort to subscribe to produces at least $10 of value for them). Some companies seem to be leaning into higher subscription pricing (Superhuman and Motion come to mind) and almost certainly produce far more value than their subscriptions cost if you ask me, but there's definitely a mental barrier to value based pricing to consumers, as well as the fact that with so many companies offering cheap/free software, the market isn't solely determined by value created but rather comparison against other software.
• yosito 4y ago
  > Due to the licensing concerns I did not use CoPilot at all in work projects I've rarely found that CoPilot produces more than a line or two of accurate code. How likely is it that one would run into licensing issues with a single line of code that looks similar to something from another codebase? While I understand the problem in principle, I am really skeptical that significant licensing issues would really come up with using CoPilot as an individual.
• euos 4y ago
  I’ve been using it as well. As annoying as it is, I am sure I would miss it enough to pay $100/year. Luckily, I somehow qualified for free access…
• wiremine 4y ago
  I'm the CTO for a small(ish) software consultancy. $10/month is a no-brainer price for just the "amazing all-rounder autocomplete". Spending $10/month/dev to help maximize highly billable engineers? It's well worth the price.
  • BrandonJung 4y ago
    would you prefer it trained ONLY on your code or are you ok with the broad use of non-permissive code used for CODEX?
    • bjconlan 4y ago
      Yeah I was thinking this... Does codex provide other directed sources of learning. I guess I should look deeper into textsynth/Bellard's gpt stuff as it would be nice to have some specialised learning here like only http level interfaces with client server or whatever... But a rails like gen probably gets me there way faster
  • deleted 4y ago
    [deleted]
• blue0bird 4y ago
  I've been using Copilot for almost 10 months, useful when learning new code but after a while become a bit more advanced auto complete. I think it is good for short lines, repeating tasks; for example when writing tests and want to assert different fields, assert string, int, etc; for these sort of lines was really good and fast. my main problems: 1. sometimes make a horrible mistake, takes couple of minutes to understand 2. repeat the same mistake over and over 3. adding a single tab take a bit of time, had to copy & paste tab to avoid copilot suggestion!
• matthewmacleod 4y ago
  A developer easily costs $100 an hour. This means that if Copilot saves you more than 360 seconds in a month it’s paid for itself. I’m honestly flabbergasted that anybody would think it isn’t worth $10 a month, despite its many serious flaws.
  • Smaug123 4y ago
    That calculation works if it doesn't also cost you any time - which by all accounts it does, e.g. in review.
• austenallred 4y ago
  I'm genuinely curious: How do you value your time? If you're an engineer who is paid $150/hour and Copilot saves you 5 minutes/month it just paid for itself.
  • kojeovo 4y ago
    Hobby projects don't make any money so it doesn't make sense if I pay for it to be more productive for the company. If they pay, great.
• mceachen 4y ago
  > licensing concerns I dismissed these concerns before I had early access. Then, _literally the first characters I typed_ after enabling the extension were `//`, and it autosuggested: // Copyright 2018 Google LLC I immediately uninstalled it. https://mobile.twitter.com/mrm/status/1410658969803051012/photo/1 https://mobile.twitter.com/mrm/status/1410658969803051012/ph...
  • eat_lemons 4y ago
    Both on a moral and practical sense copilot is a licensing nightmare
• BrandonJung 4y ago
  Licensing is a critical question that is often not considered. Code trained on non-permissive code (think Oracle API's) has very significant risk, ask Google. We took a different tact three years ago in building Tabnine BUT went with only fully permissive code for training, ability to train on your own code base, and zero sharing of your completions. Also we give the developer the flexibility to adjust the length of completions if you want faster shorter suggestions.
• datastack 4y ago
  Interesting to hear your experience. I've been using it for over a year, and I've come to appreciate the (modest) productivity boost that it's given me, to the point that I feel $10 per month is probably worth it. The completions are often trivial, but they save me from typing them by hand. Sometimes they are trivial yet still wrong so I need to make corrections, wasting some of the gained speed. In total these probably won't save me much time on a day. However, every couple of days there is one of these cases, where it can do tedious work that really saves time and headaches. Example: - After writing a Mapper that converts objects of type A to B, I needed the reverse. Co-Pilot generated it almost perfectly in an instant. This can easily save a minute or two, plus the thinking required. - For a scraper, I needed to add cookies from my browser into the request object. Basically, I pasted the cookie in a string, and typed `// add cookies`, and it generated the code to split the string, iterate over each cookie value and add it to the correct request field. So if a few of these cases can save 10 minutes in a month, I feel it's objectively worth it. Then subjectively, not having the headaches of 'dumb stuff'/boilerplate feels great, and I am glad to spend my energy on the actual hard stuff. I will sign up as soon as their sign up page lets me.
• qualudeheart 4y ago
  As a productivity booster I think it’s worth more than $10. The licensing problems make it impossible to use at work so I won’t use it for that. People need to be aware of the security risks of letting microsoft read all your code as it’s sent to the servers copilot runs on. By my lights that’s almost as big of a problem as licensing.
• Gigachad 4y ago
  I turned off copilot a week ago for the same reason. The code it generates _looks_ right but is usually wrong in really difficult to spot ways but things you’d never write yourself.
• hn_throwaway_99 4y ago
  I think your response highlights why individual developers are the worst target market and why you want to sell to businesses if you're in the tool space. Let's say the average developer in the US costs 10k a month (I think that's pretty close to the real average of around 120k a year). So copilot would cost .1% of that developer's salary. I realize calculating things around "improvements in developer productivity" involve lots of fuzzy math, but it would be stupid for any company NOT to pay this if it improves developer productivity by just 1%. Another way to think about it that I think may be more "real world": Let's say I'm CTO of a big company with 1000 software developers. Do I think it's going to be a better investment to hire another developer so I have 1001 developers, or instead use that other developer's salary to buy all the devs at my company a Copilot license? But for some reason individual developers think that anything over $1-2 dollars a month is an exhorbitant cost.
  • richiebful1 4y ago
    If you're an employee what percentage of that productivity increase accrues to you? I would not spend $10/month on a code completor for my job, because I would probably never see those $10 back in salary. I doubt the company would even notice the minor bump in productivity, say $100 a month
• SmellTheGlove 4y ago
  +1. I've also been using it for hobby projects and have largely the same conclusions. I really do like that it spits out boilerplate for me, but when doing more than that, I still have to double-check all of it because as you said it does create incorrect, good looking output. I can't justify $10 a month for it. Maybe as it improves. EDIT: To clarify, $10 a month for personal use. We can't use it at work due to licensing, or it'd be worth that just to emit boilerplate.
• naniwaduni 4y ago
  > - It's terrible if you let it write too much. The biggest problem I've had is not that it doesn't write correctly, it's that it think it knows how and then produce good looking code at a glance but with wrong logic. So the same problem ML has in every endeavor where we have a good metric of "correctness" that's distinct from plausibility, like OCR or natural language translation: very good at spitting out stuff that superficially resembles training data, and whether that happens to be right is totally accidental. Surprisingly good odds if you're working on something boring within the "bounds" of the model, sure, but also pretty likely to think that "now on sale" is a zillion times more likely to be announced on an advert than "a decision has been made to release this product (at an unspecified future date)."
• keithnz 4y ago
  working out what it is worth is tricky, I just look at what I'm paying for at the moment, like Jetbrains All product suite costs me $150 yearly.... so Copilot at $100 a year seems insanely out of proportion. In fact all of the SAAS type products that are around $10 a month per dev offer far more significant functionality. For me copilot mainly fills in boiler plate code, which is useful, from time to time it generates a function that's great, but it would have been trivial for me to write it too, I like it, but comparatively compared to other tools, its pricing seems out of whack. Some are trying to use time = money as justification, but it rarely works as a direct translation like that when coding, in fact I doubt it has that much impact on time, it just makes some things less mundane to do. I have other plugins that are free that probably save me way more time, if all the plugins that help started charging at the same rate based on time savings, it'd be a nightmare.
• culi 4y ago
  I've used it only slightly longer and had pretty much the same experience. I find it hard to believe anyone is really letting copilot write more than 2 lines of code for them at a time. It's an AI powered autocomplete. And honestly it's excellent at that. All I really want is an AI powered autocomplete and if a FLOSS project took up the challenge I'd happily donate $10/month to see it succeed. Especially if it meant none of the licensing concerns that come with GH Copilot
• gideon_b 4y ago
  it thinks it knows how and then produce good looking code at a glance but with wrong logic. This is so accurate. I still like copilot and I might even pay for it, but I will never trust the logic. It always wrong in a way that _almost_ looks right.