12 ms·

"Our diagnosis is that individual developers do not pay for tools. Their manager might, but engineering managers only want to pay for discrete new capabilities,

• arcturus17 4y ago
  Loads of devs I’ve spoken to, from junior to principal level, absolutely love Github Copilot though. Don’t know who is paying it for them, nor if Kite was significantly worse, but I think that at least Copilot has a brilliant future ahead of it.
  • dreamyfigment 4y ago
    I _love_ Copilot but the only reason I use it is because I qualify under their open source developers program, I just can't justify paying $10/month for it.
    • robertlagrant 4y ago
      How much do you love it, then? What would you pay, if it's less than $10/mo?
      • didibus 4y ago
        It saves a bit of time, but doesn't seem to make a difference on time to market of features, products, improvements or bug fixes. In my experience, it's a quality of life improvement, but the things that dictate actual time to market is bottlenecked by things that aren't solved by copilot, such as overall design, decision making, requirements gathering, code structure/architecture, solution ideation, user acceptance, infrastructure setup, etc. I think if it eventually could help with those other tasks, you'd see time to market gains, and that would start to make it really valuable.
    • deleted 4y ago
      [deleted]
  • candiddevmike 4y ago
    Until Microsoft grants Copilot users blanket protection over copyright claims from Copilot generated code, I wouldn't even think of touching it.
    • epolanski 4y ago
      Same. Never ever I'm risking breaking copyright, and I also don't like Microsoft not including their own code in the model.
      • hirvi74 4y ago
        > I also don't like Microsoft not including their own code in the model From allegations I have read across the Internet, Microsoft might be doing those who use Copilot a favor.
    • Jenk 4y ago
      We may well find out the answer to that when this lawsuit[0] concludes. [0]: https://www.theverge.com/2022/11/8/23446821/microsoft-openai-github-copilot-class-action-lawsuit-ai-copyright-violation-training-data https://www.theverge.com/2022/11/8/23446821/microsoft-openai...
      • candiddevmike 4y ago
        If Copilot generates the exact same code as the source, I don't see how that process could be exempt, it's like using the clipboard on your PC with extra steps.
        • nl 4y ago
          It can generate the same code, but generally won't.
          • drekipus 4y ago
            So Microsoft should provide blanket protections to the developers that use it?
  • urthor 4y ago
    It's interesting, the ones I've spoken to are extremely suspicious. "GitHub Copilot blocks your ability to learn." Is a common refrain. I don't see ANY industry-wide consensus on whether GitHub Copilot truly helps developers right now. The only scenario I can get anyone to agree on is generating templates. Aka, JSON or CSS files that you then edit.
    • satvikpendem 4y ago
      I use it. It's good for boilerplate code or basically anything where you can avoid looking at the docs. For example, I was writing an ML training loop and it correctly filled in the rest of the function after I wrote the first few lines. The code is basically what's in the pytorch docs, just fit to my model and scenario.
    • hanselot 4y ago
      The Internet blocks your ability to learn...
    • throwaway675309 4y ago
      It doesn't block your ability to learn any more than any auto suggestion systems. But I guess it depends on what you value in terms of learning, for me, copilot allows me to focus on the larger architectural problems while not having to worry about the exact syntax of certain things (DSL query language, middleware express, typescript def annotations, etc). Every time I don't have to context switch to look up some technical errata in my browser is a complete win for me.
      • Gigachad 4y ago
        I observed one guy become very resistant to reading the docs as they believed copilot obsoleted them. Programming in Ruby, the copilot suggestions will run but since you didn't read the docs you won't know that the output doesn't actually work or doesn't do what you think it does since it was intended for a different situation.
    • nl 4y ago
      Everyone I've spoken to who has actually used it thinks it works extremely well. This includes pretty experienced developers. I've never heard anyone claim it blocks the ability to learn - if anything it's the opposite. Many people like how it shows you APIs you weren't aware of.
  • smohare 4y ago
  • jonas21 4y ago
    Some well-known devs like Guido van Rossum and Andrej Karpathy are big fans as well [1]. [1] https://youtu.be/cdiD-9MMpb0?t=8723 https://youtu.be/cdiD-9MMpb0?t=8723
    • throwaway2037 4y ago
      Please remember that Guido van Rossum is now employed by Microsoft. Yes, I am a fan of Guido van Rossum and his work, but I am always suspicious when famous people recommend their own company's products. It feels like excellent marketing/PR that is hard to resist.
      • arcturus17 4y ago
        What about Karpathy, nobody pays him in principle.
  • spookie 4y ago
    Copilot is extremely shady, and everyone should refrain from paying for it until proven otherwise. Others have already pointed out the case as a reply.
• ShamelessC 4y ago
  Your anecdote is trivially rebutted with another. I tried Github Copilot _more than twice_ (gasp), and now pay 10$/month for it. Happily.
  • jrpt 4y ago
    That's why I tried it twice. I've been hearing people say they liked it. But I haven't found it very helpful, and often distracting, so I ended up turning it off. I'll probably try again next year when the models are improved to see if I feel any differently then.
    • ShamelessC 4y ago
      Yeah, I understand. I just see a lot of people on here who seem to be deliberately looking for reasons not to like Copilot. You don't fit that stereotype, of course. So feel free to ignore the following. Developer tools have learning curves. One doesn't simply open vim/emacs for the first time with a full understanding of how to use it (or why it's a good tool to use, even). Historically, we have had _no_ problem with the steepness of this curve. But, when it comes to Copilot, there's a lot of "tried it and it output an obvious bug! how did this make it past quality assurance?? such a liability!" Just very reactionary and all-that. Anyway, sorry for the toxic response.
      • epolanski 4y ago
        Risking breaking copyright and not supporting a model of code laundering isn't exactly looking for reasons.
        • ShamelessC 4y ago
          I don’t see how that’s relevant to anything in my comment, but since you brought it up I assume you take both of those notions as a given - but they aren’t necessarily concerns of all programmers.
      • wittycardio 4y ago
• ivalm 4y ago
  I pay for github co-pilot. It seems surprisingly bad for typescript and excellent for python.
  • cowmix 4y ago
    I tried it. Ironically, it was pretty good for Powershell and so so for Python (in my case at least).
    • ivalm 4y ago
      Yeah, where it seems very good for me is just writing flask or fast api boilerplate, esp if I have a good function sig + doc string. I kind of hoped that it would propose good tsx for my components (react+mui) but instead it’s basically useless. Maybe problem is that I chose “filter out exact code reproductions.”
• deforciant 4y ago
  Paying for copilot :) at least in go it’s great to write tests and sometimes some smaller functions :) totally worth paying for it, even from your own pocket if the company wouldn’t allow expensing it
  • janoc 4y ago
    If the company wouldn't pay for it then better think twice because you could get in hot water with legal. That's not a tool one's job or even company's business is worth risking over. Copilot has a ton of still unresolved legal and compliance issues (copyright violation problems, sending proprietary code to Microsoft as you are writing it, etc.) and most larger businesses won't touch it with a 10 foot pole for that reason. There is even a class action lawsuit against Microsoft over Copilot already.
• dgacmu 4y ago
  I pay for copilot. It saves me a modest number of minutes of time per week. That's worth a small fee. And before someone jumps in: I and my other co-founder who also uses copilot (We are the only two in the company who do, I think, without checking) are the compliance team. We're both very senior and use copilot basically a line or three at a time as a smart autocomplete. It's still worth it.
  • hanselot 4y ago
    It's really just perfect for remembering obscure things and can easily be prompted to generate the boilerplate. If you surround it with your style you will see it try to use the same techniques, however if you work on large code bases it gets annoying when it starts copying the bad habits you are trying to get rid of. In those cases it's actually kind of good for bringing to your attention that the building next door is still on fire.
  • jacurtis 4y ago
    > [we] use copilot basically a line or three at a time as a smart autocomplete. I think this is the best way to think of CoPilot. GitHub is selling it like its going to write all your code for you, but in reality it is just next-generation auto-complete. That's not a bad thing. In some ways I'd argue its actually better. GitHub needs to change its marketing because even most developers seem to think that its out there to take away our jobs. Its not and can not. But it provides the smartest auto-complete you've ever seen and that can be useful, especially when wading through mundane parts of your codebase.
    • Gigachad 4y ago
      I found copilot to be less useful than autocomplete. Typical autocomplete suggests things that actually exist and work in the codebase. While Copilot would suggest things that look superficially like names that exist or ways I might have named it, but very often just wrong. I find typed languages like Rust or Typescript make VS Code super powered and provides much more value than copilot.
      • 4y ago
• vinyl7 4y ago
  > Developers are somewhat reluctant to pay for tools but I think you can get them to pay for things that are worth it. Indeed, I payed for a debugger because MSVC is pretty terrible
• eloff 4y ago
  My experience with copilot has been very different. It easily pays for itself, and getting my employer (seed stage startup) to spring for it for the entire team was an easy sell. Yeah it's pretty dumb most of the time. But I know that, and I don't use code from it without carefully checking it out and modifying it. But it's still a huge help. Just the time saved writing tests alone pays for it. And I've had a few spooky experiences where it feels like it knows the bug fix before I do. Think of it as a smarter auto-complete. The technology has a long way to go, but I completely disagree with Kite here. It's already good enough to pay for. If my company didn't pay for it, I would. I already pay for JetBrains, and it costs more than Copilot. I would give up JetBrains before I give up Copilot. My guess here is Kite positioned themselves as a free alternative to Copilot and then couldn't monetize. There very likely is more to it though.
  • jascination 4y ago
    Out of interest, how are you using it to write tests? Do you just write "make a test for functionX" or something? (Don't have much experience with it)
    • satvikpendem 4y ago
      Yes, if you show an example, or even have the test file open, it will make the other tests for you.
      • dboreham 4y ago
        I wonder if this says something about the nature of test code?
        • djbusby 4y ago
          Sure, it's been loads of boilerplate since forever.
        • throwawaysleep 4y ago
          Tests often have tons and tons of boilerplate
• luckylion 4y ago
  > Developers are somewhat reluctant to pay for tools but I think you can get them to pay for things that are worth it. I've been paying for code editors for years. Especially when you don't market to developers in general, but freelancers/contractors specifically. It might be hard to sell to salaried developers (they'll buy because it's nicer to work with good tools), but it's easy to sell tooling to anyone who makes more money when they get more done.
• glenngillen 4y ago
  I’d recommend anybody thinking about building a devtool to read Neil Davidson’s “Don’t roll the dice”. It’s a pretty old book, but Neil has also made it available for free now and the general lessons still hold true today. Some IC developers will pay for tools, it’s very hard to have that happen at a price point that supports the scale required. So feature discriminate on the things their boss needs, and charge for that. And then the next set of features for their bosses’ boss, and so on until you’re selling into the C-suite.
• joshvm 4y ago
  "Our diagnosis is that individual developers do not pay for tools." I think this is probably true. If you need a tool for your day job, your company ought to be paying for it. Some companies have slush funds for small purchases like books, but subscription costs for services would normally need to be approved. If you're a solo consultant then perhaps you'd pay for tools that make you more productive. But for personal projects the value-add would have to be pretty high to be paying another O($10-20) a month on top of other subscriptions. The big group of "hobbyist" coders are students, and they get copilot for free via Github's very generous edu package (and so does anyone with an edu email address I think). The bigger problem is that this is a very expensive project. It's better suited to a big company with money to burn and deep pockets to give it away to junior devs who will evanglise for it at their new companies (e.g. students) for nothing. See Matlab.
  • dijit 4y ago
    If you'll allow me to go on a tangent here; The sheer volume of subscription services I've signed up for as the CTO for a startup is mind-boggling. $8 here, $19 there, $49 for something important, $99 for something essential. Some tools are easily worth it, especially when you see what is charged for other (less valuable) tools. Gitlab, Confluence, Jira, Asana, 1Password, co-pilot, codepen, sentry, jetbrains, gitlab plugins for jetbrains, Visual Studio, Docker Desktop, Perforce, Slack, etc;etc;etc;etc Then there's things like Spacelift ($250!)! The most frustrating thing is that: 1) I need to justify these expenses each for what value they bring, some things are nice to have but bring so little value on paper. 2) You can't just enable tools for some people, there's huge overlap and that overlap gets greater I get that people need to be paid, but these things very quickly add up. I'm paying about 7-13% of peoples salaries already in these subscriptions, and I feel like a total dick for saying no to people or trying to consolidate these.
    • einpoklum 4y ago
      > Some tools are easily worth it, especially when you see what is charged for other (less valuable) tools. Maybe, but most of the tools you listed are not in that category IMNSHO.
• stanislavb 4y ago
  On the contrary, I find Github Copilot extremely helpful and saving me heaps of time. Yes, it's not writing the logic instead of me, but it acts like the best companion I could have in most of the cases.
• didibus 4y ago
  The tool just has to be very well integrated and easy to use. That's why copilot is seeing adoption, because Microsoft owns VSCode and has built a very simple integration of Copilot into VSCode. That said, I'm not even sure VSCode or Copilot is lucrative, if it wasn't owned by Microsoft, could they both be sustainable businesses?
• make3 4y ago
  Copilot is really great. Kite is garbage, & they have absolutely zero consumer trust from all the bullshit they did
• throwaway675309 4y ago
  When I tried kite over a year ago I was relatively unimpressed with it. Even though it ran as a plug-in to jet brains IDE it required a windows installed service and two separate executables running in the background (kite.exe, kited.exe), and that stuff continue to run after exiting my IDE which was unacceptable for me. Kite may have been the first to market but copilot blew them out of the water in terms of overall functionality.
  • password4321 4y ago
    So they wanted to run forever in the background and they'd already gotten in trouble for silently collecting telemetry? Sounds like everyone has dodged a bullet!
• ilrwbwrkhv 4y ago
  I think the real reason is that developers are maybe some of the hardest to fool customers on the planet. Since we literally build all of this our B.S. detection meter is really high. Kite thought it can go after the up and coming new developers by doing slightly shady things. However, developers also have an incredible allergy to such tactics and it forever taints your brand. So overall, developers do pay for tools, just not useless ones with shady growth tactics.
  • esperent 4y ago
    > Kite thought it can go after the up and coming new developers by doing slightly shady things I briefly tried Kite a few years ago. I didn't notice anything shady although maybe I just didn't stick around long enough. What shady tactics are you referring to?
    • valenceelectron 4y ago
      Essentially, they bought the Atom Minimap plugin and added kite-specific code/offers. This blew up a lot in the respective GitHub repo and also on HN. https://news.ycombinator.com/item?id=14857944 https://news.ycombinator.com/item?id=14857944 https://news.ycombinator.com/item?id=14836653 https://news.ycombinator.com/item?id=14836653
    • ilrwbwrkhv 4y ago
      > I fell for this. I enabled it because I was curious about trying new development tools, only to find out later it uploaded all of the source code on my computer to their service. What the hell. An old comment from another user. https://news.ycombinator.com/item?id=14837253 https://news.ycombinator.com/item?id=14837253
  • HatchedLake721 4y ago
    I’d say it’s the opposite. Developers easily fool themselves thinking they’ll save $9 p/m by building something from scratch in 3 weeks.
  • dmitriid 4y ago
• yarg 4y ago
  I paid for intellij - damned near the entire architecture team where I worked had a copy, and the company sure as shit wasn't the one paying for it. (I eventually stopped subscribing, in part because they were too slow distancing themselves from Russia, in part because of their movement away from open source with their newer tooling.) Developers will pay for software, if the value proposition is there.
  • lolinder 4y ago
    > in part because they were too slow distancing themselves from Russia I'd cut them some slack here. They had to get their team out of there first—with the way Putin is running things, they sure as hell couldn't announce they were leaving Russia until everyone who was going to follow them was out of there. On the day of the invasion they tweeted a statement condemning the attack, and within two weeks announced they were leaving Russia. https://twitter.com/jetbrains/status/1496786254494670851?lang=en https://twitter.com/jetbrains/status/1496786254494670851?lan... https://blog.jetbrains.com/blog/2022/03/11/jetbrains-statement-on-ukraine/ https://blog.jetbrains.com/blog/2022/03/11/jetbrains-stateme...
    • yarg 4y ago
      I cancelled the subscription long before the full on invasion. Two weeks was impressive, just not impressive enough to get me to renew.
• grepLeigh 4y ago
  I'd be curious to hear about services/tools developers do pay for. The diagnosis that developers do not pay for tools seems off to me. A few tools that I put on the company card when I worked at a Big Tech Co as an IC: * DataGrip (Jet brains) * Colab Pro (Google) * Postman Pro These were all small $ enough where I didn't need to justify the expense. It was just assumed that if I thought the tool was worth the $, it was. For more expensive purchasing decisions, there was a longer purchasing/approval process. But the expense would have to be 5-6 figures per year before hitting this barrier.
  • sanjayio 4y ago
    I think “company card” is the differentiating point here. I’m not sure how many IC devs have access to that. Which makes me think you don’t fall into the group that they’ve defined as IC.
    • grepLeigh 4y ago
      I started getting purchasing authority around the senior level, when I also had some amount of hiring authority. Even if you don't have a company card, there's typically a process where you can get reimbursed for expenses. If you are an IC reading this and have never tried to expense a tool you find valuable, give it a shot. You might end up surprised at how much management appreciates the ability to trade money for business value.
  • jacurtis 4y ago
    Yeah I called BS when I read that line too. I pay for plenty of dev tools. Similar list to you. * JetBrains (PyCharm professional, DataGrip, and Goland) ~$250/yr * Lucidchart (Diagramming) ~100 /yr * Paw (HTTP Client) ~$50 /yr * Docker Pro ~$60 /yr I think there's probably more, but I'm not at my work laptop to look, but those are the big ones. Those are only individual subscriptions. There's also huge costs when associated with things like Gitlab Premium ($20 - $100 /user/month), CI/CD, Code coverage tools, security scanners, etc. Companies pay A LOT for development tools. If Kite thinks that the problem why no one will pay $9/mo for their service is because developers or their company's are cheap, they need to re-assess. The reason they couldn't sell their service is because it wasn't providing enough value to justify it. But companies are paying hundreds of dollars a month per developer in most cases for various tools. The extra $9 for Kite isn't the dealbreaker if there was enough value from it.
• lijogdfljk 4y ago
  Yea.. i happily pay for several JetBrains tools and i'd love to pay for even more. I've got several problems that i don't want to spend time solving myself. Frankly as a developer i've got more problems than i can count and if it involves a GUI i tend to prefer to pay for it. I love FOSS but UX is just not often a focus. I have better experience with paid products. Assuming the licensing isn't punishing.
  • meowmeowmoo 4y ago
    What problems would you pay to have solved for you that don’t already have a solution?
    • lijogdfljk 4y ago
      Biggest one off the top of my head is in the category of developer productivity (which i pay a lot for already), and specifically it lets me use my editor of choice (Helix/Kakoune currently) while getting powerful new features. Huge hurdle, obviously, but my thought was an expanded set of LSP features from JetBrains, disconnected from their IDEs. They spend quite a bit of time and money on developing DX but it's all inaccessible to those of us who prefer different editors. I'd pay a lot for these sorts of features that go above the existing FOSS LSP, while retaining more integration to the FOSS tooling we've come to know and love (like editors or choice). There's more i'm positive, but i just woke up :)
• Myrmornis 4y ago
  With Copilot it's important to configure your editor so that it only generates completions when you ask for them. Then it can be very useful at times, especially when you're doing something you know is routine but you don't recall off-hand how to do (e.g. opening and writing to a file in append mode in a language you only use occasionally) Having it suggesting stuff every time you hit enter quickly gets annoying. https://github.com/community/community/discussions/7553#discussioncomment-1629050 https://github.com/community/community/discussions/7553#disc... https://www.reddit.com/r/vscode/comments/qromfk/is_there_a_way_to_quickly_disable_and_enable/ https://www.reddit.com/r/vscode/comments/qromfk/is_there_a_w...
• Waterluvian 4y ago
  That last line really feels like a “I guess nobody wants to enjoy the great new taste of Pepsi Glass” kind of slant. Maybe the product is poor. Maybe people didn’t believe the claim. But nobody said, “nah I don’t want my devs to be 18% faster.”
• serjester 4y ago
  Copilot had 400k paying customers within it's first month [1]. I'm not a fan of mass generalization about large cohorts of people. Will everyone use a tool? Of course not. You just need dedicated early adopters that see the value add. Without having tried it I'm assuming either their product was not good enough or their marketing department isn't strong. Developer really tend to neglect the latter. [1] https://www.ciodive.com/news/github-copilot-microsoft-software-developer/628587/ https://www.ciodive.com/news/github-copilot-microsoft-softwa...
• johnfn 4y ago
  I feel the problem with discussions about Copilot is that they consist of roughly two groups of people talking past each other. The first group believes that Copilot should be able to write code for whatever you tell it to write code for. The second group thinks Copilot is a fairly overpowered autocomplete. The first group gets annoyed all the time because Copilot fails to write most code when prompted with comments, or writes inaccurate code at best. They get upset when they see that Copilot can reproduce GPL code when prompted in a specific way. The second group most prompt Copilot by allowing it to tab a complete a line or two at a time, and they are actually super happy because Copilot is way better than any other existing autocomplete; it's basically in a class of its own. To them, the GPL issue seems a bit more abstract, because they would never use Copilot to do that anyways. I fall pretty firmly into the second camp (can you tell?). Allow me to soliloquize for a moment. Copilot is an incredibly powerful tool, probably the most powerful one I have, but, just like any tool, you need to really learn your way around it, and understand what it can and cannot do, before you start making judgments. I'm not surprised that you turned it off after using it twice. Imagine saying you stopped using React after making two components! Maybe I should write up a bit more about how I use Copilot, but in a nutshell I feel that it falls somewhere between a 2x-better autocomplete and (and this bit is even more interesting) a tool similar to google search, but more tightly integrated with the coding environment. The second bit is why it's so good. Imagine if I were to continuously google search everything I was doing while coding, while I was coding it. Sure, most of the time it'd just confirm you were doing the right thing, but... every now and then, Google might turn up a better strategy than the one I was currently trying. That's how I feel Copilot works all the time; it's continuously "google searching" for alternate approaches, and every now and then it'll be like, "aha, did you think of [this thing]" and really take me aback, because I wouldn't have even thought to Google for that particular bit of code / problem / strategy. Of course, you could continuously google search everything you did as you did it, but it would be a massive waste of time. Just imagine Copilot is doing it for you, and returning what it found. Most of the time I know what I'm doing, but every now and then, the result is remarkable.
  • loosescrews 4y ago
    I'm not a lawyer, but my understanding is that even small snippets of licensed code can be problematic. I don't know exactly what the cutoff is, but when I tried Copilot it often suggested to auto complete snippets of code that were long enough that if I was intentionally coping them from a licensed codebase, I would handle the license. I'm not talking about whole giant functions, but small functions or large chunks of a function. It is true that more commonly it suggested at most a few lines of obvious code which could really only be written the way it suggested, but a number of people in the comments on this article mentioned using Copilot to come up with test cases, so I think people are actually using it to suggest larger snippets of code.
• _alex_ 4y ago
  I pay out of pocket for a JetBrains license because it makes me a LOT more productive. I don't spend money on a lot of dev tools, but if it saves me non-trivial time, it's a no-brainer.
• quickthrower2 4y ago
  I can see Jetbrains as being the conduit for selling such a tool, because their customers are willing and are to some extent trained to pay.
• importantbrian 4y ago
  I tried Kite and didn't even keep using the free trial or whatever the free tier was at the time. I don't think the issue is that developers won't pay for tools. Companies like Jetbrains are incredibly successful selling tools to developers. The issue for Kite is that Kite wasn't worth paying for.
• jackcviers3 4y ago
  Tabnine, in emacs, with lsp mode gives you single line completions and predicts and fills out word by word in a very effective manner. It's an actual timesaver and worth the money (in emacs). The vscode experience is more problematic, but that's vscode's completions guis fault, not the tabnine server's.
• stevage 4y ago
  Wow, Copilot's must be very domain-dependent. For me, it saves a ton of time, I'd hate to have to code without it.
• serverlessmania 4y ago
  I disagree, Github copilot makes me happy, helps a lot with guessing patterns in my own project base code, I just write the good function name and he guess what I want to do.