5 ms·

Thus, the price of Github Copilot is going to go up, 2x to 5x. For all of the criticism of Github Copilot, for a lot of developers (but not all), the value Git

Thus, the price of Github Copilot is going to go up, 2x to 5x. For all of the criticism of Github Copilot, for a lot of developers (but not all), the value Github Copilot is incredibly high, much more than $20/month. The current rock bottom pricing is low compared to the value provided for those users. As such there is a big opportunity to multiply the price here being charged. Probably an increase between 3x to 5x. The excuse is because they are losing money, but the underlying reason is that the value it is providing is so high in terms of developer productivity.

insanitybit 3y ago
I'd probably pay ~2-3x, idk about 5x. I'd consider it though. Hopefully they can cut the costs and meet in the middle.
- bhouston 3y ago
  At some point, the open source models you can run locally are going to start to get competitive. I suspect the cost problem came from upgrading to ChatGPT 4 over ChatGPT 3 - right there their costs more than 5x.
  - refulgentis 3y ago
    Did they? I'm surprised to hear that. Hard to tell for autocomplete of course. For conversational answers, inference speed is _far_ higher than I'm used to from GPT-4, and lower quality. As in, it matches GPT 3.5/ChatGPT.
    - nithril 3y ago
      Not yet, it is still using GPT 3.5. The access to GitHub Copilot X (GPT-4) is in closed beta [1]. [1] https://github.com/features/preview/copilot-x https://github.com/features/preview/copilot-x
    - laurent_du 3y ago
      Quality is definitely way below GPT4.
- DoesntMatter22 3y ago
  100 bucks a month I'll gladly pay. It stops me from having to worry about so many annoying coding tasks. I'd maybe consider 2 or 300 a month
baq 3y ago
I’m ok to pay for usage privately and my company is ok to pay $100/mo for unlimited (probably, haven’t asked)
- bosie 3y ago
  100 per developer?
  - baq 3y ago
    Obviously. At $1200/yr it's pocket change.
    - bosie 3y ago
      maybe. but at 300 devs you are talking 360k a year for example. but then i don't live in a country where devs make 300k a year ;)
      - baq 3y ago
        In California it’s like 1% payroll at most for much more than 1% gain in efficiency. Very easy decision.
dist-epoch 3y ago
Mark my words, developers will soon pay $5k per month for access to GPT-7 Copilot.
- AnimalMuppet 3y ago
  Marked. Mark my reply: No, they won't. For score keeping, let's define "soon". Does "by the end of 2025" sound reasonable?
antoineMoPa 3y ago
I feel like part of the productivity is just moved to reviewers. Every once in a while, I have to review large PRs and find out which comment is AI-generated nonsense. LLMs are really good at fooling us into thinking that code comments make sense. That's why I prefer if devs don't use copilot, but instead cherry pick good output from ChatGPT.
- _cenw 3y ago
  This is why I'm concerned about AI. It enables people who don't value their colleagues time to value it even less. Be that code or contributing to a discussion with unedited and false LLM output "as a starting point".
- jacurtis 3y ago
  Earlier this year, we had this one developer who has worked with us for a while (8 months or so). He was never a top developer, but he was average. Anyway, I have to review all of his code, and I've been doing this the whole time he worked for us. But, starting around the beginning of the year, I started to notice problem after problem coming through his merge requests (MRs). At first I gave the guy the benefit of the doubt, like I said this was unusual for him. He always needed moderate coaching, but the mistake ratio was way higher than usual. Anyway, I did some basic coaching and whatever. I talked to him about the increase in errors and he said he was stressed with moving to a new house at the time. It made sense, so I decided not to even record the first performance review with HR. But the problems kept coming and getting worse. It was getting to the point that he had so many problems, I wouldn't even address them all. I would only kick back the top dozen or so. I was noticing that his code had entirely different styling and voice across methods like it wasn't written by him. I also started to see weird lines that were trying to catch unusual edge cases that would never even apply in our scenario. There was one time where there was this insanely complex filter that fed into an insane regex that I couldn't decipher. I asked him about what it is even trying to do and he had no clue, he couldn't even explain the purpose of the line, let alone how it worked. I pushed him on where it came from and he said "StackOverflow". But I reversed searched it and couldn't find it. We ended up doing performance reviews almost weekly for a while, and now I was formally writing them up. I told him that he was more of a burden on the company at this point. He was offering a negative value. I was getting stressed out because I was spending so long fixing his code and coaching him, that I was working longer hours because of him. Furthermore, I could have fired him and not even replaced him, just taken over his job myself and it would have been less work than this back and forth of trying to repair his work. Eventually I did fire him. After he left and I talked to the team about him being let go, one of his friends told me that this guy had been using ChatGPT and Copilot for all of his work. It was a secret because our company doesn't allow AI tools in our codebase right now because of compliance reasons. Which is why he would rather say his mistakes were "copypasta from StackOverflow" as he always put it, than admit it was copilot. That's not to say that CoPilot is bad. But its not ready to replace developers just yet. Even junior developers still need to wrangle and check the work coming out of the AI. And yes, people do notice. Even after this employee polishing the work coming out of the AI model, it still wouldn't work in our codebase. And to the original point. Yes, these AI tools really are just passing increased burden onto the experienced developers. The juniors are using these tools as crutches to speed things up so they can watch more YouTube videos. But the code gatekeepers who have to defend the application are the ones with increased burden and workloads of fixing the mistakes the AI is generating. I've been saying for a long time that AI will simply separate the good from the bad in software development. For the past 15 years you could make $150k or more a year as a developer that is only capable of producing sub-par code. But those days are gone. The sub-par code is being produced by AI now. So you need to be at least mediocre now.
sureglymop 3y ago
I don't think so. It helps increase productivity but it's just too inaccurate and generates too much stupid output. Not that I could do better but it's just not good enough yet.
- deleted 3y ago
  [deleted]
marcosdumay 3y ago
> for a lot of developers (but not all), the value Github Copilot is incredibly high, much more than $20/month. I really doubt this. IMO, the jury is still out deciding if the value is above zero for enough people to matter.
- simonw 3y ago
  From that article: 1.5m users and up to 50% of their code is now being produced by Copilot. Sounds to me like the verdict on its value is in already.
  - woolion 3y ago
    They don't give any source or anything; the way it's worded sounds ambiguous enough that I suspect it's not that 50% of their code is generated by Copilot. Is there something a bit more convincing elsewhere?
  - jamala1 3y ago
    Having used Copilot for a year now I very much doubt this figure. In my experience it only works well in boilerplate kind of situations where most code is copy/paste work anyhow. As soon as the code gets a little complicated it stops working well. It has also gotten quite slow for me lately. So I doubt it increases my work efficiency by more than 5%, but I do like it for reducing strain on the hands. For that I find the price appropriate.
    - ptero 3y ago
      > In my experience it only works well in boilerplate kind of situations where most code is copy/paste work anyhow. As a data point, this matches my personal observations. But reducing the time spent on that bolierplate plus not needing to search where the "copy" portion comes from, may justify $30/month (and probably much more than that). My 2c.
  - smoe 3y ago
    The quote from the article says "more than 1.5 million people have used it and it is helping build nearly half of Copilot users’ code" Not a native speaker, but to me this sounds much more ambiguous than up to 50% of code is produced by Copilot. Also, how different from previous solutions is this actually? I use autocomplete and code snippets extensively. Never measured it , but I wouldn't be surprised if my IDE had generated more source code than I myself typed over the last 10 or so years.
Klaus23 3y ago
I doubt the price will go up that much, if at all. Maybe there will be more expensive tiers, but $20 will probably stay. Between better quantization, pruning, smaller models through better training/distilling/architecture, hardware price drops and purpose-built hardware, it will probably be much cheaper to run such a model in the future.
tristanb 3y ago
Im not getting $20 of value as is.
thiht 3y ago
I happily pay it for $10 a month, but I won’t pay $20. My company could pay it for me though.

insanitybit 3y ago

I'd probably pay ~2-3x, idk about 5x. I'd consider it though. Hopefully they can cut the costs and meet in the middle.

bhouston 3y ago
At some point, the open source models you can run locally are going to start to get competitive. I suspect the cost problem came from upgrading to ChatGPT 4 over ChatGPT 3 - right there their costs more than 5x.
- refulgentis 3y ago
  Did they? I'm surprised to hear that. Hard to tell for autocomplete of course. For conversational answers, inference speed is _far_ higher than I'm used to from GPT-4, and lower quality. As in, it matches GPT 3.5/ChatGPT.
  - nithril 3y ago
    Not yet, it is still using GPT 3.5. The access to GitHub Copilot X (GPT-4) is in closed beta [1]. [1] https://github.com/features/preview/copilot-x https://github.com/features/preview/copilot-x
  - laurent_du 3y ago
    Quality is definitely way below GPT4.
DoesntMatter22 3y ago
100 bucks a month I'll gladly pay. It stops me from having to worry about so many annoying coding tasks. I'd maybe consider 2 or 300 a month

baq 3y ago

I’m ok to pay for usage privately and my company is ok to pay $100/mo for unlimited (probably, haven’t asked)

bosie 3y ago
100 per developer?
- baq 3y ago
  Obviously. At $1200/yr it's pocket change.
  - bosie 3y ago
    maybe. but at 300 devs you are talking 360k a year for example. but then i don't live in a country where devs make 300k a year ;)
    - baq 3y ago
      In California it’s like 1% payroll at most for much more than 1% gain in efficiency. Very easy decision.

dist-epoch 3y ago

Mark my words, developers will soon pay $5k per month for access to GPT-7 Copilot.

AnimalMuppet 3y ago
Marked. Mark my reply: No, they won't. For score keeping, let's define "soon". Does "by the end of 2025" sound reasonable?

antoineMoPa 3y ago

I feel like part of the productivity is just moved to reviewers. Every once in a while, I have to review large PRs and find out which comment is AI-generated nonsense. LLMs are really good at fooling us into thinking that code comments make sense. That's why I prefer if devs don't use copilot, but instead cherry pick good output from ChatGPT.

_cenw 3y ago
This is why I'm concerned about AI. It enables people who don't value their colleagues time to value it even less. Be that code or contributing to a discussion with unedited and false LLM output "as a starting point".
jacurtis 3y ago
Earlier this year, we had this one developer who has worked with us for a while (8 months or so). He was never a top developer, but he was average. Anyway, I have to review all of his code, and I've been doing this the whole time he worked for us. But, starting around the beginning of the year, I started to notice problem after problem coming through his merge requests (MRs). At first I gave the guy the benefit of the doubt, like I said this was unusual for him. He always needed moderate coaching, but the mistake ratio was way higher than usual. Anyway, I did some basic coaching and whatever. I talked to him about the increase in errors and he said he was stressed with moving to a new house at the time. It made sense, so I decided not to even record the first performance review with HR. But the problems kept coming and getting worse. It was getting to the point that he had so many problems, I wouldn't even address them all. I would only kick back the top dozen or so. I was noticing that his code had entirely different styling and voice across methods like it wasn't written by him. I also started to see weird lines that were trying to catch unusual edge cases that would never even apply in our scenario. There was one time where there was this insanely complex filter that fed into an insane regex that I couldn't decipher. I asked him about what it is even trying to do and he had no clue, he couldn't even explain the purpose of the line, let alone how it worked. I pushed him on where it came from and he said "StackOverflow". But I reversed searched it and couldn't find it. We ended up doing performance reviews almost weekly for a while, and now I was formally writing them up. I told him that he was more of a burden on the company at this point. He was offering a negative value. I was getting stressed out because I was spending so long fixing his code and coaching him, that I was working longer hours because of him. Furthermore, I could have fired him and not even replaced him, just taken over his job myself and it would have been less work than this back and forth of trying to repair his work. Eventually I did fire him. After he left and I talked to the team about him being let go, one of his friends told me that this guy had been using ChatGPT and Copilot for all of his work. It was a secret because our company doesn't allow AI tools in our codebase right now because of compliance reasons. Which is why he would rather say his mistakes were "copypasta from StackOverflow" as he always put it, than admit it was copilot. That's not to say that CoPilot is bad. But its not ready to replace developers just yet. Even junior developers still need to wrangle and check the work coming out of the AI. And yes, people do notice. Even after this employee polishing the work coming out of the AI model, it still wouldn't work in our codebase. And to the original point. Yes, these AI tools really are just passing increased burden onto the experienced developers. The juniors are using these tools as crutches to speed things up so they can watch more YouTube videos. But the code gatekeepers who have to defend the application are the ones with increased burden and workloads of fixing the mistakes the AI is generating. I've been saying for a long time that AI will simply separate the good from the bad in software development. For the past 15 years you could make $150k or more a year as a developer that is only capable of producing sub-par code. But those days are gone. The sub-par code is being produced by AI now. So you need to be at least mediocre now.

sureglymop 3y ago

I don't think so. It helps increase productivity but it's just too inaccurate and generates too much stupid output. Not that I could do better but it's just not good enough yet.

deleted 3y ago
[deleted]

marcosdumay 3y ago

> for a lot of developers (but not all), the value Github Copilot is incredibly high, much more than $20/month. I really doubt this. IMO, the jury is still out deciding if the value is above zero for enough people to matter.

simonw 3y ago
From that article: 1.5m users and up to 50% of their code is now being produced by Copilot. Sounds to me like the verdict on its value is in already.
- woolion 3y ago
  They don't give any source or anything; the way it's worded sounds ambiguous enough that I suspect it's not that 50% of their code is generated by Copilot. Is there something a bit more convincing elsewhere?
- jamala1 3y ago
  Having used Copilot for a year now I very much doubt this figure. In my experience it only works well in boilerplate kind of situations where most code is copy/paste work anyhow. As soon as the code gets a little complicated it stops working well. It has also gotten quite slow for me lately. So I doubt it increases my work efficiency by more than 5%, but I do like it for reducing strain on the hands. For that I find the price appropriate.
  - ptero 3y ago
    > In my experience it only works well in boilerplate kind of situations where most code is copy/paste work anyhow. As a data point, this matches my personal observations. But reducing the time spent on that bolierplate plus not needing to search where the "copy" portion comes from, may justify $30/month (and probably much more than that). My 2c.
- smoe 3y ago
  The quote from the article says "more than 1.5 million people have used it and it is helping build nearly half of Copilot users’ code" Not a native speaker, but to me this sounds much more ambiguous than up to 50% of code is produced by Copilot. Also, how different from previous solutions is this actually? I use autocomplete and code snippets extensively. Never measured it , but I wouldn't be surprised if my IDE had generated more source code than I myself typed over the last 10 or so years.

Klaus23 3y ago

I doubt the price will go up that much, if at all. Maybe there will be more expensive tiers, but $20 will probably stay. Between better quantization, pruning, smaller models through better training/distilling/architecture, hardware price drops and purpose-built hardware, it will probably be much cheaper to run such a model in the future.

tristanb 3y ago

Im not getting $20 of value as is.

thiht 3y ago

I happily pay it for $10 a month, but I won’t pay $20. My company could pay it for me though.