8 ms·

> Copilot excels at low-to-medium complexity tasks in well-tested codebases, from adding features and fixing bugs to extending tests, refactoring, and improving

> Copilot excels at low-to-medium complexity tasks in well-tested codebases, from adding features and fixing bugs to extending tests, refactoring, and improving documentation. Bounds bounds bounds bounds. The important part for humans seems to be maintaining boundaries for AI. If your well-tested codebase has the tests built thru AI, its probably not going to work. I think its somewhat telling that they can't share numbers for how they're using it internally. I want to know that Microsoft, the company famous for dog-fooding is using this day in and day out, with success. There's real stuff in there, and my brain has an insanely hard time separating the trillion dollars of hype from the usefulness.

twodave 1y ago
I feel like I saw a quote recently that said 20-30% of MS code is generated in some way. [0] In any case, I think this is the best use case for AI in programming—as a force multiplier for the developer. It’s for the best benefit of both AI and humanity for AI to avoid diminishing the creativity, agency and critical thinking skills of its human operators. AI should be task oriented, but high level decision-making and planning should always be a human task. So I think our use of AI for programming should remain heavily human-driven for the long term. Ultimately, its use should involve enriching humans’ capabilities over churning out features for profit, though there are obvious limits to that. [0] https://www.cnbc.com/2025/04/29/satya-nadella-says-as-much-as-30percent-of-microsoft-code-is-written-by-ai.html https://www.cnbc.com/2025/04/29/satya-nadella-says-as-much-a...
- tmpz22 1y ago
  How much of that is protobuf stubs and other forms of banal autogenerate code?
  - twodave 1y ago
    Updated my comment to include the link. As much as 30% specifically generated by AI.
    - shafyy 1y ago
      I would still wager that most of the 30% is some boilterplate stuff. Which is ok. But sounds less impressive with that caveat.
    - OnionBlender 1y ago
      The 2nd paragraph contradicts the title. The actual quote by Satya says, "written by software".
      - twodave 1y ago
        Sure but then he says in his next sentence he expects 50% by AI in the next year. He’s clearly using the terms interchangeably.
- greatwhitenorth 1y ago
  How much was previously generated by intellisense and other code gen tools before AI? What is the delta?
- ilaksh 1y ago
  You might want to study the history of technology and how rapidly compute efficiency has increased as well as how quickly the models are improving. In this context, assuming that humans will still be able to do high level planning anywhere near as well as an AI, say 3-5 years out, is almost ludicrous.
  - _se 1y ago
    Reality check time for you: people were saying this exact thing 3 years ago. You cannot extrapolate like that.
- DeepYogurt 1y ago
  > I feel like I saw a quote recently that said 20-30% of MS code is generated in some way. [0] Similar to google. MS now requires devs to use ai
  - spooneybarger 1y ago
    I know a lot of devs at MSFT, none of them are required to use AI.
    - diggan 1y ago
      From reading around on Hacker News and Reddit, it seems like half of commentators say what you say, and the other half says "I work at Microsoft/know someone who works at Microsoft, and our/their manager just said we have to use AI", someone mentioned being put on PIP for not "leveraging AI" as well. I guess maybe different teams have different requirements/workflows?
      - spooneybarger 1y ago
        O ya, when I was at Microsoft, everything varied widely by team and organization. I can imagine there are groups where it is true. I mostly want to push back on the idea that there's one monolithic culture that is Microsoft.
    - _heimdall 1y ago
      The GitHub org is required to for sure, with a very similar mandate to the one Shopify's CEO put out. LLM use is now part of the annual review process, its self reported if I'm not mistaken but at least at Microsoft they would have plenty of data to know how often you use the tools.
  - beefnugs 1y ago
    So demanding all employees use it... results in less than 30% compliance. That does tell me a lot
- rcarmo 1y ago
  That quote was completely misrepresented.
deleted 1y ago
[deleted]
deleted 1y ago
[deleted]
timrogers 1y ago
We've been using Copilot coding agent internally at GitHub, and more widely across Microsoft, for nearly three months. That dogfooding has been hugely valuable, with tonnes of valuable feedback (and bug bashing!) that has helped us get the agent ready to launch today. So far, the agent has been used by about 400 GitHub employees in more than 300 our our repositories, and we've merged almost 1,000 pull requests contributed by Copilot. In the repo where we're building the agent, the agent itself is actually the #5 contributor - so we really are using Copilot coding agent to build Copilot coding agent ;) (Source: I'm the product lead at GitHub for Copilot coding agent.)
- binarymax 1y ago
  So I need to ask: what is the overall goal of your project? What will you do in, say, 5 years from now?
  - timrogers 1y ago
    What I'm most excited about is allowing developers to spend more of their time working on the work they enjoy, and less of their time working on mundane, boring or annoying tasks. Most developers don't love writing tests, or updating documentation, or working on tricky dependency updates - and I really think we're heading to a world where AI can take the load of that and free me up to work on the most interesting and complex problems.
    - binarymax 1y ago
      Thanks for the response… do you see a future where engineers are just prompting all the time? Do you see a timeline in which todays programming languages are “low level” and rarely coded by hand?
    - petetnt 1y ago
      What about developers who do enjoy writing for example high quality documentation? Do you expect that the status quo will be that most of the documentation will be AI slop and AI itself will just bruteforce itself through the issues? How close are we to the point where the AI could handle "tricky dependency updates", but not being able to handle "most interesting and complex problems"? Who writes the tests that are required for the "well tested" codebases for GitHub Copilot Coding Agent to work properly? What is the job for the developer now? Writing tickets and reviewing low quality PRs? Isn't that the most boring and mundane job in the world?
ctkhn 1y ago
That's great, our leadership is heavily pushing ai-generated tests! Lol
mjr00 1y ago
From talking to colleagues at Microsoft it's a very management-driven push, not developer-driven. Friend on an Azure team had a team member who was nearly put on a PIP because they refused to install the internal AI coding assistant. Every manager has "number of developers using AI" as an OKR, but anecdotally most devs are installing the AI assistant and not using it or using it very occasionally. Allegedly it's pretty terrible at C# and PowerShell which limits its usefulness at MS.
- shepherdjerred 1y ago
  [flagged]
  - evantbyrne 1y ago
    It's just tooling. Costs nothing to wait for it to be better. It's not like you're going miss out on AGI. The cost of actually testing every slop code generator is non-trivial.
  - mjr00 1y ago
    What does this have to do with my comment? Did you mean to reply to someone else? I don't understand what this has to do with AI adoption at MS (and Google/AWS, while we're at it) being management-driven.
  - antihipocrat 1y ago
    That's exactly what senior executives who aren't coding are saying everywhere. Meanwhile, engineers are using it for code completion and as a Google search alternative. I don't see much difference here at all, the only habit to change is learning to trust an AI solution as much as a Stack Overflow answer. Though the benefit of SO is each comment is timestamped and there are alternative takes, corrections, caveats in the comments.
    - AdieuToLogic 1y ago
      > I don't see much difference here at all, the only habit to change is learning to trust an AI solution as much as a Stack Overflow answer. Though the benefit of SO is each comment is timestamped and there are alternative takes, corrections, caveats in the comments. That's a pretty big benefit, considering the feedback was by people presumably with relevant expertise/experience to contribute (in the pre-LLM before-time).
mrcsharp 1y ago
> Microsoft, the company famous for dog-fooding This was true up around 15 years ago. Hasn't been the case since.
lacoolj 1y ago
They have released numbers, but I can't say they are for this specific product or something else. They are apparently having AI generate "30%" of their code. https://techcrunch.com/2025/04/29/microsoft-ceo-says-up-to-30-of-the-companys-code-was-written-by-ai/ https://techcrunch.com/2025/04/29/microsoft-ceo-says-up-to-3...
- rcarmo 1y ago
  That article is wrong, that is not what was said.
  - _heimdall 1y ago
    What was said?
    - rcarmo 1y ago
      The quote was actually "in some of our projects". Journalists completely mis-understood it.
codebolt 1y ago
Whatever the true stats for mistakes or blunders are now, remember that this is the worst its ever going to be. And there is no clear ceiling in sight that would prevent it from quickly getting better and better, especially given the current levels of investment.
- _heimdall 1y ago
  That sounds reasonable enough, but the pace or end result is by no means guaranteed. We have invested plenty of money and time into nuclear fusion with little progress. The list of key acheivments from CERN[1] is also meager in comparison to the investment put in, especially if you consider their ultimate goal to ultimately be towards applying research to more than just theory. [1] https://home.cern/about/key-achievements https://home.cern/about/key-achievements
k__ 1y ago
"I want to know that Microsoft, the company famous for dog-fooding is using this day in and day out, with success." They just cut down their workforce, letting some of their AI people go. So, I assume there isn't that much success.
sensanaty 1y ago
> I want to know that Microsoft, the company famous for dog-fooding is using this day in and day out, with success Have they tried dogfooding their dogshit little tool called Teams in the last few years? Cause if that's what their "famed" dogfooding gets us, I'm terrified to see what lays in wait with copilot.

twodave 1y ago

I feel like I saw a quote recently that said 20-30% of MS code is generated in some way. [0] In any case, I think this is the best use case for AI in programming—as a force multiplier for the developer. It’s for the best benefit of both AI and humanity for AI to avoid diminishing the creativity, agency and critical thinking skills of its human operators. AI should be task oriented, but high level decision-making and planning should always be a human task. So I think our use of AI for programming should remain heavily human-driven for the long term. Ultimately, its use should involve enriching humans’ capabilities over churning out features for profit, though there are obvious limits to that. [0] https://www.cnbc.com/2025/04/29/satya-nadella-says-as-much-as-30percent-of-microsoft-code-is-written-by-ai.html https://www.cnbc.com/2025/04/29/satya-nadella-says-as-much-a...

tmpz22 1y ago
How much of that is protobuf stubs and other forms of banal autogenerate code?
- twodave 1y ago
  Updated my comment to include the link. As much as 30% specifically generated by AI.
  - shafyy 1y ago
    I would still wager that most of the 30% is some boilterplate stuff. Which is ok. But sounds less impressive with that caveat.
  - OnionBlender 1y ago
    The 2nd paragraph contradicts the title. The actual quote by Satya says, "written by software".
    - twodave 1y ago
      Sure but then he says in his next sentence he expects 50% by AI in the next year. He’s clearly using the terms interchangeably.
greatwhitenorth 1y ago
How much was previously generated by intellisense and other code gen tools before AI? What is the delta?
ilaksh 1y ago
You might want to study the history of technology and how rapidly compute efficiency has increased as well as how quickly the models are improving. In this context, assuming that humans will still be able to do high level planning anywhere near as well as an AI, say 3-5 years out, is almost ludicrous.
- _se 1y ago
  Reality check time for you: people were saying this exact thing 3 years ago. You cannot extrapolate like that.
DeepYogurt 1y ago
> I feel like I saw a quote recently that said 20-30% of MS code is generated in some way. [0] Similar to google. MS now requires devs to use ai
- spooneybarger 1y ago
  I know a lot of devs at MSFT, none of them are required to use AI.
  - diggan 1y ago
    From reading around on Hacker News and Reddit, it seems like half of commentators say what you say, and the other half says "I work at Microsoft/know someone who works at Microsoft, and our/their manager just said we have to use AI", someone mentioned being put on PIP for not "leveraging AI" as well. I guess maybe different teams have different requirements/workflows?
    - spooneybarger 1y ago
      O ya, when I was at Microsoft, everything varied widely by team and organization. I can imagine there are groups where it is true. I mostly want to push back on the idea that there's one monolithic culture that is Microsoft.
  - _heimdall 1y ago
    The GitHub org is required to for sure, with a very similar mandate to the one Shopify's CEO put out. LLM use is now part of the annual review process, its self reported if I'm not mistaken but at least at Microsoft they would have plenty of data to know how often you use the tools.
- beefnugs 1y ago
  So demanding all employees use it... results in less than 30% compliance. That does tell me a lot
rcarmo 1y ago
That quote was completely misrepresented.

deleted 1y ago

[deleted]

deleted 1y ago

[deleted]

timrogers 1y ago

We've been using Copilot coding agent internally at GitHub, and more widely across Microsoft, for nearly three months. That dogfooding has been hugely valuable, with tonnes of valuable feedback (and bug bashing!) that has helped us get the agent ready to launch today. So far, the agent has been used by about 400 GitHub employees in more than 300 our our repositories, and we've merged almost 1,000 pull requests contributed by Copilot. In the repo where we're building the agent, the agent itself is actually the #5 contributor - so we really are using Copilot coding agent to build Copilot coding agent ;) (Source: I'm the product lead at GitHub for Copilot coding agent.)

binarymax 1y ago
So I need to ask: what is the overall goal of your project? What will you do in, say, 5 years from now?
- timrogers 1y ago
  What I'm most excited about is allowing developers to spend more of their time working on the work they enjoy, and less of their time working on mundane, boring or annoying tasks. Most developers don't love writing tests, or updating documentation, or working on tricky dependency updates - and I really think we're heading to a world where AI can take the load of that and free me up to work on the most interesting and complex problems.
  - binarymax 1y ago
    Thanks for the response… do you see a future where engineers are just prompting all the time? Do you see a timeline in which todays programming languages are “low level” and rarely coded by hand?
  - petetnt 1y ago
    What about developers who do enjoy writing for example high quality documentation? Do you expect that the status quo will be that most of the documentation will be AI slop and AI itself will just bruteforce itself through the issues? How close are we to the point where the AI could handle "tricky dependency updates", but not being able to handle "most interesting and complex problems"? Who writes the tests that are required for the "well tested" codebases for GitHub Copilot Coding Agent to work properly? What is the job for the developer now? Writing tickets and reviewing low quality PRs? Isn't that the most boring and mundane job in the world?

ctkhn 1y ago

That's great, our leadership is heavily pushing ai-generated tests! Lol

mjr00 1y ago

From talking to colleagues at Microsoft it's a very management-driven push, not developer-driven. Friend on an Azure team had a team member who was nearly put on a PIP because they refused to install the internal AI coding assistant. Every manager has "number of developers using AI" as an OKR, but anecdotally most devs are installing the AI assistant and not using it or using it very occasionally. Allegedly it's pretty terrible at C# and PowerShell which limits its usefulness at MS.

shepherdjerred 1y ago
[flagged]
- evantbyrne 1y ago
  It's just tooling. Costs nothing to wait for it to be better. It's not like you're going miss out on AGI. The cost of actually testing every slop code generator is non-trivial.
- mjr00 1y ago
  What does this have to do with my comment? Did you mean to reply to someone else? I don't understand what this has to do with AI adoption at MS (and Google/AWS, while we're at it) being management-driven.
- antihipocrat 1y ago
  That's exactly what senior executives who aren't coding are saying everywhere. Meanwhile, engineers are using it for code completion and as a Google search alternative. I don't see much difference here at all, the only habit to change is learning to trust an AI solution as much as a Stack Overflow answer. Though the benefit of SO is each comment is timestamped and there are alternative takes, corrections, caveats in the comments.
  - AdieuToLogic 1y ago
    > I don't see much difference here at all, the only habit to change is learning to trust an AI solution as much as a Stack Overflow answer. Though the benefit of SO is each comment is timestamped and there are alternative takes, corrections, caveats in the comments. That's a pretty big benefit, considering the feedback was by people presumably with relevant expertise/experience to contribute (in the pre-LLM before-time).

mrcsharp 1y ago

> Microsoft, the company famous for dog-fooding This was true up around 15 years ago. Hasn't been the case since.

lacoolj 1y ago

They have released numbers, but I can't say they are for this specific product or something else. They are apparently having AI generate "30%" of their code. https://techcrunch.com/2025/04/29/microsoft-ceo-says-up-to-30-of-the-companys-code-was-written-by-ai/ https://techcrunch.com/2025/04/29/microsoft-ceo-says-up-to-3...

rcarmo 1y ago
That article is wrong, that is not what was said.
- _heimdall 1y ago
  What was said?
  - rcarmo 1y ago
    The quote was actually "in some of our projects". Journalists completely mis-understood it.

codebolt 1y ago

Whatever the true stats for mistakes or blunders are now, remember that this is the worst its ever going to be. And there is no clear ceiling in sight that would prevent it from quickly getting better and better, especially given the current levels of investment.

_heimdall 1y ago
That sounds reasonable enough, but the pace or end result is by no means guaranteed. We have invested plenty of money and time into nuclear fusion with little progress. The list of key acheivments from CERN[1] is also meager in comparison to the investment put in, especially if you consider their ultimate goal to ultimately be towards applying research to more than just theory. [1] https://home.cern/about/key-achievements https://home.cern/about/key-achievements

k__ 1y ago

"I want to know that Microsoft, the company famous for dog-fooding is using this day in and day out, with success." They just cut down their workforce, letting some of their AI people go. So, I assume there isn't that much success.

sensanaty 1y ago

> I want to know that Microsoft, the company famous for dog-fooding is using this day in and day out, with success Have they tried dogfooding their dogshit little tool called Teams in the last few years? Cause if that's what their "famed" dogfooding gets us, I'm terrified to see what lays in wait with copilot.