4 ms·

I suspect Microsoft would earn more money by doing this. Their own engineers would get productivity boosts - with copilot already being familiar with data stru

I suspect Microsoft would earn more money by doing this. Their own engineers would get productivity boosts - with copilot already being familiar with data structures, code style, etc. would be a big boost to accuracy. But also, third party code would end up being more similar. Code style of the whole world would be pushed towards 'Microsoft style', which probably makes hiring easier, less training time for engineers, etc. And the downside, that is outsiders might learn tiny nuggets of info about microsoft sources, is probably irrelevant when outsiders can already decompile binaries and learn far more.

dh2022 3y ago
You are assuming Microsoft code base is superior to Linux / Git / MySql / whatever else is in github right now. That is a .... big assumption. And if Microsoft's code ends up influencing the rest of the world code that would be a .... big downside.
- eru 3y ago
  > You are assuming Microsoft code base is superior to Linux / Git / MySql / whatever else is in github right now. How do you get that impression from the comment? I don't see anything implying that.
- ryanwaggoner 3y ago
  Not for Microsoft it wouldn’t, which was their point.
- giantg2 3y ago
  Without a myriad of dumbasses like me being able to commit to Microsoft vs Github, I'd assume Microsoft's average is better than Github's.
  - blackhole 3y ago
    That is a... bold assumption to make. Not just for Microsoft but for any large corporation.
    - giantg2 3y ago
      I'm not saying bad code doesn't exist there. My thought is that the percent of bad code increases with volume (or at least higher number of producers). Tens of millions of people committing to Github should mean its more cluttered with garbage than in MS. I at least assume MS has some automated code standard or security scans. That's at least more than nothing.
      - dh2022 3y ago
        " I at least assume MS has some automated code standard or security scans." -- that is a .... big assumption.
        giantg2 3y ago
        No, it really isn't when we're dealing with an organization that is audited for SOC 1/2, DoD, and likely others.
        clankyclanker 3y ago
        Are you sure? https://arstechnica.com/security/2023/09/hack-of-a-microsoft-corporate-account-led-to-azure-breach-by-chinese-hackers/ https://arstechnica.com/security/2023/09/hack-of-a-microsoft... The Azure-State-Department breach had nearly a half dozen contributing bugs...
        giantg2 3y ago
        And how does that compare to all the bugs on Github?
        dh2022 3y ago
        My friend - Chinese secret services read Secretary of Commerce's emails because of Microsoft's security leaks: https://abcnews.go.com/Politics/commerce-secretary-gina-raimondos-emails-hacked-microsoft-cyber/story?id=101201179 https://abcnews.go.com/Politics/commerce-secretary-gina-raim... So yeah, assuming Microsoft systems are up to standard or have security reviews or whatever is a .... big assumption.
  - zik 3y ago
    I have a friend who worked at Microsoft... if his opinion is anything to go by that's very far from true.
- Karellen 3y ago
  I don't think you should be looking at the best of the Microsoft/GitHub corpora to gauge their overall quality. You probably want to be looking at the quality of the median project, which is going to be heavily influenced by the long tail of low quality projects. IMO, the long tail of non-code-reviewed, written-by-someone-in-their-first-month-of-coding, barely-even-compiles noob code[0] in Github is going to be orders of magnitude larger than the long tail of crap in Microsoft's internal repos. [0] Hey, everyone has to start somewhere. There's nothing wrong with your first "hello world" program being buggy - that's what being a beginner means. But it's probably not the sort of code you want to train an LLM on.
  - singleshot_ 3y ago
    I would think if the LLM knew it was rookie code, it could actually be pretty useful, no?
  - Wowfunhappy 3y ago
    Now I'm wondering if the copilot AI (GPT3/4?) takes number of stars/forks/etc into account during the training process.
eitally 3y ago
I don't know about MSFT, but I bet this would really help Google a ton. With a mono-repo and huge focus on readability, not to mention how many thousands of SWEs spend the majority of their time slinging protobufs around, it seems a significant fraction of day-to-day code could be largely automated.
- liliumregale 3y ago
  Google absolutely has their own internal models that do exactly this. It wouldn't surprise me if Microsoft indeed does have an internal Copilot that is trained on their data, but even on the smallest risk that they leak their code, they wouldn't share that particular model.
  - versteegen 3y ago
    What does "absolutely has" mean here? Have you actually heard anything about such internal models?
    - phyrex 3y ago
      Why wouldn’t they? Meta does, and they write openly about it
zargon 3y ago
It would be surprising to me if their internal engineers don't already have access to a model trained on internal Microsoft code.
totallywrong 3y ago
> Code style of the whole world would be pushed towards 'Microsoft style' Yes, that's exactly what the world needs, more software like Teams.
- trgdr 3y ago
  I mean, microsoft's code is probably better than the github average. There's an awful lot of horrific code out there.
chii 3y ago
> is probably irrelevant when outsiders can already decompile binaries and learn far more. most, if not all microsoft products can have their sources be available for viewing, if you are one of those vip development partners. microsoft doesn't really have any secret source (pardon the pun) of which the leaking would undo their value proposition. In fact, if microsft opened up their system a bit more, they might even gain some PR or mindshare, and have no effect on, if not increase, their bottom line.
dtagames 3y ago
This is incorrect and not how Copilot works. My company just hosted two MS engineers to explain it live to 175 of us. The style applied by Copilot comes from your surrounding code context, not from the LLM. And that base, trained on all public repos from GitHub, knows everything about data structures, etc, in the languages that were scanned. Nothing new would be gained by scanning MS's own repositories and nothing would be leaked or color the output in actual use.

dh2022 3y ago

You are assuming Microsoft code base is superior to Linux / Git / MySql / whatever else is in github right now. That is a .... big assumption. And if Microsoft's code ends up influencing the rest of the world code that would be a .... big downside.

eru 3y ago
> You are assuming Microsoft code base is superior to Linux / Git / MySql / whatever else is in github right now. How do you get that impression from the comment? I don't see anything implying that.
ryanwaggoner 3y ago
Not for Microsoft it wouldn’t, which was their point.
giantg2 3y ago
Without a myriad of dumbasses like me being able to commit to Microsoft vs Github, I'd assume Microsoft's average is better than Github's.
- blackhole 3y ago
  That is a... bold assumption to make. Not just for Microsoft but for any large corporation.
  - giantg2 3y ago
    I'm not saying bad code doesn't exist there. My thought is that the percent of bad code increases with volume (or at least higher number of producers). Tens of millions of people committing to Github should mean its more cluttered with garbage than in MS. I at least assume MS has some automated code standard or security scans. That's at least more than nothing.
    - dh2022 3y ago
      " I at least assume MS has some automated code standard or security scans." -- that is a .... big assumption.
      - giantg2 3y ago
        No, it really isn't when we're dealing with an organization that is audited for SOC 1/2, DoD, and likely others.
        clankyclanker 3y ago
        Are you sure? https://arstechnica.com/security/2023/09/hack-of-a-microsoft-corporate-account-led-to-azure-breach-by-chinese-hackers/ https://arstechnica.com/security/2023/09/hack-of-a-microsoft... The Azure-State-Department breach had nearly a half dozen contributing bugs...
        giantg2 3y ago
        And how does that compare to all the bugs on Github?
        dh2022 3y ago
        My friend - Chinese secret services read Secretary of Commerce's emails because of Microsoft's security leaks: https://abcnews.go.com/Politics/commerce-secretary-gina-raimondos-emails-hacked-microsoft-cyber/story?id=101201179 https://abcnews.go.com/Politics/commerce-secretary-gina-raim... So yeah, assuming Microsoft systems are up to standard or have security reviews or whatever is a .... big assumption.
- zik 3y ago
  I have a friend who worked at Microsoft... if his opinion is anything to go by that's very far from true.
Karellen 3y ago
I don't think you should be looking at the best of the Microsoft/GitHub corpora to gauge their overall quality. You probably want to be looking at the quality of the median project, which is going to be heavily influenced by the long tail of low quality projects. IMO, the long tail of non-code-reviewed, written-by-someone-in-their-first-month-of-coding, barely-even-compiles noob code[0] in Github is going to be orders of magnitude larger than the long tail of crap in Microsoft's internal repos. [0] Hey, everyone has to start somewhere. There's nothing wrong with your first "hello world" program being buggy - that's what being a beginner means. But it's probably not the sort of code you want to train an LLM on.
- singleshot_ 3y ago
  I would think if the LLM knew it was rookie code, it could actually be pretty useful, no?
- Wowfunhappy 3y ago
  Now I'm wondering if the copilot AI (GPT3/4?) takes number of stars/forks/etc into account during the training process.

eitally 3y ago

I don't know about MSFT, but I bet this would really help Google a ton. With a mono-repo and huge focus on readability, not to mention how many thousands of SWEs spend the majority of their time slinging protobufs around, it seems a significant fraction of day-to-day code could be largely automated.

liliumregale 3y ago
Google absolutely has their own internal models that do exactly this. It wouldn't surprise me if Microsoft indeed does have an internal Copilot that is trained on their data, but even on the smallest risk that they leak their code, they wouldn't share that particular model.
- versteegen 3y ago
  What does "absolutely has" mean here? Have you actually heard anything about such internal models?
  - phyrex 3y ago
    Why wouldn’t they? Meta does, and they write openly about it

zargon 3y ago

It would be surprising to me if their internal engineers don't already have access to a model trained on internal Microsoft code.

totallywrong 3y ago

> Code style of the whole world would be pushed towards 'Microsoft style' Yes, that's exactly what the world needs, more software like Teams.

trgdr 3y ago
I mean, microsoft's code is probably better than the github average. There's an awful lot of horrific code out there.

chii 3y ago

> is probably irrelevant when outsiders can already decompile binaries and learn far more. most, if not all microsoft products can have their sources be available for viewing, if you are one of those vip development partners. microsoft doesn't really have any secret source (pardon the pun) of which the leaking would undo their value proposition. In fact, if microsft opened up their system a bit more, they might even gain some PR or mindshare, and have no effect on, if not increase, their bottom line.

dtagames 3y ago

This is incorrect and not how Copilot works. My company just hosted two MS engineers to explain it live to 175 of us. The style applied by Copilot comes from your surrounding code context, not from the LLM. And that base, trained on all public repos from GitHub, knows everything about data structures, etc, in the languages that were scanned. Nothing new would be gained by scanning MS's own repositories and nothing would be leaked or color the output in actual use.