5 ms·

The data governance concerns are not that Microsoft has access to the data (that is no doubt covered by contract clauses). The concerns are that most corporate

The data governance concerns are not that Microsoft has access to the data (that is no doubt covered by contract clauses). The concerns are that most corporate implementations of network roles and permissions are not up to date or accurate, so CoPilot will show data to an employee that they should not be allowed to see. Salary info is an example. Basically, CoPilot is following “the rules” (technical settings) but corporate IT teams have not kept the technical rules up to date with the business rules. So they need to pause CoPilot until they get their own rules straight. Edit to add: if your employer has CoPilot turned on, maybe try asking for sensitive stuff and see what you get. ;-)

gxd 2y ago
I don't think this is the core of the concern, other aspects are much worse IMHO. For instance, the article says "Microsoft positions its Copilot tool as a way to make users more creative and productive by capturing all the human labor latent in the data used to train its AI models and reselling it.". The Copilot data could make it a lot easier to steal sensitive business procedures and intellectual property - the data would allow any third parties to fully inspect the company procedures and sensitive data in a scale that we've never seen. It would be next to impossible to manage, categorize and protect this data. It's an intellectual property nightmare. A good version of the technology, which we don't have yet, would allow competitors to create a copy of every employee (in the business sense) and perhaps much more efficiently compete with that company.
- simonw 2y ago
  Are you talking here about the idea that Microsoft are training their models on private customer data in a way that could later expose details of it to people outside the company? That’s not happening. Microsoft Copilot doesn’t train on data it has access to. https://learn.microsoft.com/en-us/power-platform/faqs-copilot-data-security-privacy https://learn.microsoft.com/en-us/power-platform/faqs-copilo...
  - blibble 2y ago
    Microsoft of course having an excellent reputation for respecting the law and their customers whenever there's significant money to be made by doing the exact opposite
    - muglug 2y ago
      There’s significant money to be lost (lawsuits, massive reputations damage) if it was discovered they weee deliberately training on data they had guaranteed that they were not training on.
      - blibble 2y ago
        when has the threat of lawsuits or massive reputational damage ever previously stopped Microsoft?
        bongodongobob 2y ago
        You're confusing corporate enterprise plans with home users. Completely different ballgames.
    - simonw 2y ago
      Your company’s data isn’t nearly as valuable for training models as you might think. Certainly not as valuable as the revenue you can make from companies that would instantly cancel their Copliot 365 subscriptions if they heard any hint of data being used for training without permission. Convincing people that you don’t train on their data remains one of the hardest problems: https://simonwillison.net/2023/Dec/14/ai-trust-crisis/ https://simonwillison.net/2023/Dec/14/ai-trust-crisis/
      - muglug 2y ago
        This exactly. If Microsoft had created its own fine-tuned-MSFT-data LLM and seen vastly better results on internal tasks, then they’d be publishing papers about it, and also packaging that up & selling to customers.
Ylpertnodi 2y ago
[flagged]
toomuchtodo 2y ago
Recommend the Zenity Blackhat materials for a deep dive on the risk. https://labs.zenity.io/p/links-materials-living-off-microsoft-copilot https://labs.zenity.io/p/links-materials-living-off-microsof... https://www.youtube.com/playlist?list=PLM_RIPYi59BN6BeHyJQ_9fJ4WVsaKPMfv https://www.youtube.com/playlist?list=PLM_RIPYi59BN6BeHyJQ_9...

gxd 2y ago

I don't think this is the core of the concern, other aspects are much worse IMHO. For instance, the article says "Microsoft positions its Copilot tool as a way to make users more creative and productive by capturing all the human labor latent in the data used to train its AI models and reselling it.". The Copilot data could make it a lot easier to steal sensitive business procedures and intellectual property - the data would allow any third parties to fully inspect the company procedures and sensitive data in a scale that we've never seen. It would be next to impossible to manage, categorize and protect this data. It's an intellectual property nightmare. A good version of the technology, which we don't have yet, would allow competitors to create a copy of every employee (in the business sense) and perhaps much more efficiently compete with that company.

simonw 2y ago
Are you talking here about the idea that Microsoft are training their models on private customer data in a way that could later expose details of it to people outside the company? That’s not happening. Microsoft Copilot doesn’t train on data it has access to. https://learn.microsoft.com/en-us/power-platform/faqs-copilot-data-security-privacy https://learn.microsoft.com/en-us/power-platform/faqs-copilo...
- blibble 2y ago
  Microsoft of course having an excellent reputation for respecting the law and their customers whenever there's significant money to be made by doing the exact opposite
  - muglug 2y ago
    There’s significant money to be lost (lawsuits, massive reputations damage) if it was discovered they weee deliberately training on data they had guaranteed that they were not training on.
    - blibble 2y ago
      when has the threat of lawsuits or massive reputational damage ever previously stopped Microsoft?
      - bongodongobob 2y ago
        You're confusing corporate enterprise plans with home users. Completely different ballgames.
  - simonw 2y ago
    Your company’s data isn’t nearly as valuable for training models as you might think. Certainly not as valuable as the revenue you can make from companies that would instantly cancel their Copliot 365 subscriptions if they heard any hint of data being used for training without permission. Convincing people that you don’t train on their data remains one of the hardest problems: https://simonwillison.net/2023/Dec/14/ai-trust-crisis/ https://simonwillison.net/2023/Dec/14/ai-trust-crisis/
    - muglug 2y ago
      This exactly. If Microsoft had created its own fine-tuned-MSFT-data LLM and seen vastly better results on internal tasks, then they’d be publishing papers about it, and also packaging that up & selling to customers.

Ylpertnodi 2y ago

[flagged]

toomuchtodo 2y ago

Recommend the Zenity Blackhat materials for a deep dive on the risk. https://labs.zenity.io/p/links-materials-living-off-microsoft-copilot https://labs.zenity.io/p/links-materials-living-off-microsof... https://www.youtube.com/playlist?list=PLM_RIPYi59BN6BeHyJQ_9fJ4WVsaKPMfv https://www.youtube.com/playlist?list=PLM_RIPYi59BN6BeHyJQ_9...