4 ms·

As someone else mentioned the file isnt actually accessed by copilot, rather copilot is reading the pre-indexed contents of the file in a search engine... Real

As someone else mentioned the file isnt actually accessed by copilot, rather copilot is reading the pre-indexed contents of the file in a search engine... Really Microsoft should be auditing the search that copilot executes, its actually a bit misleading to be auditing the file as accessed when copilot has only read the indexed content of the file, I don't say I've visited a website when I've found a result of it in Google

deleted 1y ago
[deleted]
internetter 1y ago
> I don't say I've visited a website when I've found a result of it in Google I mean, it depends on how large the index window is, because if google returned the entire webpage content without leaving (amp moment), you did visit the website. fine line.
- bw86 1y ago
  The challenge then is to differentiate between "I wanted to access the secret website/document" and "Google/Copilot gave me the secret website/document, but it was not my intention to access that".
  - swiftcoder 1y ago
    Access is access. Regardless of whether you intended to view the document, you are now aware of its content in either case, and an audit entry must be logged.
    - ceridwyn 1y ago
      Strongly agree. Consider the case of a healthcare application where, during the course of business, staff may perform searches for patients by name. When "Ada Lovelace" appears even briefly in the search results of a "search-as-you-type" for some "Adam _lastname", has their privacy has been compromised? I think so, and the audit log should reflect that. I'm a fan of FHIR (a healthcare api standard, but far from widely adopted), and they have a secondary set of definitions for Audit log patterns (BALP) that recommends this kind of behaviour. https://profiles.ihe.net/ITI/BALP/StructureDefinition-IHE.BasicAudit.PatientQuery.html https://profiles.ihe.net/ITI/BALP/StructureDefinition-IHE.Ba... "[Given a query for patients,] When multiple patient results are returned, one AuditEvent is created for every Patient identified in the resulting search set. Note this is true when the search set bundle includes any number of resources that collectively reference multiple Patients."
eleveriven 1y ago
That makes sense on a technical level, but from a security and compliance perspective, it still doesn't really hold up
- whizzter 1y ago
  Usage of Ai's almost by definition need everything indexed at all times to be useful, letting one rummage through your stuff without 100% ownership is just madness to begin with and avoiding deep indexing would make the shit mostly useless unless regular permission systems were put in (and then we're kinda back at were we were without AI's).
Faark 1y ago
What's the solution then? Chain 2 AIs, the first one is fine tuned on / has RAG access to your content telling a second that actually produces content what files are relevant (and logged)? Or just a system prompt "log where all the info comes from"...
- chrisweekly 1y ago
  Someone please confirm my idea (or remedy my ignorance) about this rule of thumb: Don't train a model on sensitive info, if there will ever be a need for authZ more granular than implied by access to that model. IOW, given a user's ability to interact w/ a model, assume that everything it was trained on is visible to that user.
oeitho 1y ago
> Really Microsoft should be auditing the search that copilot executes, its actually a bit misleading to be auditing the file as accessed when copilot has only read the indexed content of the file, I don't say I've visited a website when I've found a result of it in Google Not my domain of expertise, but couldn't you at some point argue that the indexed content itself is an auditable file? It's not literally a file necessarily, but if they contain enough information that they can be considered sensitive, then where is the significant difference?
- jacquesm 1y ago
  Not only could you do that, you should do that.
eschneider 1y ago
Oh, so there's a complete copy (or something that can be reassembled into a copy) completely OUTSIDE of audit controls. That's so much worse. :0
- Drakim 1y ago
  It's roughly the same problem as letting a search engine build indexes (with previews!) of sites without authentication. It's kinda crazy that things were allowed to go this far with such a fundamental flaw.
  - falcor84 1y ago
    I'm unclear on what the "flaw" is - isn't this precisely the "feature" that search engines provide to both sides and that site owners put a ton of SEO effort into optimizing?
    - Drakim 1y ago
      If you have public documents, you can obviously let a public search engine index them and show previews. All is good. If you have private documents, you can't let a public search engine index and show previews of those private documents. Even if you add an authentication wall for normal users if they try to open the document directly. They could still see part of the document in google's preview. My explanation sounds silly because surely nobody is that dumb, but this is exactly what they have done. They gave access to ALL documents, both public and private, to an AI, and then got surprised when the AI leaked some private document details. They thought they were safe because users would be faced with an authentication wall if they tried to open the document directly. But that doesn't help if copilot simply tells you all the secret in it's own words.
      - andrewaylett 1y ago
        You say that, but it happens — "Experts Exchange", for example, certainly used to try to hide the answers from users who hadn't paid while encouraging search engines to index them.

deleted 1y ago

[deleted]

internetter 1y ago

> I don't say I've visited a website when I've found a result of it in Google I mean, it depends on how large the index window is, because if google returned the entire webpage content without leaving (amp moment), you did visit the website. fine line.

bw86 1y ago
The challenge then is to differentiate between "I wanted to access the secret website/document" and "Google/Copilot gave me the secret website/document, but it was not my intention to access that".
- swiftcoder 1y ago
  Access is access. Regardless of whether you intended to view the document, you are now aware of its content in either case, and an audit entry must be logged.
  - ceridwyn 1y ago
    Strongly agree. Consider the case of a healthcare application where, during the course of business, staff may perform searches for patients by name. When "Ada Lovelace" appears even briefly in the search results of a "search-as-you-type" for some "Adam _lastname", has their privacy has been compromised? I think so, and the audit log should reflect that. I'm a fan of FHIR (a healthcare api standard, but far from widely adopted), and they have a secondary set of definitions for Audit log patterns (BALP) that recommends this kind of behaviour. https://profiles.ihe.net/ITI/BALP/StructureDefinition-IHE.BasicAudit.PatientQuery.html https://profiles.ihe.net/ITI/BALP/StructureDefinition-IHE.Ba... "[Given a query for patients,] When multiple patient results are returned, one AuditEvent is created for every Patient identified in the resulting search set. Note this is true when the search set bundle includes any number of resources that collectively reference multiple Patients."

eleveriven 1y ago

That makes sense on a technical level, but from a security and compliance perspective, it still doesn't really hold up

whizzter 1y ago
Usage of Ai's almost by definition need everything indexed at all times to be useful, letting one rummage through your stuff without 100% ownership is just madness to begin with and avoiding deep indexing would make the shit mostly useless unless regular permission systems were put in (and then we're kinda back at were we were without AI's).

Faark 1y ago

What's the solution then? Chain 2 AIs, the first one is fine tuned on / has RAG access to your content telling a second that actually produces content what files are relevant (and logged)? Or just a system prompt "log where all the info comes from"...

chrisweekly 1y ago
Someone please confirm my idea (or remedy my ignorance) about this rule of thumb: Don't train a model on sensitive info, if there will ever be a need for authZ more granular than implied by access to that model. IOW, given a user's ability to interact w/ a model, assume that everything it was trained on is visible to that user.

oeitho 1y ago

> Really Microsoft should be auditing the search that copilot executes, its actually a bit misleading to be auditing the file as accessed when copilot has only read the indexed content of the file, I don't say I've visited a website when I've found a result of it in Google Not my domain of expertise, but couldn't you at some point argue that the indexed content itself is an auditable file? It's not literally a file necessarily, but if they contain enough information that they can be considered sensitive, then where is the significant difference?

jacquesm 1y ago
Not only could you do that, you should do that.

eschneider 1y ago

Oh, so there's a complete copy (or something that can be reassembled into a copy) completely OUTSIDE of audit controls. That's so much worse. :0

Drakim 1y ago
It's roughly the same problem as letting a search engine build indexes (with previews!) of sites without authentication. It's kinda crazy that things were allowed to go this far with such a fundamental flaw.
- falcor84 1y ago
  I'm unclear on what the "flaw" is - isn't this precisely the "feature" that search engines provide to both sides and that site owners put a ton of SEO effort into optimizing?
  - Drakim 1y ago
    If you have public documents, you can obviously let a public search engine index them and show previews. All is good. If you have private documents, you can't let a public search engine index and show previews of those private documents. Even if you add an authentication wall for normal users if they try to open the document directly. They could still see part of the document in google's preview. My explanation sounds silly because surely nobody is that dumb, but this is exactly what they have done. They gave access to ALL documents, both public and private, to an AI, and then got surprised when the AI leaked some private document details. They thought they were safe because users would be faced with an authentication wall if they tried to open the document directly. But that doesn't help if copilot simply tells you all the secret in it's own words.
    - andrewaylett 1y ago
      You say that, but it happens — "Experts Exchange", for example, certainly used to try to hide the answers from users who hadn't paid while encouraging search engines to index them.