23 ms·
Copilot broke audit logs, but Microsoft won't tell customers
- jayofdoom 1y agoGenerally speaking, anyone can file a CVE. Go file one yourself and force their response. This blogpost puts forth reasonably compelling evidence.
- db48x 1y agoFun, but it doesn’t deserve a CVE. CVEs are for vulnerabilities that are common across multiple products from multiple sources. Think of a vulnerability in a shared library that is used in most Linux distributions, or is statically linked into multiple programs. Copilot doesn’t meet that criteria. Honestly, the worst thing about this story is that apparently the Copilot LLM is given the instructions to create audit log entries. That’s the worst design I could imagine! When they use an API to access a file or a url then the API should create the audit log. This is just engineering 101.
- gpm 1y agoHuh, there are CVEs for windows components all the time, random example: https://msrc.microsoft.com/update-guide/vulnerability/CVE-2025-24993 https://msrc.microsoft.com/update-guide/vulnerability/CVE-20... Including for end user applications, not libraries, another random example: https://msrc.microsoft.com/update-guide/vulnerability/CVE-2025-53784 https://msrc.microsoft.com/update-guide/vulnerability/CVE-20...
- ecb_penguin 1y ago> CVEs are for vulnerabilities that are common across multiple products from multiple sources. This is absolutely not true. I have no idea where you came up with this. > Honestly, the worst thing about this story is that apparently the Copilot LLM is given the instructions to create audit log entries. That's not at all what the article says. > That’s the worst design I could imagine! Ok, well, that's not how they designed it. > This is just engineering 101. Where is the class for reading 101?
- ThrowMeAway1618 1y ago>> CVEs are for vulnerabilities that are common across multiple products from multiple sources. >This is absolutely not true. I have no idea where you came up with this. Perhaps they asked Copilot?
- immibis 1y agoMore accurately, CVEs are for vulnerabilities that may be present on many systems. Then, the CVE number is a reference point that helps you when discussing the vulnerability, like asking whether it's present on a particular system, or what percentage of systems are patched. This vulnerability was only present on one system, so it doesn't need a CVE number. It could have a Microsoft-assigned bug number, but it doesn't need a CVE.
- fulafel 1y agoThis may be a stated reason but it's questionable logic. There are of course many cases where people need to reference and discuss this vulnerability and its impact.
- immibis 1y agoThere are many cases where people need to reference and discuss the weather, but the weather doesn't need a CVE number. If you could hypothetically put it in a known vulnerability scanner then it should have a CVE. Otherwise no.
- fulafel 1y agoIt's for communication. "The Common Vulnerabilities and Exposures (CVE) Program’s primary purpose is to uniquely identify vulnerabilities and to associate specific versions of code bases (e.g., software and shared libraries) to those vulnerabilities. The use of CVEs ensures that two or more parties can confidently refer to a CVE identifier (ID) when discussing or sharing information about a unique vulnerability" (from https://nvd.nist.gov/vuln https://nvd.nist.gov/vuln)
- HelloImSteven 1y agoBut this isn't a problem on one system, it's potentially a problem in any system with Copilot enabled. It's akin to a vulnerability in a software library (which often means a separate CVE for every affected product, not just one for the library). CVEs also limited to issues impacting multiple systems; even if a vulnerability only affects one product, ideally a CVE should get made. The 'common' aspect is the shared reporting standard. See my other comment on this thread for more on that, or Redhat's explanation here: https://www.redhat.com/en/topics/security/what-is-cve https://www.redhat.com/en/topics/security/what-is-cve
- HelloImSteven 1y agoCVEs aren’t just for common dependencies. The “Common” part of the name is about having standardized reporting that over time helps reveal common issues occurring across multiple CVEs. Individually they’re just a way to catalog known vulnerabilities and indicate their severity to anyone impacted, whether that’s a hundred people or billions. There are high severity CVEs for individual niche IoT thermostats and light strips with obscure weaknesses. Technically, CVEs are meant to only affect one codebase, so a vulnerability in a shared library often means a separate CVE for each affected product. It’s only when there’s no way to use the library without being vulnerable that they’d generally make just one CVE covering all affected products. [1] Even ignoring all that, people are incorporating Copilot into their development process, which makes it a common dependency. [1]: https://www.redhat.com/en/topics/security/what-is-cve https://www.redhat.com/en/topics/security/what-is-cve
- aspenmayer 1y agoIt’s true. The form is right here. When they support PGP, I suspect they know what they’re doing and why, and have probably been continuously doing so for longer than I have been alive. Just look at their sponsors and partners. https://cveform.mitre.org/ https://cveform.mitre.org/ Please only use this for legitimate submissions.
- thombles 1y agoIs there value in requesting a CVE for a service that only Microsoft runs? What's a user supposed to do with that?
- fulafel 1y agoCVEs are supposed to be unambigous references to vulnerabilities for communication, nothing more. So you can say stuff like "this happened was before CVE-XXXX was fixed, do we need to notify anyone about the risk of undetected insider info access?"
- fulafel 1y agoNot exactly. There are several CVE numbering authorities and some of them (including the original MITRE, national CERTs etc), accept submissions from anyone, but there's evaluation and screening. Since Microsoft is their own CNA, most of them probably wouldn't issue a MS CVE without some kind of exceptional reason.
- jayofdoom 1y agoMakes sense. I was wondering if that would be an issue. Thanks for the detail.
- nzeid 1y agoHard to count the number of things that can go wrong by relying directly on an LLM to manage audit/activity/etc. logs. What was their bug fix? Shadow prompts?
- gpm 1y agoI'd hope that if a tool the LLM uses reveals any part of the file to the LLM it counts as a read by every user who sees any part of the output that occurred after that revelation was added to the context.
- tatersolid 1y agoHow would you handle “company name” or other common phrases in search? Log 1M documents every time that phrase appeared in a copilot response for any user?
- jsnell 1y ago> Hard to count the number of things that can go wrong by relying directly on an LLM to manage audit/activity/etc. logs. Nothing in this post suggests that they're relying on the LLM itself to append to the audit logs. That would be a preposterous design. It seems far more likely the audit logs are being written by the scaffolding, not by the LLM, but they instrumented the wrong places. (I.e. emitting on a link or maybe a link preview being output, rather than e.g. on the document being fed to the LLM as a result of RAG or a tool call.) (Writing the audit logs in the scaffolding is probably also the wrong design, but at least it's just a bad design rather than a totally absurd one.)
- nzeid 1y agoHeard, but since the content or its metadata must be surfaced by the LLM, what's the fix?
- nzeid 1y agoThinking about this a bit - you'd have to isolate any interaction the LLM has with any content to some sort of middle end that can audit the LLM itself. I'm a bit out of my depth here, though. I don't know what Microsoft does or doesn't do with Copilot.
- thenaturalist 1y agoHardly have I ever seen corporate incentives so aligned to overhype the capabilities of a technology while it being so raw and unpolished as this one. The bubble bursting will be epic.
- bigbuppo 1y agoAt some point one of the big tech companies is going to be the next Sears.
- lokar 1y agoWait, copilot operates as some privileged user (that can bypass audit?), not as you (or better, you with some restrictions) That can’t be right, can it?
- ceejayoz 1y ago> That can’t be right, can it? https://knowyourmeme.com/memes/james-franco-first-time https://knowyourmeme.com/memes/james-franco-first-time
- lokar 1y agolol. I’ve avoided MS my entire (30+ year) career. Every now and then I’m reminded I made the right choice.
- trinsic2 1y agoI woke up to MS in 2023[0]. Never again. [0]: https://www.scottrlarson.com/publications/publication-transition-windows-to-linux/ https://www.scottrlarson.com/publications/publication-transi...
- tomrod 1y agoBrilliant.
- dhosek 1y agoThat was a laugh-out-loud moment in that film.
- tomrod 1y agoSure sounds like, for Microsoft, an audit log is optional when it comes to cramming garbage AI integrations in places they don't belong.
- Spooky23 1y agoNo, it accesses data with the users privilege.
- heywire 1y agoI am so tired of Microsoft cramming Copilot into everything. Search at $dayjob is completely borked right now. It shows a page of results, but the immediately pops up some warning dialog you cannot dismiss that Copilot can’t access some file “” or something. Every VSCode update I feel like I have to turn off Copilot in some new way. And now apparently it’ll be added to Excel as well. Thankfully I don’t have to use anything from Microsoft after work hours.
- keyle 1y agoEverything except the best thing they could have brought back: Clippy! </3
- fragmede 1y agoSo Louis Rossmann put out a YouTube video encouraging internet users to change their profile pictures to an image of Clippy, as a form of silent protest against unethical conduct by technology companies, so it's making a comeback!
- candiddevmike 1y agoRE: VSCode copilot, you're not crazy, I'm seeing it too. And across multiple machines, even with settings sync enabled, I have to periodically go on each one and uninstall the copilot extension _again_. I'll notice the Add to chat... in the right click context menu and immediately know it got reinstalled somehow. I'd switch to VSCodium but I use the WSL and SSH extensions :(
- userbinator 1y agoThankfully I don’t have to use anything from Microsoft after work hours. There are employers where you don't have to use anything from Microsoft during work hours either.
- troad 1y ago> Every VSCode update I feel like I have to turn off Copilot in some new way. This has genuinely made me work on switching to neovim. I previously demurred because I don't trust supply chains that are random public git repos full of emojis and Discords, but we've reached the point now where they're no less trustworthy than Microsoft. (And realistically, if you use any extensions on VS Code you're already trusting random repos, so you might as well cut out the middle man with an AI + spyware addiction and difficulties understanding consent.)
- xet7 1y agohttps://archive.is/PRTRA https://archive.is/PRTRA
- Josh5 1y agoare they even sure that the AI even accessed the content that second time? LLMs are really good and making up shit. I have tested this by asking various LLMs to scrape data from my websites while watching access logs. Many times, they don't and just rely on some sort of existing data or spout a bunch of BS. Gemini is especially bad like this. I have not used copilot myself, but my experience with other AI makes me curious about this.
- bongodongobob 1y agoThis is it. M365 uses RAG on your enterprise data that you allow it to access. It's not actually accessing the files directly in the cases he provided. It's working as intended.
- crooked-v 1y agoIf that's the case, then as noted in the article, the 'as intended' is probably violating liability requirements around various things.
- sailfast 1y agoCorrect. It is precisely that a user can ask about someone’s medical history (or whatever else) and not be reported that would be in violation of any heavily audited system. LLM Summaries break the compliance.
- bongodongobob 1y agoYou allow what it can and can't see. If you include PII and medical records, that's your fault, not MS's.
- sailfast 1y agoThat’s fair - unless they’re marketing the bot as compliant.
- 1y ago
- micromacrofoot 1y ago[flagged]
- QuadmasterXLII 1y agoThis seems like a five alarm fire for HIPPA, is there something I’m missing?
- loeg 1y agoIt's HIPAA.
- adzm 1y agoThe HIPAA hippo certainly encourages this confusion
- ivewonyoung 1y agoIt's HIPPA now for all intensive purposes.
- FergusArgyll 1y agoFor all intents and purposes
- deleted 1y ago[deleted]
- Spooky23 1y agoIt’s a bug. He reported it, they fixed it. It is not a five alarm fire for HIPAA. HIPAA doesn’t require that all file access be logged at all. HIPAA also doesn’t require that a CVE be created for each defect in a product. End of the day, it’s a hand-wavy, “look at me” security blog. Don’t get too crazy.
- waffleiron 1y agoI am more on the privacy side of things like HIPAA, but I would like to link the following. https://www.hhs.gov/sites/default/files/january-2017-cyber-newsletter.pdf https://www.hhs.gov/sites/default/files/january-2017-cyber-n...
- jeanlucas 1y agoA better title would be: Microsoft Copilot isn't HIPAA compliant A title like this will get it fixed faster.
- rst 1y agoIt already is fixed -- the complaint is that customers haven't been notified.
- whizzter 1y agoEven better, _ALL USEFUL_ AI retrival systems are insecure by design, because all those RAG vectors that sells vector-databases? That's basically your documents lossily encoded.
- adtac 1y ago>That's basically your documents lossily encoded. Vector embeddings are lossy encodings of documents roughly in the same way a SHA256 hash is a lossy encoding. It's virtually impossible to reverse the embedding vector to recover the original document. Note: when vectors are combined with other components for search and retrieval, it's trivial to end up with a horribly insecure system, but just vector embeddings are useful by themselves and you said "all useful AI retrieval systems are insecure by design", so I felt it necessary to disagree with that part.
- deleted 1y ago[deleted]
- troad 1y agoMicrosoft's ham-fisted strategy for trying to build a moat around its AI offering, by shoving everyone's documents in it without any real informed consent, genuinely beggars belief. It will not successfully create a moat - turns out files are portable - but it will successfully peeve a huge number of users and institutions off, and inevitably cause years of litigation and regulatory attention. Are there no adults left at Microsoft? Or is it now just Copilot all the way up?
- p_ing 1y agoCopilot pulls from the substrate, like many other apps. No files are store in Copilot. They’re usually on ODSP but could be in Dataverse or a non-Microsoft product like Confluence (there goes your moat!).
- throwaway984393 1y ago[dead]
- TheRoque 1y agoIn my opinion, using AI tools for programming at the moment, unless in a sandboxed environment and on a toy project, is just ludicrous. The amount of shady things going on in this domain (AI trained on stolen content, no proper attribution, not proper way to audit what's going out to third party servers etc.) should be a huge red flag for any professional developer.
- neuroelectron 1y agoThe icing on the shit cake is a text editor programmed in typeScript with an impossible to secure plugin architecture.
- ThrowawayTestr 1y agoCompanies won't use open source software because of licencing concerns but if you launder it through an LLM it's hunky-dory.
- JTbane 1y agoThis is kind of not true. Companies will gladly use MIT-style license open source software for on-premises proprietary products, and everything but AGPL-style software for cloud products.
- AdieuToLogic 1y ago> In my opinion, using AI tools for programming at the moment, unless in a sandboxed environment and on a toy project, is just ludicrous. Well put. The fundamental flaw is in trying to employ nondeterministic content generation based on statistical relevance defined by an unknown training data set, which is what commercial LLM offerings are, in an effort to repeatably produce content satisfying a strict mathematical model (program source code).
- mlyle 1y agoNearly as bad: trying to use systems made out of meat, evolved from a unrelated background and trained on an undocumented and chaotic corpus of data, to try and produce content satisfying a strict mathematical model.
- overgard 1y agoI don’t know much about audit logs, but the more concerning thing to me is it sounds like it’s up to the program reading the file to register an access? Shouldn’t that be something at the file system level? I’m a bit baffled why this is a copilot bug instead of a file system bug unless copilot has special privileges? (Also to that: ick!)
- IcyWindows 1y agoI suspect this might be typical RAG where there is a vector index or chucked data it looks at.
- degamad 1y agoOne thing that's not clear in the write-up here: *which* audit log is he talking about? Sharepoint file accesses? Copilot actions? Purview? Something else?
- RachelF 1y agoLots of things aren't clear. Copilot is accessing the indexed contents of the file, not the file itself, when you tell it not to access the file. The blog writer/marketer needs to look at the index access logs.
- internetter 1y ago> The blog writer/marketer needs to look at the index access logs. How can you say this if microsoft is issuing a fix?
- pnt12 1y agoBut those are technicalities. I imagine the intended feature is learning about who read some information, and who modified it. The implementation varies, but on a CRUD app it seems easy: an authenticated GET or PUT request against a file path - easy audit log. If you are copying information to another place, and make it accessible there in a lossy way that is hard to audit... you broke your auditing system. Maybe it's useful, maybe it's a trade-off, but is something that should be disclosed.
- mr_toad 1y agoNot all vector databases are lossy. And even if a lossy index is used it’s totally possible to identify the original source of the information.
- poemxo 1y agoI asked ChatGPT the same thing and got > The system being referred to in that explanation is Microsoft 365 (M365) / Office 365 audit logging, specifically the Unified Audit Log in the Microsoft Purview Compliance Portal.
- deleted 1y ago[deleted]
- zavec 1y agoJust to make sure I'm understanding footnote one correctly: it shows up (sometimes before and hopefully every time now) as a copilot event in the log, and there's no corresponding sharepoint event? From a brief glance at the O365 docs it seems like the 'AISystemPluginData` field indicates that the event in the screenshot showing the missing access is a copilot event (or maybe they all get collapsed into one event, I'm not super familiar with O365 audit logs), and I'm inferring from the footnote that there's not another sharepoint event somewhere in either the old or new version. But if there is one that could at least be a mitigation if you needed to do such a search on the activity before the fix.
- fulafel 1y ago> CVEs are given to fixes deployed in security releases when customers need to take action to stay protected. In this case, the mitigation will be automatically pushed to Copilot, where users do not need to manually update the product and a CVE will not be assigned. Is this a feature of CVE or of Microsoft's way of using CVE? It would seem this vulnerability would still benefit from having a common ID to be refrenced in various contexts (eg vulnerability research). Maybe there needs to be another numbering system that will enumerate these kinds of cases and doesn't depend on the vendor.
- deleted 1y ago[deleted]
- eleveriven 1y agoYeah, this feels more like Microsoft bending the CVE process to fit their PR needs than a limitation of CVEs themselves
- dathinab 1y agoMicrosoft CVE track security incidents/vulnerabilities just because you can emergency patch it out of band does not make it not an incident but it falls under a trend of Microsoft acting increasingly negligent/non trusteable when it comes to security, especially when it comes to clear reporting about incidents. Which when it comes to a provider of fundamental components like an OS or Claude is as important as getting security right.
- immibis 1y agoIt's a feature of CVE. The C stands for Common.
- kuschku 1y agoI'd argue it'd still make sense to assign a CVE here. While you don't need to coordinate patching, many companies will need to issue reports to hipaa/gdpr oversight agencies, customers, employees, etc and having a common id for this vulnerability would make it easier to reference it and any related information.
- stogot 1y agoRemember when CISA called Microsoft’s security culture deficient? https://www.cisa.gov/sites/default/files/2025-03/CSRBReviewOfTheSummer2023MEOIntrusion508.pdf https://www.cisa.gov/sites/default/files/2025-03/CSRBReviewO... And remember when the Microsoft CEO responded that they will care about security above all else? https://blogs.microsoft.com/blog/2024/05/03/prioritizing-security-above-all-else/ https://blogs.microsoft.com/blog/2024/05/03/prioritizing-sec... Doesn’t seem they’re doing that does it?
- userbinator 1y agoThey do care about security --- they care a lot about telling you about it.
- sub7 1y agoWindows and any softwaqre coming out of Redmond today is pure spyware with little to 0 utility. This Clippy 2.0 wave of apps will obviously be rejected by the market but it can't come soon enough. The higher $msft gets, the more pressure they have to be invasive and shittify everything they do.
- usr1106 1y agoHow does their auditing even work? Auditing should happen at kernel level, I sure hope they don't have Copilot in their kernel. So how can any access go unaudited? Well, the article did not say whether the unaudited access was possible in the opposite order after boot. First ask without reference and get it without audit log. Then ask without any limitation and get an audit log entry. Did Copilot just keep a buffer/copy/context of what it had before in the sequence described. I guess that would go without log entry for any program. So what did MS change or fix? Producing extra audit log entries from user space?
- catmanjan 1y agoIn this scenario Copilot is performing RAG, so the auditing occurs when Copilot returns hits from the vector search engine its connected to - it seems there was a bug where it would only audit when Copilot referenced the hits in its result. The correct thing to do would be to have the vector search engine do the auditing (it probably already does, it just isn't exposed via Copilot) because it sounds like Copilot is deciding if/when to audit things that it does...
- dmitrijbelikov 1y agoNobody usually bothers with logging actions with files, well, that is, it is like that almost everywhere. Downloading files is not a joke, there are many nuances, for example: - format - where to store - logging - info via headers
- Foobar8568 1y agoWe have cases were purview were missing logs. Fun stuff when we tried to figure out a postmortem at my work. Microsoft tools can't be trust anymore, something really broke since COVID...
- eleveriven 1y agoBetween product sprawl, rushed AI integrations, and weird transparency decisions, it feels like reliability and accountability took a backseat
- throw-qqqqq 1y agoAre reliability and accountability core values at MSFT? I don’t personally see that company as reliable or trustworthy at all.
- mrweasel 1y agoWhen Nadella became chairman of the board at Microsoft? That happened in 2021. It's entirely possible that with a new chairman came a new business strategy. Satya Nadella is a cloud guy and a lot of the complaints people have of the changes in Microsoft products is that they are increasingly reliant on cloud infrastructure.
- deleted 1y ago[deleted]
- dang 1y agoPlease don't post unsubstantive comments here.
- conartist6 1y agoSorry. I'd delete it, but I can't.
- dang 1y agoNo worries, I've done that now since we were the only ones in the subthread.
- eleveriven 1y agoThis is exactly the kind of issue that makes trust in large vendors like Microsoft feel more like a gamble than a guarantee
- deadbabe 1y agoSo who do you trust? A small mom and pop software biz?
- smolder 1y agoGenerally speaking, yes, relative to the giants. Smallsoft co at least isn't going to monetize every bit of knowledge about you directly. They'll probably leak stuff to chatgpt and so on, but it depends on the business purpose whether I'd be worried about that.
- myaccountonhn 1y agoMicrosoft has an incredibly abysmal track record. They really shouldn't be trusted.
- degrees57 1y agoI'd rather be a big fish in a small pond than a minnow in Microsoft's ocean.
- self_awareness 1y ago> You might be thinking, “Yikes, but I guess not too many people figured that out, so it’s probably fine.” To you, the reader of this comment: if you thought like this, the problem is also in you.
- OhioMan2943 1y agoGood old moralsoft
- aetherspawn 1y agoTrying to get off Microsoft right now for LOB apps … the incompetence (multiple hacks over the last few months, SSO zero day, and now learn Copilot ignores permissions when searching because the indexer runs as global admin) is getting just plain scary.
- Pavilion2095 1y agoWhoever designed this should be fired: https://ibb.co/yGHf2yB https://ibb.co/yGHf2yB
- planb 1y agoI am assigned to develop a company internal chatbot that accesses confidential documents and I am having a really hard time communicating this problem to executives: As long as not ALL the data the agent hat access too is checked against the rights of the current user placing the request, there WILL be ways to leak data. This means Vector databases, Search Indexes or fancy "AI Search Databases" would be required on a per user basis or track the access rights along with the content, which is infeasible and does not scale. And as access rights are complex and can change at any given moment, that would still be prone to race conditions.
- cryptonym 1y agoTrue, per user doesn't scale. Knowledge should be properly grouped and have rights on database, documents, and chatbot managed by groups. For instance specific user can use the Engineering chatbot but not the Finance one. If you fail to define these groups, feels like you don't have a solid strategy. In the end, if that's what they want, let them experience open knowledge.
- planb 1y agoYeah. If you have knowledge stored in a structured form like that, you don't need an AI...
- cryptonym 1y agoIf organisation is that bad that finance docs are mixed with engineering docs, how do you even onboard people? You manually go through every single doc and decide if the newcomer can or can't access it? You should see our Engineering knowledge base before saying an AI would be useless.
- 9dev 1y agoAs if knowledge was ever that clear cut. Sometimes you need a cross-department insight, some data points from finance may not be confidential, some engineering content may be relevant to sales support… there’s endless reasons why neat little compartments like this don’t work in reality.
- nerdjon 1y agoI am very curious realistically how can they reliably fix this. So my understanding is that this is that the database/index that copilot used already crawled this file so of course it would not need to access the file to be able to tell the information in it. But then, how do you fix that? Do you then tie audit reports to accessing parts of the database directly? Or are we instructing the LLM to do something like... "If you are accessing knowledge pinky promise you are going to report it so we can add an audit log" This really needs some communication from Microsoft on exactly what happened here and how it is being addressed since as of right now this should raise alarm bells for any company using Copilot and people have access to sensitive data that needs to be strictly monitored.
- roywiggins 1y agoIt seems to me that the contents of the file cached in the index has to be dumped into the LLM's context at some point for it to show up in the result, so you can do the audit reports at that point.
- fud101 1y ago[flagged]
- thayne 1y agoI'm curious how they fixed this. Did they actually ensure the audit log is updated, or did they just give copilot some new instructions to update the audit log that could possibly be bypassed with the right prompt?
- selinkocalar 1y agoThis is terrifying from a compliance perspective. Audit logs are literally the foundation of every security framework - SOC 2, HIPAA, ISO 27001, you name it. If your audit logs are broken and you don't know it, you're not just non-compliant - you have no idea what's happening in your environment. The fact that Microsoft isn't proactively notifying customers makes this 10x worse. How many companies are going into audits with incomplete logs and don't even know it?