12 ms·

Reverse engineering a $1B Legal AI tool exposed 100k+ confidential files

821 points by bearsyankees 11mo ago

observationist 11mo ago
I think this class of problems can be protected against. It's become clear that the first and most important and most valuable agent, or team of agents, to build is the one that responsibly and diligently lays out the opsec framework for whatever other system you're trying to automate. A meta-security AI framework, cursor for opsec, would be the best, most valuable general purpose AI tool any company could build, imo. Everything from journalism to law to coding would immediately benefit, and it'd provide invaluable data for post training, reducing the overall problematic behaviors in the underlying models. Move fast and break things is a lot more valuable if you have a red team mechanism that scales with the product. Who knows how many facepalm level failures like this are out there?
- croes 11mo ago
  > I think this class of problems can be protected against. Of course, it’s called proper software development
  - jeffbee 11mo ago
    The techniques for non-disclosure of confidential materials processed by multi-tenant services are obvious, well-known, and practiced by very few.
  - venturecruelty 11mo ago
    And jail time for executives who are responsible for data leaks.
    - marginalx 11mo ago
      Are you saying executives cannot make mistakes ever (ask because you didn't qualify your statement)?
      - venturecruelty 11mo ago
        I'm saying that if executives get praise and bonuses for when good things happen, they should also have negative consequences when bad things happen. Litigate that further how you wish.
      - dghlsakjg 11mo ago
        The key word in that is "responsible". The legal world has plenty of ways for determining if you are legally responsible for the outcome of an event. Right now the standard is civil punishments for provable negligence. It sounds like GP is proposing a framework where we tighten up the definition of negligence, and add criminal penalties in addition to civil ones.
      - pbhjpbhj 10mo ago
        Are you saying the OP was just a single error, effectively an executives typo.
- dghlsakjg 11mo ago
  In this case, AI was a red herring. This was just plain terrible web security.
canopi 11mo ago
The first thing that comes to my mind is SOC2 HIPAA and the whole security theater. I am one of the engineers that had to suffer through countless screenshots and forms to get these because they show that you are compliant and safe. While the real impactful things are ignored
- latchkey 11mo ago
  SemiAnalysis made this a base requirement for being appropriately ranked on their ClusterMAX report, telling me it is akin to FAA certifications, and then getting hacked themselves for not enforcing simple security controls. https://jon4hotaisle.substack.com/i/180360455/anatomy-of-the-hijacking https://jon4hotaisle.substack.com/i/180360455/anatomy-of-the... It is crazy how this gets perpetuated in the industry as actually having security value, when in reality, it is just a pay-to-play checkbox.
- chickensong 11mo ago
  You have to start somewhere though. Security theater sucks, and it's not like compliance is a silver bullet, but at least it's something. Having been through implementing standards compliance, it did help the company in some areas. Was it perfect? Definitely not. Was it driven by financial goals? Absolutely. It did tighten up some weak spots though. If the options mainly consist of "trust me bro" vs "we can demonstrate that we put in some effort", the latter seems more preferable, even if it's not perfect.
quapster 11mo ago
[flagged]
- j45 11mo ago
  It's a little hilarious. First, as an organization, do all this cybersecurity theatre, and then create an MCP/LLM wormhole that bypasses it all. All because non-technical folks wave their hands about AI and not understanding the most fundamental reality about LLM software being fundamentally so different than all the software before it that it becomes an unavoidable black hole. I'm also a little pleased I used two space analogies, something I can't expect LLMs to do because they have to go large with their language or go home.
  - jimbokun 11mo ago
    My first reaction to the announcement of MCP was that I must be missing something. Surely giving an LLM unlimited access to protected data is going to introduce security holes?
    - j45 10mo ago
      Agree. It’s assuming and estimating it will behave like other software before it when it’s nothing like the software that came before it. LLMs today won’t behave like the software we’re used to where 1+1 will equal 2 every time.
      - jimbokun 10mo ago
        It’s like handing access to your sensitive data to a complete stranger and hoping for the best.
  - RansomStark 11mo ago
    Maybe this is the key takeaway of GenAI: that some access to data, even partially hallucinated data, is better than the hoops that the security theatre puts in place that prevents average Joe doing their job. This might just be a golden age for getting access to the data you need for getting the job done. Next security will catch up and there'll be a good balance between access and control. Then, as always security goes to far and nobody can get anything done. It's a tale as old as computer security.
electric_muse 11mo ago
[flagged]
- snapcaster 11mo ago
  What makes you think that? it would need some prompt engineering if so since ChatGPT won't write like that (bad capitalization, lazy quoting) unless you ask it to
  - lazide 11mo ago
    “Chat, write me a blog article that seems like a lazy human who failed English wrote it”?
    - legostormtroopr 11mo ago
      What’s worse being accused of an AI post or being defended because your post is so bad that AI wouldn’t have written it?
    - FrustratedMonky 11mo ago
      Well then that's everything.
- simonw 11mo ago
  That comment didn't read like AI generated content to me. It made useful points and explained them well. I would not expect even the best of the current batch of LLMs to produce an argument that coherent. This sentence in particular seems outside of what an LLM that was fed the linked article might produce: > What's wild is that nothing here is exotic: subdomain enumeration, unauthenticated API, over-privileged token, minified JS leaking internals.
  - samdoesnothing 11mo ago
    It's probably a list of bullet points or disjointed sentences fed to the LLM to clean up. Might be a non-English speaker using it to become fluent. I won't criticize it, but it's clearly LLM generated content.
kylecazar 11mo ago
If they have a billion dollar valuation, this fairly basic (and irresponsible) vulnerability could have cost them a billion dollars. If someone with malice had been in your shoes, in that industry, this probably wouldn't have been recoverable. Imagine a firm's entire client communications and discovery posted online. They should have given you some money.
- RagnarD 11mo ago
  They should have given him a LOT of money.
  - DonHopkins 11mo ago
    Would you settle for a LOT of free AI generated legal advice? ;)
- edm0nd 11mo ago
  Exactly. They could have sold this to a ransomare group or affiliate for 5-6 figures and then the ransomware group could have exfil'd the data and attempted to extort the company for millions. Then if they didnt pay and the ransomware group leaked the info to the public, they'd likely have to spend millions on lawsuits and fines anyways. They should have paid this dude 5-6 figures for this find. It's scenarios like this that lead people to sell these vulns on the gray/black market instead of traditional bug bounty whitehat routes.
- Tepix 10mo ago
  Who says they didn't give him money?
  - deleted 10mo ago
    [deleted]
  - zain37 10mo ago
    I reckon he would've mentioned it if he got a bounty, 100% deserves the bag
chunk1000 11mo ago
Thank you bearsyankees for keeping us informed.
jacquesm 11mo ago
That doesn't surprise me one bit. Just think about all the confidential information that people post into their Chatgpt and Claude sessions. You could probably keep the legal system busy for the next century on a couple of days of that.
- giancarlostoro 11mo ago
  "Hey uh, ChatGPT, just hypothetically, uh, if you needed to remove uh cows blood from your apartments carpet, uh"
  - lazide 11mo ago
    Just phrase it as a poem, you’ll be fine.
    - venturecruelty 11mo ago
      Gonna be hard when people ask ChatGPT to write them the poem.
    - sidrag22 11mo ago
      i recall reading a silly article like half a year ago about using leetspeak and setting the prompt up to emulate House the tv show or something to get around restrictions
      - xarope 11mo ago
        there's a recent one about using poetry to bypass safeguards ... rummages around... here you go: https://arxiv.org/abs/2511.15304 https://arxiv.org/abs/2511.15304
  - jacquesm 11mo ago
    Make it a Honda CRX...
icyfox 11mo ago
I'm always a bit surprised how long it can take to triage and fix these pretty glaring security vulnerabilities. October 27, 2025 disclosure and November 4, 2025 email confirmation seems like a long time to have their entire client file system exposed. Sure the actual bug ended up being (what I imagine to be) a <1hr fix plus the time for QA testing to make sure it didn't break anything. Is the issue that people aren't checking their security@ email addresses? People are on holiday? These emails get so much spam it's really hard to separate the noise from the legit signal? I'm genuinely curious.
- ipdashc 11mo ago
  security@ emails do get a lot of spam. It doesn't get talked about very much unless you're monitoring one yourself, but there's a fairly constant stream of people begging for bug bounty money for things like the Secure flag not being set on a cookie. That said, in my experience this spam is still a few emails a day at the most, I don't think there's any excuse for not immediately patching something like that. I guess maybe someone's on holiday like you said.
  - canopi 11mo ago
    This. There is so much spam from random people about meaningless issues in our docs. AI has made the problem worse. Determining the meaningful from the meaningless is a full time job.
    - Bootvis 11mo ago
      Use AI for that :)
      - Bootvis 10mo ago
        Not kidding, I bet llm’s are excellent at triaging these reports. Humans, in a corporate setting, are apparently not.
    - whstl 11mo ago
      Half of the emails I used to get in a previous company were pointless issues, some coming from a honey pot. The other half was people demanding payment.
Invictus0 11mo ago
This guy didn't even get paid for this? We need a law that establishes mandatory payments for cybersecurity bounty hunters.
- deleted 11mo ago
  [deleted]
_f9cu 11mo ago
Legal attacks engineering - font type license fee on japan consumers. Engineering attacks legal - AI info dump in above post. How does above sound like and what kind of professional write like that?
sys32768 11mo ago
I work for a finance firm and everyone is wondering why we can store reams of client data with SaaS Company X, but not upload a trust document or tax return to AI SaaS Company Y. My argument is we're in the Wild West with AI and this stuff is being built so fast with so many evolving tools that corners are being cut even when they don't realize it. This article demonstrates that, but it does sort of beg the question as to why not trust one vs the other when they both promise the same safeguards.
- layer8 11mo ago
  The question is what reason did you have to trust SaaS Company X in the first place?
  - sys32768 11mo ago
    Because it's the Cloud and we're told the cloud is better and more secure. In truth the company forced our hand by pricing us out of the on-premise solution and will do that again with the other on-premise we use, which is set to sunset in five years or so.
    - ansgri 10mo ago
      Probably has more to do with responsibility outsourcing: if SaaS has security breach AND they tell in the contract that they’re secure, then you’re not responsible. Sure, there may be reputational damage for you, but it’s a gamble with good odds in most cases. Storing lots of legal data doesn’t seem to be one of these cases though.
      - bonesss 10mo ago
        I see profits and outsourcing. Selling an on-premise service requires customer support, engineering, and duplication of effort if you’re pushing to the cloud as well. Then you get the temptations and lock in of cloud-only tooling and an army of certified consultant drones whose resumes really really need time on AWS-doc-solution-2035, so the on premise becomes a constant weight on management. SaaS and the cloud is great for some things some of the time, but often you’re just staring at the marketing playbook of MS or Amazon come to life like a golem.
lupire 11mo ago
Who is Margolis, and are they happy that OP publicly announced accessing all their confidential files? Clever work by OP. Surely there is automatic prober tool that already hacked this product?
- dghlsakjg 11mo ago
  > Who is Margolis, and are they happy that OP publicly announced accessing all their confidential files? Google tells me they are a NY law firm specializing in Real Estate and Immigration law. There are other firms with Margolis in the name too. Kinda doesn't matter; see below. I doubt that they are thrilled to have their name involved in this, but that is covered by the US constitution's protections on free press.
richwater 11mo ago
Of course there will be no accountability or punishment.
valbaca 11mo ago
Given the absurd amount startups I see lately that have the words "healthcare" and "AI", I'm actually incredibly concerned that in just a couple of months we're going to have an multiple, enormous HIPAA-data disasters Just search "healthcare" in https://news.ycombinator.com/item?id=46108941 https://news.ycombinator.com/item?id=46108941
mattfrommars 11mo ago
This might be off topic since we are in topic of AI tool and on HackerNews. I've been pondering a long time how does one build a startup company in domain they are not familiar with but ... Just have this urge to 'crave a pie' in this space. For the longest time, I had this dream of starting or building a 'AI Legal Tech Company' -- big issue is, I don't work in legal space at all. I did some cold reach on lawfirm related forums which did not take any traction. I later searched around and came across the term, 'case management software'. From what I know, this is what Cilo fundamentally is and make millions if not billion. This was close to two years or 1.5 years ago and since then, I stopped thinking about it because of this understanding or belief I have, "how can I do a startup in legal when I don't work in this domain" But when I look around, I have seen people who start companies in totally unrelated industry. From starting a 'dental tech's company to, if I'm not mistaken, the founder of hugging face doesn't seem to have PHD in AI/ML and yet founded HuggingFace. Given all said, how does one start a company in unrelated domain? Say I want to start another case management system or attempt to clone FileVine, do I first read up what case management software is or do I cold reach to potential lawfirm who would partner up to built a SAAS from scratch? Other school of thought goes like, "find customer before you have a product to validate what you want to build", how does this realistically work? Apologies for the scattered thoughts...
- jimbokun 11mo ago
  One approach is to partner with someone who is an expert in that space.
- strgcmc 11mo ago
  I think it comes down to, having some insight about the customer need and how you would solve it. Having prior experience in the same domain is helpful but is neither a guarantee nor a blocker, towards having a customer insight (lots of people might work in a domain but have no idea how to improve it; alternatively an outsider might see something that the "domain experts" have been overlooking). I just randomly happened to read about the story of, some surgeons asking a Formula 1 team to help improve its surgical processes, with spectacular results in the long term... The F1 team had zero medical background, but they assessed the surgical processes and found huge issues with communication and lack of clarity, people reaching over each other to get to tools, or too many people jumping to fix something like a hose coming loose (when you just need 1 person to do that 1 thing). F1 teams were very good at designing hyper efficient and reliable processes to get complex pit stops done extremely quickly, and the surgeons benefitted a lot from those process engineering insights, even though it had nothing specifically to do with medical/surgical domain knowledge. Reference: https://www.thetimes.com/sport/formula-one/article/professor-martin-elliott-interview-surgeon-f1-pitstops-babies-ferrari-j3sbkjssk https://www.thetimes.com/sport/formula-one/article/professor... Anyways, back to your main question -- I find that it helps to start small... Are you someone who is good at using analogies to explain concepts in one domain, to a layperson outside that domain? Or even better, to use analogies that would help a domain expert from domain A, to instantly recognize an analogous situation or opportunity in domain B (of which they are not an expert)? I personally have found a lot of benefit, from both being naturally curious about learning/teaching through analogies, finding the act of making analogies to be a fun hobby just because, and also honing it professionally to help me be useful in cross-domain contexts. I think you don't need to blow this up in your head as some big grand mystery with some big secret cheat code to unlock how to be a founder in a domain you're not familiar with -- I think you can start very small, and just practice making analogies with your friends or peers, see if you can find fun ways of explaining things across domains with them (either you explain to them with an analogy, or they explain something to you and you try to analogize it from your POV).
fallinditch 11mo ago
> ... after looking through minified code, which SUCKS to do ... AI tends to be good at un-minifying code.
- a_victorp 11mo ago
  Legit question: when working on finding security issues, are there any guidelines on what you can send to LLMs/AI?
  - fallinditch 11mo ago
    I got downvoted, so maybe that means someone thinks un-minifying code is not advised for dealing with security issues? But on reflection surely you can just use the 'format code' command in the ide? I am no expert but surely it's ok to use AI to help track down and identify security issues with the usual caveats of 'don't believe it blindly, do your double checking and risk assessing.'
  - CER10TY 11mo ago
    Personally, I'd just use common sense and good judgment. At the end of the day, would you want someone to hand your address, and other private data to OpenAI just like that? Probably not. So don't paste customer data into it if you can avoid it. On the other hand, minified code is literally published by the company. Everyone can see it and do with it as they please. So handing that over to an AI to un-minify is not really your problem, since you're not the developer working on the tool internally.
- nodesocket 11mo ago
  Doesn't Chrome Developer tools automatically un-minify?
yieldcrv 11mo ago
I've worked in several "agentic" roles this year alone (I'm very poachable lol) and otherwise well structured engineering orgs have lost their goddamn minds with move fast and break things because they're worried that OpenAI/Google/Meta/Amazon/Anthropic will release the tool they're working on tomorrow literally all of them are like this
- trollbridge 11mo ago
  Old school blue chip type of companies are like this too. They’ve thrown all the process and caution they used to have to the wind so that they can… apply AI to their IT org which isn’t even their core business?
2ndatblackrock 11mo ago
now that's just great hacking
corry 11mo ago
"Companies often have a demo environment that is open" - huh? And... Margolis allowed this open demo environment to connect to their ENTIRE Box drive of millions of super sensitive documents? HUH???! Before you get to the terrible security practices of the vendor, you have to place a massive amount of blame on the IT team of Margolis for allowing the above. No amount of AI hype excuses that kind of professional misjudgement.
- me_again 11mo ago
  I don't think we have enough information to conclude exactly what happened. But my read is the researcher was looking for demo.filevine.com and found margolis.filevine.com instead. The implication is that many other customers may have been vulnerable in the same way.
  - corry 10mo ago
    Ah, I see now that I read too quickly - the "open demo environment" was clearly referencing the idea that the vendor (Filevine) would have a live demo, NOT that each client wanted an open playground demo account that is linked to a subset of their data (which would be utterly insane).
etamponi 11mo ago
I don't disagree with the sentiment. But let's also be honest. There is a lot of improvement to be made in security software, in terms of ease of use and overcomplicating things. I worked at Google and then at Meta. Man, the amount of "nonsense" of the ACL system was insane. I write nonsense in quotes because for sure from a security point of view it all made a lot of sense. But there is exactly zero chance that such a system can be used in a less technical company. It took me 4 years to understand how it worked... So I'll take this as another data point to create a startup that simplifies security... Seems a lot more complicated than AI
deep_thinker26 11mo ago
It's so great that they allowed him to publish a technical blog post. I once discovered a big vulnerability in a listed consumer tech company -- exposing users' private messages and also allowing to impersonate any user. The company didn't allow me to write a public blogpost.
- qmr 11mo ago
  "Allow"? Go on write your blog post. Don't let your dreams be dreams.
  - bigmadshoe 11mo ago
    Presumably they were paid for finding the bug and inn accepting relinquished their right to blog about it.
    - hsbauauvhabzb 11mo ago
      No, you relinquish the right when you agree to their TOS irrespective of if they pay you.
      - amackera 11mo ago
        TOS != law They will stop letting you use the service. That's the recourse for breaking the TOS.
        advisedwang 11mo ago
        Up until Van Buren v. United States in 2020, ToS violations were sometimes prosecuted as unauthorized access under the CFAA. I suspect there are other jurisdictions that still do the equivalent to that.
        hsbauauvhabzb 11mo ago
        I don’t want to pay for a lawyer to argue that for me. != law does not equate to ‘won’t come with a cost’. I say this as someone threatened by a billion dollar company for this very thing.
hbarka 11mo ago
> November 20, 2025: I followed up to confirm the patch was in place from my end, and informed them of my intention to write a technical blog post. Can that company tell you to cease and desist? How does the law work?
- me_again 11mo ago
  Lawyers can and will send cease and desist letters to people whether or not there is any legal basis for it. Often the threat of a lawsuit, even a meritless one, is enough to keep people quiet.
- dghlsakjg 11mo ago
  FYI, a "cease and desist" carries the same legal weight as me sending a one-liner saying "Knock it off". They are strongly worded requests from a legal point of view. The only real message they send is that the sender is serious enough about the issue to have involved a lawyer, unless of course you write it yourself, which is something that literally anyone can do. If you want to actually force an action, you need a court order of some type. NB for the actual lawyers: I'm oversimplifying, since they can be used in court to prove that you tried to get the other party to stop, and tried to resolve the issue outside of court.
stanfordkid 11mo ago
I mean... in what world would you send a customers private root key to a web browsing client. Like even if the user was authenticated why would they need this? This sort of secret shouldn't even be in an environment variable or database but stored with encryption at rest. There could easily have been a proxy service between client and box if the purpose is to search or download files. It's very bad, even for a prototype... this researcher deserves a bounty!
bzmrgonz 11mo ago
My thing is, even ingesting the BOK should have been done in phases, to avoid having all your virtual eggs in one basket or nest at any ONE time. Staggering tokens to these compartments would not have cost them anything at all . I always say, whatever convenience you enjoy yourself, will be highly appreciated by bad actors... WHEN, not if.. they get thru.
deleted 11mo ago
[deleted]
badbird33 11mo ago
You'd think with a $1B valuation they could afford a pentest
magnetowasright 11mo ago
I am at a loss for words. This wasn't a sophisticated attack. I'd love to know who filevine uses for penetration testing (which they do, according to their website) because holy shit, how do you miss this? I mean, they list their bug bounty program under a pentesting heading, so I guess it's just nice internet people. It's inexcusable.
- rashidujang 11mo ago
  This was my impression after reading the article too. I have no doubt that the team at Filevine attempted to secure their systems and have probably thwarted other attackers, but got their foot stuck in what is an unsophisticated attack. It only takes one chain vulnerability to bring down the site. Security reminds me of the Anna Karenina principle: All happy families are alike; each unhappy family is unhappy in its own way.
- GJim 10mo ago
  > I am at a loss for words. This wasn't a sophisticated attack. To be fair, data security breaches seldom are.
ethin 11mo ago
I'll be honest... I'm not at all surprised that this happened. Purely because it seems like everyone who wants to implement AI just forgot all of the institutional knowledge that cybersecurity has acquired over the last 30-40 years. When you "forget" all of that because you want to rush out something really fast, well, you know what they say: play stupid games, win stupid prizes and all that.
culanuchachamim 11mo ago
-The Filevine team was responsive, professional, and took the findings seriously throughout the disclosure process. They acknowledged the severity, worked to remediate the issues, allowed responsible disclosure, and maintained clear communication. This is another great example of how organizations should handle security disclosures. In the same tenure I think that a professional etical hacker or a curious fellow that is poking around with no harm intent, shouldn't disclose the name of the company that had a security issue if they resolve it professionally. You can write the same blog post without mentioning that it was Filevine. If they didn't take care of the incident that's a different story...
- CBMPET2001 11mo ago
  Eh, with something this horrendously egregious I think their customers have a right to know how carelessly their data was handled, regardless of the remediation steps taken after disclosure; that aside, who knows how many other AI SaaS vendors might stumble across this article and realize they've made a similarly boneheaded error, and save both themselves and their customers a huge amount of pain . . .
- evan_a_a 11mo ago
  This is a very standard part of responsible disclosure. Hacker finds bugs -> discloses them to the vendor -> (hopefully) the vendor communicates with them and remediates -> both sides publish the technical details. It also helps to demonstrate to the rest of the security world which companies will take reports seriously and which ones won’t, which is very useful information to have.
- deelowe 11mo ago
  That's not how ethical disclosure works. Both parties should publish and we, the wider tech industry should see this as a good thing both for the hacker and the company that worked with them.
- manbash 11mo ago
  How else can you take responsibility if you don't make it public? You can't have integrity if you hide away your faults.
theoldgreybeard 11mo ago
The bigwigs at my company want to build out a document management suite. After talking to VP of technology about requirements I ask about security as well as what the regulatory requirements are and all I get is a blank stare. I used to think developers had to be supremely incompetent to end up with vulnerabilities like this. But now I understand it’s not the developers who are incompetent…
- eru 11mo ago
  There's enough incompetence at all levels to go around.
  - theoldgreybeard 11mo ago
    Maybe I have just been lucky, but I have not had the displeasure of working with people either tha incompetent or willfully ignorant yet.
    - eru 11mo ago
      Oh, I should have been more careful in my formulation: There are organisations that are generally competent, and there are places that are less competent. It's not all that uncommon for the whole organisation to be generally incompetent. The saddest places (for me) are those where almost every individual you talk to seems generally competent, but judging by their output the company might as well be stuffed by idiots. Something in the way they are organised suppresses the competence. (I worked at one such company.) > Maybe I have just been lucky, but I have not had the displeasure of working with people either tha incompetent or willfully ignorant yet. It's very important before you start any new job to suss out how competent people and the organisation are. Ideally, you probably want to work for a competent company. But at least you want to know what you are getting into. There's a bit of luck involved, if you go in blindly, but you can also use skill and elbow-grease to investigate.
      - chii 11mo ago
        > suss out how competent people and the organisation are. how does one do this, without first having the job and being embedded in there? From the outside, it's near impossible to see these details imho.
habosa 11mo ago
They took a month to fix this? That’s beyond inexcusable. I can’t imagine how any customer could justify working with them going forward. Also … shows you what a SOC 2 audit is worth: https://www.filevine.com/news/filevine-proves-industry-leadership-in-data-security-with-soc-2-and-soc-3-compliance/ https://www.filevine.com/news/filevine-proves-industry-leade... Even the most basic pentest would have caught this.
- aitchnyu 11mo ago
  Is there any stricter standard? Should one strive for PCI-DSS even if they are a regular SaaS?
  - eru 10mo ago
    Whatever Google does internally would be a much stricter standard, but I'm not sure they've written it up for outsiders to use, alas.
- eru 11mo ago
  It looks like SOC 2 (and the other SOCs) where developed by accountants? I wouldn't expect them to find any computer problems either to be honest.
  - anticensor 10mo ago
    There are only 3 books of SOC: SOC I, SOC II Part 1, SOC II Part II.
- jonny_eh 11mo ago
  Where did it say that they took a month to fix? The hacker just checked in 2 weeks later and it was fixed by that point.
  - theodorejb 11mo ago
    According to the timeline it took more than a week just for Filevine to respond saying they would review and fix the vulnerability. It was 24 days after initial disclosure when he confirmed the fix was in place.
satya71 11mo ago
I had a look at some of FileVine's output, and I can say I'm not surprised. This is not an organization that prizes engineering at all.
BrenBarn 11mo ago
I'm less and less sure that when a billion-dollar company screws up this bad, the right thing to do is privately disclose it and let them fix it. This kind of thing just allows companies to go on taking people's money without facing the consequences of their mistakes.
- barbazoo 11mo ago
  Does a disclosure like this absolve them of any responsibility? They still violated whatever user privacy act.
  - mvkel 11mo ago
    It does. Most privacy laws are based on time-from-discovery. If they immediately sprung into action at the moment they were informed and remediated the issue, they're in compliance.
    - mattmaroon 11mo ago
      So is that true if they find out when the public does too? It seems that disclosing it privately has some upside (protecting the users) and no downside.
      - mvkel 11mo ago
        That depends more on what the Privacy Policy is of the service, which you agree to when you sign up and use it
    - BrenBarn 10mo ago
      Right, that's the problem. There need to be standards that govern what can ever be released to customers/the public in the first place. When violations of those are discovered, the penalties should be based on time from release, so the longer it was out in the wild, the greater the penalty.
      - 10mo ago
0xbadcafebee 11mo ago
So, 1) a public service, 2) with no authentication, 3) and no encryption? (http only??), 4) sent every single response with a token, 5) giving full admin access to every client's legal documents. This is like a law firm with an open back door, open back window, and all the confidential legal papers sprawled out on the floor. Imagine the potential impact. You're a single mother, fighting for custody of your kids. Your lawyer has some documentation of something that happened to you, that wasn't your fault, but would look bad if brought up in court. Suddenly you receive a phone call - it's a mysterious voice, demanding $10,000 or they will send the documents to the opposition. Neither of them knows each other; someone just found a trove of documents in an open back door and wanted to make a quick buck. This is exactly what a software building code would address (if we had one!). Just like you can't open a new storefront in a new building without it being inspected, you should not be able to process millions of sensitive files without having your software's building inspected. The safety and privacy of all of us shouldn't be optional.
- Fnoord 11mo ago
  http-only makes it also very easy to sniff for LE if they decide to. This allows them to get knowledge about cases. Like, they could be scanning it with their own AI tool for all we know. In a free country with proper LE, this would neither be legal nor happening. But I am not sure the USA is remaining one, given the leader is a convicted felon with very dubious moral standards. The problem here however is that they get away with their sloppiness as long as the security researcher who found this is a whitehat, and the regular news won't pick it up. Once regular media pick this news up (and the local ones should), their name is tarnished and they may regret their sloppiness. Which is a good way to ensure they won't make the same mistake. After all, money talks.
  - zwnow 10mo ago
    All the big tech companies are in the news every week. Everybody knows how bad they are. Their names are tarnished and yet everyone is still using their junk and they face zero repercussions when fucking up. I dont think things in the media would do any harm.
kitschman 11mo ago
[dead]
1vuio0pswjnm7 11mo ago
"(after looking through minified code, which SUCKS to do)" Would there be a "pretty printer" or some other "unminifier" for this task If not, then is minification effectively a form of obfuscation
- gu5 11mo ago
  https://webcrack.netlify.app/ https://webcrack.netlify.app/ was made exactly for this, it's a really useful tool
densone 11mo ago
If I had a dollar for every time I saw something like this or for every time someone exposed a .git with a .env full of secrets
keernan 10mo ago
Attorneys are ethically obligated to follow very stringent rules to protect their client's confidential information. Having been a practicing litigator for 40+ years, I can confidently state I came across very few attorneys who truly understood their obligations. Things were easier when I first began practicing in the 1970s. There weren't too many ways confidential materials in our files could be compromised. Leaving my open file spread out on the conference room table while I went to lunch while attorneys arriving for a deposition on my partner's case were one by one seated into the conference room. That's the kind of thing we had to keep an eye on. But things soon got complicated. Computers. Digital copies of files that didn't disappear into an external site for storage like physical files. Then email. What were our obligations to know what could - and could not - be intercepted while email traveled the internet. Then most dangerous of all. Digital storage that was outside our physical domain. How could we now know if the cloud vendor had access to our confidential data? Where were the backups stored? How exactly was the data securely compartmentalized by a cloud vendor? Did we need our own IT experts to control the data located on the external cloud? What did the contracts with the cloud vendor say about the fact we were a law firm and that we, as the lawyers responsible for our clients confidential information, needed to know that they - the cloud vendor - understood the legal obligations and that they - the cloud vendor - would hire lawyers to oversee the manner in which the cloud vendor blocked all access to the legal data located on their own servers. And so on and so forth. I'm no longer in active practice but these issues were a big part of my practice my last few years at a Fortune 500 insurance company that used in-house attorneys nationwide to represent insureds in litigation - and the corporation was in engaged in signing onto a cloud service to hold all of the corporate data - including the legal departments across all 50 states. It was a nightmare. I'm confident it still is.
tonyhart7 10mo ago
so the future is software engineer cosplaying an cybersecurity so we can implement BASIC security????
hansmayer 10mo ago
For all the talk in the blog of how "super-professional" their team was (probably just a courtesy on the side of the author, I don´t think he believes his own words either)... I have noticed using AI to produce some kind of API -OR- use a 3rd party point with integration into frontend, is almost guaranteed to produce code in which the frontend either exposes the API secrets directly in the frontend code (literally injecting it into a variable as string), or if you ask it for authentication, it will build some half-built lazy solution which makes no sense. So I imagine their "super-professional" team built this with AI, blindly trusting, probably even allowing it to commit and merge changes itself because if you are not merging 10K LoC a day with all this "great" technology, what are you even doing, right? It is not super-professional to work effectively with a blindfold on, I´d argue.
MangoToupe 10mo ago
> countless data protected by HIPAA People should really look this law up before they reference it
xp84 10mo ago
Through that API, the frontend is handed a high-privilege API key for Box. Not only was that a huge blunder on the backend, but it reveals what passes for architecture these days. Should our application's backend speak to the super sensitive file store? No, we should hand over the keys to that to the React app, because it literally did not occur to them that there's anything that physically could be driven by the frontend that shouldn't be. My apologies to the frontend engineers out there who know what they're doing.
nstj 10mo ago
The post is a disclosure about a vuln which seem to have anything to do with AI.
Fokamul 10mo ago
"Coding is done", Adam Wolff, Anthropic
aperture147 10mo ago
Hey I think I've just found a new marketing stunt for a new vibe-coding platform: "Worried your vibe-coded app is about to be broadcast on the internet’s biggest billboard? Chill. ACME AI now wraps it in “NSA-grade” security armor." I've never thought that there will be multiple billion-dollar-AI-features that fixes all the monkey patching problems that no one saw them coming from the older billion-dollar-AI-features that fixes all the monkey patching problems that no one saw them coming from...
canto 10mo ago
How is following a http request and guessing some variables a "reverse" engineering now?
testemailfordg2 10mo ago
When liability of a corporation and its owners is limited, does it benefit their business to be dilligent in every step or have a mentality of move fast and break things?
6thbit 10mo ago
Man. Hopefully their remediation steps included a full audit of their Box's account. One could only imagine that if OP wasn't the first to discover it, people could've generated tons of shared links for all kinds of folders, for instance, which would remain active even if they invalidated the API token.
larrysanchez77 10mo ago
[dead]

821 points by bearsyankees 11mo ago

I think this class of problems can be protected against. It's become clear that the first and most important and most valuable agent, or team of agents, to build is the one that responsibly and diligently lays out the opsec framework for whatever other system you're trying to automate. A meta-security AI framework, cursor for opsec, would be the best, most valuable general purpose AI tool any company could build, imo. Everything from journalism to law to coding would immediately benefit, and it'd provide invaluable data for post training, reducing the overall problematic behaviors in the underlying models. Move fast and break things is a lot more valuable if you have a red team mechanism that scales with the product. Who knows how many facepalm level failures like this are out there?

croes 11mo ago
> I think this class of problems can be protected against. Of course, it’s called proper software development
- jeffbee 11mo ago
  The techniques for non-disclosure of confidential materials processed by multi-tenant services are obvious, well-known, and practiced by very few.
- venturecruelty 11mo ago
  And jail time for executives who are responsible for data leaks.
  - marginalx 11mo ago
    Are you saying executives cannot make mistakes ever (ask because you didn't qualify your statement)?
    - venturecruelty 11mo ago
      I'm saying that if executives get praise and bonuses for when good things happen, they should also have negative consequences when bad things happen. Litigate that further how you wish.
    - dghlsakjg 11mo ago
      The key word in that is "responsible". The legal world has plenty of ways for determining if you are legally responsible for the outcome of an event. Right now the standard is civil punishments for provable negligence. It sounds like GP is proposing a framework where we tighten up the definition of negligence, and add criminal penalties in addition to civil ones.
    - pbhjpbhj 10mo ago
      Are you saying the OP was just a single error, effectively an executives typo.
dghlsakjg 11mo ago
In this case, AI was a red herring. This was just plain terrible web security.

canopi 11mo ago

The first thing that comes to my mind is SOC2 HIPAA and the whole security theater. I am one of the engineers that had to suffer through countless screenshots and forms to get these because they show that you are compliant and safe. While the real impactful things are ignored

latchkey 11mo ago
SemiAnalysis made this a base requirement for being appropriately ranked on their ClusterMAX report, telling me it is akin to FAA certifications, and then getting hacked themselves for not enforcing simple security controls. https://jon4hotaisle.substack.com/i/180360455/anatomy-of-the-hijacking https://jon4hotaisle.substack.com/i/180360455/anatomy-of-the... It is crazy how this gets perpetuated in the industry as actually having security value, when in reality, it is just a pay-to-play checkbox.
chickensong 11mo ago
You have to start somewhere though. Security theater sucks, and it's not like compliance is a silver bullet, but at least it's something. Having been through implementing standards compliance, it did help the company in some areas. Was it perfect? Definitely not. Was it driven by financial goals? Absolutely. It did tighten up some weak spots though. If the options mainly consist of "trust me bro" vs "we can demonstrate that we put in some effort", the latter seems more preferable, even if it's not perfect.

quapster 11mo ago

[flagged]

j45 11mo ago
It's a little hilarious. First, as an organization, do all this cybersecurity theatre, and then create an MCP/LLM wormhole that bypasses it all. All because non-technical folks wave their hands about AI and not understanding the most fundamental reality about LLM software being fundamentally so different than all the software before it that it becomes an unavoidable black hole. I'm also a little pleased I used two space analogies, something I can't expect LLMs to do because they have to go large with their language or go home.
- jimbokun 11mo ago
  My first reaction to the announcement of MCP was that I must be missing something. Surely giving an LLM unlimited access to protected data is going to introduce security holes?
  - j45 10mo ago
    Agree. It’s assuming and estimating it will behave like other software before it when it’s nothing like the software that came before it. LLMs today won’t behave like the software we’re used to where 1+1 will equal 2 every time.
    - jimbokun 10mo ago
      It’s like handing access to your sensitive data to a complete stranger and hoping for the best.
- RansomStark 11mo ago
  Maybe this is the key takeaway of GenAI: that some access to data, even partially hallucinated data, is better than the hoops that the security theatre puts in place that prevents average Joe doing their job. This might just be a golden age for getting access to the data you need for getting the job done. Next security will catch up and there'll be a good balance between access and control. Then, as always security goes to far and nobody can get anything done. It's a tale as old as computer security.

electric_muse 11mo ago

[flagged]

snapcaster 11mo ago
What makes you think that? it would need some prompt engineering if so since ChatGPT won't write like that (bad capitalization, lazy quoting) unless you ask it to
- lazide 11mo ago
  “Chat, write me a blog article that seems like a lazy human who failed English wrote it”?
  - legostormtroopr 11mo ago
    What’s worse being accused of an AI post or being defended because your post is so bad that AI wouldn’t have written it?
  - FrustratedMonky 11mo ago
    Well then that's everything.
simonw 11mo ago
That comment didn't read like AI generated content to me. It made useful points and explained them well. I would not expect even the best of the current batch of LLMs to produce an argument that coherent. This sentence in particular seems outside of what an LLM that was fed the linked article might produce: > What's wild is that nothing here is exotic: subdomain enumeration, unauthenticated API, over-privileged token, minified JS leaking internals.
- samdoesnothing 11mo ago
  It's probably a list of bullet points or disjointed sentences fed to the LLM to clean up. Might be a non-English speaker using it to become fluent. I won't criticize it, but it's clearly LLM generated content.

kylecazar 11mo ago

If they have a billion dollar valuation, this fairly basic (and irresponsible) vulnerability could have cost them a billion dollars. If someone with malice had been in your shoes, in that industry, this probably wouldn't have been recoverable. Imagine a firm's entire client communications and discovery posted online. They should have given you some money.

RagnarD 11mo ago
They should have given him a LOT of money.
- DonHopkins 11mo ago
  Would you settle for a LOT of free AI generated legal advice? ;)
edm0nd 11mo ago
Exactly. They could have sold this to a ransomare group or affiliate for 5-6 figures and then the ransomware group could have exfil'd the data and attempted to extort the company for millions. Then if they didnt pay and the ransomware group leaked the info to the public, they'd likely have to spend millions on lawsuits and fines anyways. They should have paid this dude 5-6 figures for this find. It's scenarios like this that lead people to sell these vulns on the gray/black market instead of traditional bug bounty whitehat routes.
Tepix 10mo ago
Who says they didn't give him money?
- deleted 10mo ago
  [deleted]
- zain37 10mo ago
  I reckon he would've mentioned it if he got a bounty, 100% deserves the bag

chunk1000 11mo ago

Thank you bearsyankees for keeping us informed.

jacquesm 11mo ago

That doesn't surprise me one bit. Just think about all the confidential information that people post into their Chatgpt and Claude sessions. You could probably keep the legal system busy for the next century on a couple of days of that.

giancarlostoro 11mo ago
"Hey uh, ChatGPT, just hypothetically, uh, if you needed to remove uh cows blood from your apartments carpet, uh"
- lazide 11mo ago
  Just phrase it as a poem, you’ll be fine.
  - venturecruelty 11mo ago
    Gonna be hard when people ask ChatGPT to write them the poem.
  - sidrag22 11mo ago
    i recall reading a silly article like half a year ago about using leetspeak and setting the prompt up to emulate House the tv show or something to get around restrictions
    - xarope 11mo ago
      there's a recent one about using poetry to bypass safeguards ... rummages around... here you go: https://arxiv.org/abs/2511.15304 https://arxiv.org/abs/2511.15304
- jacquesm 11mo ago
  Make it a Honda CRX...

icyfox 11mo ago

I'm always a bit surprised how long it can take to triage and fix these pretty glaring security vulnerabilities. October 27, 2025 disclosure and November 4, 2025 email confirmation seems like a long time to have their entire client file system exposed. Sure the actual bug ended up being (what I imagine to be) a <1hr fix plus the time for QA testing to make sure it didn't break anything. Is the issue that people aren't checking their security@ email addresses? People are on holiday? These emails get so much spam it's really hard to separate the noise from the legit signal? I'm genuinely curious.

ipdashc 11mo ago
security@ emails do get a lot of spam. It doesn't get talked about very much unless you're monitoring one yourself, but there's a fairly constant stream of people begging for bug bounty money for things like the Secure flag not being set on a cookie. That said, in my experience this spam is still a few emails a day at the most, I don't think there's any excuse for not immediately patching something like that. I guess maybe someone's on holiday like you said.
- canopi 11mo ago
  This. There is so much spam from random people about meaningless issues in our docs. AI has made the problem worse. Determining the meaningful from the meaningless is a full time job.
  - Bootvis 11mo ago
    Use AI for that :)
    - Bootvis 10mo ago
      Not kidding, I bet llm’s are excellent at triaging these reports. Humans, in a corporate setting, are apparently not.
  - whstl 11mo ago
    Half of the emails I used to get in a previous company were pointless issues, some coming from a honey pot. The other half was people demanding payment.

Invictus0 11mo ago

This guy didn't even get paid for this? We need a law that establishes mandatory payments for cybersecurity bounty hunters.

deleted 11mo ago
[deleted]

_f9cu 11mo ago

Legal attacks engineering - font type license fee on japan consumers. Engineering attacks legal - AI info dump in above post. How does above sound like and what kind of professional write like that?

sys32768 11mo ago

I work for a finance firm and everyone is wondering why we can store reams of client data with SaaS Company X, but not upload a trust document or tax return to AI SaaS Company Y. My argument is we're in the Wild West with AI and this stuff is being built so fast with so many evolving tools that corners are being cut even when they don't realize it. This article demonstrates that, but it does sort of beg the question as to why not trust one vs the other when they both promise the same safeguards.

layer8 11mo ago
The question is what reason did you have to trust SaaS Company X in the first place?
- sys32768 11mo ago
  Because it's the Cloud and we're told the cloud is better and more secure. In truth the company forced our hand by pricing us out of the on-premise solution and will do that again with the other on-premise we use, which is set to sunset in five years or so.
  - ansgri 10mo ago
    Probably has more to do with responsibility outsourcing: if SaaS has security breach AND they tell in the contract that they’re secure, then you’re not responsible. Sure, there may be reputational damage for you, but it’s a gamble with good odds in most cases. Storing lots of legal data doesn’t seem to be one of these cases though.
    - bonesss 10mo ago
      I see profits and outsourcing. Selling an on-premise service requires customer support, engineering, and duplication of effort if you’re pushing to the cloud as well. Then you get the temptations and lock in of cloud-only tooling and an army of certified consultant drones whose resumes really really need time on AWS-doc-solution-2035, so the on premise becomes a constant weight on management. SaaS and the cloud is great for some things some of the time, but often you’re just staring at the marketing playbook of MS or Amazon come to life like a golem.

lupire 11mo ago

Who is Margolis, and are they happy that OP publicly announced accessing all their confidential files? Clever work by OP. Surely there is automatic prober tool that already hacked this product?

dghlsakjg 11mo ago
> Who is Margolis, and are they happy that OP publicly announced accessing all their confidential files? Google tells me they are a NY law firm specializing in Real Estate and Immigration law. There are other firms with Margolis in the name too. Kinda doesn't matter; see below. I doubt that they are thrilled to have their name involved in this, but that is covered by the US constitution's protections on free press.

richwater 11mo ago

Of course there will be no accountability or punishment.

valbaca 11mo ago

Given the absurd amount startups I see lately that have the words "healthcare" and "AI", I'm actually incredibly concerned that in just a couple of months we're going to have an multiple, enormous HIPAA-data disasters Just search "healthcare" in https://news.ycombinator.com/item?id=46108941 https://news.ycombinator.com/item?id=46108941

mattfrommars 11mo ago

This might be off topic since we are in topic of AI tool and on HackerNews. I've been pondering a long time how does one build a startup company in domain they are not familiar with but ... Just have this urge to 'crave a pie' in this space. For the longest time, I had this dream of starting or building a 'AI Legal Tech Company' -- big issue is, I don't work in legal space at all. I did some cold reach on lawfirm related forums which did not take any traction. I later searched around and came across the term, 'case management software'. From what I know, this is what Cilo fundamentally is and make millions if not billion. This was close to two years or 1.5 years ago and since then, I stopped thinking about it because of this understanding or belief I have, "how can I do a startup in legal when I don't work in this domain" But when I look around, I have seen people who start companies in totally unrelated industry. From starting a 'dental tech's company to, if I'm not mistaken, the founder of hugging face doesn't seem to have PHD in AI/ML and yet founded HuggingFace. Given all said, how does one start a company in unrelated domain? Say I want to start another case management system or attempt to clone FileVine, do I first read up what case management software is or do I cold reach to potential lawfirm who would partner up to built a SAAS from scratch? Other school of thought goes like, "find customer before you have a product to validate what you want to build", how does this realistically work? Apologies for the scattered thoughts...

jimbokun 11mo ago
One approach is to partner with someone who is an expert in that space.
strgcmc 11mo ago
I think it comes down to, having some insight about the customer need and how you would solve it. Having prior experience in the same domain is helpful but is neither a guarantee nor a blocker, towards having a customer insight (lots of people might work in a domain but have no idea how to improve it; alternatively an outsider might see something that the "domain experts" have been overlooking). I just randomly happened to read about the story of, some surgeons asking a Formula 1 team to help improve its surgical processes, with spectacular results in the long term... The F1 team had zero medical background, but they assessed the surgical processes and found huge issues with communication and lack of clarity, people reaching over each other to get to tools, or too many people jumping to fix something like a hose coming loose (when you just need 1 person to do that 1 thing). F1 teams were very good at designing hyper efficient and reliable processes to get complex pit stops done extremely quickly, and the surgeons benefitted a lot from those process engineering insights, even though it had nothing specifically to do with medical/surgical domain knowledge. Reference: https://www.thetimes.com/sport/formula-one/article/professor-martin-elliott-interview-surgeon-f1-pitstops-babies-ferrari-j3sbkjssk https://www.thetimes.com/sport/formula-one/article/professor... Anyways, back to your main question -- I find that it helps to start small... Are you someone who is good at using analogies to explain concepts in one domain, to a layperson outside that domain? Or even better, to use analogies that would help a domain expert from domain A, to instantly recognize an analogous situation or opportunity in domain B (of which they are not an expert)? I personally have found a lot of benefit, from both being naturally curious about learning/teaching through analogies, finding the act of making analogies to be a fun hobby just because, and also honing it professionally to help me be useful in cross-domain contexts. I think you don't need to blow this up in your head as some big grand mystery with some big secret cheat code to unlock how to be a founder in a domain you're not familiar with -- I think you can start very small, and just practice making analogies with your friends or peers, see if you can find fun ways of explaining things across domains with them (either you explain to them with an analogy, or they explain something to you and you try to analogize it from your POV).

fallinditch 11mo ago

> ... after looking through minified code, which SUCKS to do ... AI tends to be good at un-minifying code.

a_victorp 11mo ago
Legit question: when working on finding security issues, are there any guidelines on what you can send to LLMs/AI?
- fallinditch 11mo ago
  I got downvoted, so maybe that means someone thinks un-minifying code is not advised for dealing with security issues? But on reflection surely you can just use the 'format code' command in the ide? I am no expert but surely it's ok to use AI to help track down and identify security issues with the usual caveats of 'don't believe it blindly, do your double checking and risk assessing.'
- CER10TY 11mo ago
  Personally, I'd just use common sense and good judgment. At the end of the day, would you want someone to hand your address, and other private data to OpenAI just like that? Probably not. So don't paste customer data into it if you can avoid it. On the other hand, minified code is literally published by the company. Everyone can see it and do with it as they please. So handing that over to an AI to un-minify is not really your problem, since you're not the developer working on the tool internally.
nodesocket 11mo ago
Doesn't Chrome Developer tools automatically un-minify?

yieldcrv 11mo ago

I've worked in several "agentic" roles this year alone (I'm very poachable lol) and otherwise well structured engineering orgs have lost their goddamn minds with move fast and break things because they're worried that OpenAI/Google/Meta/Amazon/Anthropic will release the tool they're working on tomorrow literally all of them are like this

trollbridge 11mo ago
Old school blue chip type of companies are like this too. They’ve thrown all the process and caution they used to have to the wind so that they can… apply AI to their IT org which isn’t even their core business?

2ndatblackrock 11mo ago

now that's just great hacking

corry 11mo ago

"Companies often have a demo environment that is open" - huh? And... Margolis allowed this open demo environment to connect to their ENTIRE Box drive of millions of super sensitive documents? HUH???! Before you get to the terrible security practices of the vendor, you have to place a massive amount of blame on the IT team of Margolis for allowing the above. No amount of AI hype excuses that kind of professional misjudgement.

me_again 11mo ago
I don't think we have enough information to conclude exactly what happened. But my read is the researcher was looking for demo.filevine.com and found margolis.filevine.com instead. The implication is that many other customers may have been vulnerable in the same way.
- corry 10mo ago
  Ah, I see now that I read too quickly - the "open demo environment" was clearly referencing the idea that the vendor (Filevine) would have a live demo, NOT that each client wanted an open playground demo account that is linked to a subset of their data (which would be utterly insane).

etamponi 11mo ago

I don't disagree with the sentiment. But let's also be honest. There is a lot of improvement to be made in security software, in terms of ease of use and overcomplicating things. I worked at Google and then at Meta. Man, the amount of "nonsense" of the ACL system was insane. I write nonsense in quotes because for sure from a security point of view it all made a lot of sense. But there is exactly zero chance that such a system can be used in a less technical company. It took me 4 years to understand how it worked... So I'll take this as another data point to create a startup that simplifies security... Seems a lot more complicated than AI

deep_thinker26 11mo ago

It's so great that they allowed him to publish a technical blog post. I once discovered a big vulnerability in a listed consumer tech company -- exposing users' private messages and also allowing to impersonate any user. The company didn't allow me to write a public blogpost.

qmr 11mo ago
"Allow"? Go on write your blog post. Don't let your dreams be dreams.
- bigmadshoe 11mo ago
  Presumably they were paid for finding the bug and inn accepting relinquished their right to blog about it.
  - hsbauauvhabzb 11mo ago
    No, you relinquish the right when you agree to their TOS irrespective of if they pay you.
    - amackera 11mo ago
      TOS != law They will stop letting you use the service. That's the recourse for breaking the TOS.
      - advisedwang 11mo ago
        Up until Van Buren v. United States in 2020, ToS violations were sometimes prosecuted as unauthorized access under the CFAA. I suspect there are other jurisdictions that still do the equivalent to that.
      - hsbauauvhabzb 11mo ago
        I don’t want to pay for a lawyer to argue that for me. != law does not equate to ‘won’t come with a cost’. I say this as someone threatened by a billion dollar company for this very thing.

hbarka 11mo ago

> November 20, 2025: I followed up to confirm the patch was in place from my end, and informed them of my intention to write a technical blog post. Can that company tell you to cease and desist? How does the law work?

me_again 11mo ago
Lawyers can and will send cease and desist letters to people whether or not there is any legal basis for it. Often the threat of a lawsuit, even a meritless one, is enough to keep people quiet.
dghlsakjg 11mo ago
FYI, a "cease and desist" carries the same legal weight as me sending a one-liner saying "Knock it off". They are strongly worded requests from a legal point of view. The only real message they send is that the sender is serious enough about the issue to have involved a lawyer, unless of course you write it yourself, which is something that literally anyone can do. If you want to actually force an action, you need a court order of some type. NB for the actual lawyers: I'm oversimplifying, since they can be used in court to prove that you tried to get the other party to stop, and tried to resolve the issue outside of court.

stanfordkid 11mo ago

I mean... in what world would you send a customers private root key to a web browsing client. Like even if the user was authenticated why would they need this? This sort of secret shouldn't even be in an environment variable or database but stored with encryption at rest. There could easily have been a proxy service between client and box if the purpose is to search or download files. It's very bad, even for a prototype... this researcher deserves a bounty!

bzmrgonz 11mo ago

My thing is, even ingesting the BOK should have been done in phases, to avoid having all your virtual eggs in one basket or nest at any ONE time. Staggering tokens to these compartments would not have cost them anything at all . I always say, whatever convenience you enjoy yourself, will be highly appreciated by bad actors... WHEN, not if.. they get thru.

deleted 11mo ago

[deleted]

badbird33 11mo ago

You'd think with a $1B valuation they could afford a pentest

magnetowasright 11mo ago

I am at a loss for words. This wasn't a sophisticated attack. I'd love to know who filevine uses for penetration testing (which they do, according to their website) because holy shit, how do you miss this? I mean, they list their bug bounty program under a pentesting heading, so I guess it's just nice internet people. It's inexcusable.

rashidujang 11mo ago
This was my impression after reading the article too. I have no doubt that the team at Filevine attempted to secure their systems and have probably thwarted other attackers, but got their foot stuck in what is an unsophisticated attack. It only takes one chain vulnerability to bring down the site. Security reminds me of the Anna Karenina principle: All happy families are alike; each unhappy family is unhappy in its own way.
GJim 10mo ago
> I am at a loss for words. This wasn't a sophisticated attack. To be fair, data security breaches seldom are.

ethin 11mo ago

I'll be honest... I'm not at all surprised that this happened. Purely because it seems like everyone who wants to implement AI just forgot all of the institutional knowledge that cybersecurity has acquired over the last 30-40 years. When you "forget" all of that because you want to rush out something really fast, well, you know what they say: play stupid games, win stupid prizes and all that.

culanuchachamim 11mo ago

-The Filevine team was responsive, professional, and took the findings seriously throughout the disclosure process. They acknowledged the severity, worked to remediate the issues, allowed responsible disclosure, and maintained clear communication. This is another great example of how organizations should handle security disclosures. In the same tenure I think that a professional etical hacker or a curious fellow that is poking around with no harm intent, shouldn't disclose the name of the company that had a security issue if they resolve it professionally. You can write the same blog post without mentioning that it was Filevine. If they didn't take care of the incident that's a different story...

CBMPET2001 11mo ago
Eh, with something this horrendously egregious I think their customers have a right to know how carelessly their data was handled, regardless of the remediation steps taken after disclosure; that aside, who knows how many other AI SaaS vendors might stumble across this article and realize they've made a similarly boneheaded error, and save both themselves and their customers a huge amount of pain . . .
evan_a_a 11mo ago
This is a very standard part of responsible disclosure. Hacker finds bugs -> discloses them to the vendor -> (hopefully) the vendor communicates with them and remediates -> both sides publish the technical details. It also helps to demonstrate to the rest of the security world which companies will take reports seriously and which ones won’t, which is very useful information to have.
deelowe 11mo ago
That's not how ethical disclosure works. Both parties should publish and we, the wider tech industry should see this as a good thing both for the hacker and the company that worked with them.
manbash 11mo ago
How else can you take responsibility if you don't make it public? You can't have integrity if you hide away your faults.

theoldgreybeard 11mo ago

The bigwigs at my company want to build out a document management suite. After talking to VP of technology about requirements I ask about security as well as what the regulatory requirements are and all I get is a blank stare. I used to think developers had to be supremely incompetent to end up with vulnerabilities like this. But now I understand it’s not the developers who are incompetent…

eru 11mo ago
There's enough incompetence at all levels to go around.
- theoldgreybeard 11mo ago
  Maybe I have just been lucky, but I have not had the displeasure of working with people either tha incompetent or willfully ignorant yet.
  - eru 11mo ago
    Oh, I should have been more careful in my formulation: There are organisations that are generally competent, and there are places that are less competent. It's not all that uncommon for the whole organisation to be generally incompetent. The saddest places (for me) are those where almost every individual you talk to seems generally competent, but judging by their output the company might as well be stuffed by idiots. Something in the way they are organised suppresses the competence. (I worked at one such company.) > Maybe I have just been lucky, but I have not had the displeasure of working with people either tha incompetent or willfully ignorant yet. It's very important before you start any new job to suss out how competent people and the organisation are. Ideally, you probably want to work for a competent company. But at least you want to know what you are getting into. There's a bit of luck involved, if you go in blindly, but you can also use skill and elbow-grease to investigate.
    - chii 11mo ago
      > suss out how competent people and the organisation are. how does one do this, without first having the job and being embedded in there? From the outside, it's near impossible to see these details imho.

habosa 11mo ago

They took a month to fix this? That’s beyond inexcusable. I can’t imagine how any customer could justify working with them going forward. Also … shows you what a SOC 2 audit is worth: https://www.filevine.com/news/filevine-proves-industry-leadership-in-data-security-with-soc-2-and-soc-3-compliance/ https://www.filevine.com/news/filevine-proves-industry-leade... Even the most basic pentest would have caught this.

aitchnyu 11mo ago
Is there any stricter standard? Should one strive for PCI-DSS even if they are a regular SaaS?
- eru 10mo ago
  Whatever Google does internally would be a much stricter standard, but I'm not sure they've written it up for outsiders to use, alas.
eru 11mo ago
It looks like SOC 2 (and the other SOCs) where developed by accountants? I wouldn't expect them to find any computer problems either to be honest.
- anticensor 10mo ago
  There are only 3 books of SOC: SOC I, SOC II Part 1, SOC II Part II.
jonny_eh 11mo ago
Where did it say that they took a month to fix? The hacker just checked in 2 weeks later and it was fixed by that point.
- theodorejb 11mo ago
  According to the timeline it took more than a week just for Filevine to respond saying they would review and fix the vulnerability. It was 24 days after initial disclosure when he confirmed the fix was in place.

satya71 11mo ago

I had a look at some of FileVine's output, and I can say I'm not surprised. This is not an organization that prizes engineering at all.

BrenBarn 11mo ago

I'm less and less sure that when a billion-dollar company screws up this bad, the right thing to do is privately disclose it and let them fix it. This kind of thing just allows companies to go on taking people's money without facing the consequences of their mistakes.

barbazoo 11mo ago
Does a disclosure like this absolve them of any responsibility? They still violated whatever user privacy act.
- mvkel 11mo ago
  It does. Most privacy laws are based on time-from-discovery. If they immediately sprung into action at the moment they were informed and remediated the issue, they're in compliance.
  - mattmaroon 11mo ago
    So is that true if they find out when the public does too? It seems that disclosing it privately has some upside (protecting the users) and no downside.
    - mvkel 11mo ago
      That depends more on what the Privacy Policy is of the service, which you agree to when you sign up and use it
  - BrenBarn 10mo ago
    Right, that's the problem. There need to be standards that govern what can ever be released to customers/the public in the first place. When violations of those are discovered, the penalties should be based on time from release, so the longer it was out in the wild, the greater the penalty.
    - 10mo ago

0xbadcafebee 11mo ago

So, 1) a public service, 2) with no authentication, 3) and no encryption? (http only??), 4) sent every single response with a token, 5) giving full admin access to every client's legal documents. This is like a law firm with an open back door, open back window, and all the confidential legal papers sprawled out on the floor. Imagine the potential impact. You're a single mother, fighting for custody of your kids. Your lawyer has some documentation of something that happened to you, that wasn't your fault, but would look bad if brought up in court. Suddenly you receive a phone call - it's a mysterious voice, demanding $10,000 or they will send the documents to the opposition. Neither of them knows each other; someone just found a trove of documents in an open back door and wanted to make a quick buck. This is exactly what a software building code would address (if we had one!). Just like you can't open a new storefront in a new building without it being inspected, you should not be able to process millions of sensitive files without having your software's building inspected. The safety and privacy of all of us shouldn't be optional.

Fnoord 11mo ago
http-only makes it also very easy to sniff for LE if they decide to. This allows them to get knowledge about cases. Like, they could be scanning it with their own AI tool for all we know. In a free country with proper LE, this would neither be legal nor happening. But I am not sure the USA is remaining one, given the leader is a convicted felon with very dubious moral standards. The problem here however is that they get away with their sloppiness as long as the security researcher who found this is a whitehat, and the regular news won't pick it up. Once regular media pick this news up (and the local ones should), their name is tarnished and they may regret their sloppiness. Which is a good way to ensure they won't make the same mistake. After all, money talks.
- zwnow 10mo ago
  All the big tech companies are in the news every week. Everybody knows how bad they are. Their names are tarnished and yet everyone is still using their junk and they face zero repercussions when fucking up. I dont think things in the media would do any harm.

kitschman 11mo ago

[dead]

1vuio0pswjnm7 11mo ago

"(after looking through minified code, which SUCKS to do)" Would there be a "pretty printer" or some other "unminifier" for this task If not, then is minification effectively a form of obfuscation

gu5 11mo ago
https://webcrack.netlify.app/ https://webcrack.netlify.app/ was made exactly for this, it's a really useful tool

densone 11mo ago

If I had a dollar for every time I saw something like this or for every time someone exposed a .git with a .env full of secrets

keernan 10mo ago

Attorneys are ethically obligated to follow very stringent rules to protect their client's confidential information. Having been a practicing litigator for 40+ years, I can confidently state I came across very few attorneys who truly understood their obligations. Things were easier when I first began practicing in the 1970s. There weren't too many ways confidential materials in our files could be compromised. Leaving my open file spread out on the conference room table while I went to lunch while attorneys arriving for a deposition on my partner's case were one by one seated into the conference room. That's the kind of thing we had to keep an eye on. But things soon got complicated. Computers. Digital copies of files that didn't disappear into an external site for storage like physical files. Then email. What were our obligations to know what could - and could not - be intercepted while email traveled the internet. Then most dangerous of all. Digital storage that was outside our physical domain. How could we now know if the cloud vendor had access to our confidential data? Where were the backups stored? How exactly was the data securely compartmentalized by a cloud vendor? Did we need our own IT experts to control the data located on the external cloud? What did the contracts with the cloud vendor say about the fact we were a law firm and that we, as the lawyers responsible for our clients confidential information, needed to know that they - the cloud vendor - understood the legal obligations and that they - the cloud vendor - would hire lawyers to oversee the manner in which the cloud vendor blocked all access to the legal data located on their own servers. And so on and so forth. I'm no longer in active practice but these issues were a big part of my practice my last few years at a Fortune 500 insurance company that used in-house attorneys nationwide to represent insureds in litigation - and the corporation was in engaged in signing onto a cloud service to hold all of the corporate data - including the legal departments across all 50 states. It was a nightmare. I'm confident it still is.

tonyhart7 10mo ago

so the future is software engineer cosplaying an cybersecurity so we can implement BASIC security????

hansmayer 10mo ago

For all the talk in the blog of how "super-professional" their team was (probably just a courtesy on the side of the author, I don´t think he believes his own words either)... I have noticed using AI to produce some kind of API -OR- use a 3rd party point with integration into frontend, is almost guaranteed to produce code in which the frontend either exposes the API secrets directly in the frontend code (literally injecting it into a variable as string), or if you ask it for authentication, it will build some half-built lazy solution which makes no sense. So I imagine their "super-professional" team built this with AI, blindly trusting, probably even allowing it to commit and merge changes itself because if you are not merging 10K LoC a day with all this "great" technology, what are you even doing, right? It is not super-professional to work effectively with a blindfold on, I´d argue.

MangoToupe 10mo ago

> countless data protected by HIPAA People should really look this law up before they reference it

xp84 10mo ago

Through that API, the frontend is handed a high-privilege API key for Box. Not only was that a huge blunder on the backend, but it reveals what passes for architecture these days. Should our application's backend speak to the super sensitive file store? No, we should hand over the keys to that to the React app, because it literally did not occur to them that there's anything that physically could be driven by the frontend that shouldn't be. My apologies to the frontend engineers out there who know what they're doing.

nstj 10mo ago

The post is a disclosure about a vuln which seem to have anything to do with AI.

Fokamul 10mo ago

"Coding is done", Adam Wolff, Anthropic

aperture147 10mo ago

Hey I think I've just found a new marketing stunt for a new vibe-coding platform: "Worried your vibe-coded app is about to be broadcast on the internet’s biggest billboard? Chill. ACME AI now wraps it in “NSA-grade” security armor." I've never thought that there will be multiple billion-dollar-AI-features that fixes all the monkey patching problems that no one saw them coming from the older billion-dollar-AI-features that fixes all the monkey patching problems that no one saw them coming from...

canto 10mo ago

How is following a http request and guessing some variables a "reverse" engineering now?

testemailfordg2 10mo ago

When liability of a corporation and its owners is limited, does it benefit their business to be dilligent in every step or have a mentality of move fast and break things?

6thbit 10mo ago

Man. Hopefully their remediation steps included a full audit of their Box's account. One could only imagine that if OP wasn't the first to discover it, people could've generated tons of shared links for all kinds of folders, for instance, which would remain active even if they invalidated the API token.

larrysanchez77 10mo ago

[dead]