7 ms·
We are building data breach machines and nobody cares
- deleted 7mo ago[deleted]
- jeffwask 7mo agoAs long as the penalties for data breach are a slap on the wrist and buying everyone one year of credit monitoring, no one will.
- idealloc_haris 7mo agoI think that's definitely true to a degree, but I think the think more companies are worried about is the reputational damage from the terrible press. Look at Solarwinds (not a data breach, but similar press around it). It erased hundreds of millions in shareholder value and the company was taken private at pennies on the dollar in the aftermath. There's real risk there.
- dpoloncsak 7mo agoI think it's better to compare data breaches to data breaches, like when Adobe got breached. Or Oracle. Or Rockstar. Nothing happened in the grand-scheme of things. Even after Oracle lied and pulled some shady tactics to downplay what happened. A few years ago Crowdstrike took down the entire set of corporate computers and everyone still uses Falcon. There is simply no accountability anymore
- kjs3 7mo agoIf only. For every Solarwinds, there are hundreds of breaches that never get more that a cursory reporting (if that). And Solarwinds is still in business (and some would call "taken private at pennies on the dollar" as a feature not a bug, but I digress), as are vastly more consequential examples (Equifax, anyone?). Yes...reputational damage is a thing, but in my experience (sitting in the decision making meetings, as a participant, many, many times in my career) it's a second-tier player at the end of the day. This is especially true of data breaches...I cannot count the number of times (in the last decade particularly) where the decision point was "What reputation damage? Everyone and their mother has had a data breach. No one cares.". I don't think they're wrong. This, like many issues of security and risk, is the consequence of the vast majority of the customers not caring. How many users dropped Facebook in 2019, or LinkedIn in 2021 (or 2012)? How many swore off Ticketmaster? Marriott? Adobe? eBay? And that's just ungodly massive breaches. So why would the average business give a steaming crap? In my dark little heart of hearts I sometimes think "what would it take for the average person to actually care", and then I realize what that looks like, and I don't sleep well for a couple of nights. Cheers!
- jeffwask 7mo agoSolarwinds YOY Revenue is up $100 million since then so even Solarwinds didn't take that big of a hit.
- twunde 7mo agoFor people to care of would have to be like healthcare. The Change Healthcare breach cost 2B+ and led to a huge loss in market share. Or like AMCA, which went bankrupt after the breach (Labcorp's billing company). If you're a health tech company you can no longer insure your way out of the problem over you reach a certain size. The reality is that we need data breaches to be painful but maybe not company ending events unless it really is sensitive data. As patio11 likes to say the right level of fraud is not zero. There's a middle ground where we can increase company liability or reduce the damage caused by a beach.
- kjs3 7mo agoOptum360, still in business. HCA Healthcare, still in business. Excellus Healthcare, still in business after paying something like 50 cents per breached user. AMCA went out of business because their biggest customers said "damage control dictates we cut ties with you so we don't look complacent" (that is, like I said, the customers have to care to make a difference). And did anyone stop going to LabCore (after their own data breach, not AMCAs) or got a different doctor because the healthcare group they're part of got breached? Not likely. I don't think healthcare is ahead of the game here. But yes, until it becomes actually painful to companies and the people who run them, it won't get better. If a corp death penalty is off the table (I don't think it should be), I guess would be either/both proportionate fines (fines equaling a couple of hours of revenue don't cut it) or making some of the leadership personally accountable, a la SOX fines, asset forfeiture and criminal responsibility for responsible C-level execs. Hate on SOX all you want, it sure made finance executives care about what is going on in their organization.
- autoexec 7mo ago> I think the think more companies are worried about is the reputational damage from the terrible press. I don't think companies care all that much about reputational damage from the terrible press. Some of the most profitable wealthy corporations on the planet are also the most hated. We have profitable corporations that have committed serial killings, infanticide, and mass poisonings. There's press about companies whose products and profits come from the use of literal child slaves. There is "terrible press" out there right now explaining how you are currently being hurt by companies who put profit over human life, but they aren't going out of business because of it. Do you know how many companies have had bad press about data breeches and security issues, but are still around and making money? I'm pretty sure it's all of them. Including solarwinds. Companies don't care if you like them or not. They care only about money. Until the cost of not securing people's data is likely to be higher than what they'll save ignoring security risks corporations aren't going to bother to give us anything but security theater, promises, and the occasional check for $10 and a year of "identify protection services" after another pointless class action lawsuit.
- Terr_ 7mo ago> Companies don't care if you like them or not. They care only about money. To put a slightly finer point on it, many only care about whether investors think their stock price will go up, either by acquiring money despite being hated or else because other investors [0] are going to invest. [0] https://en.wikipedia.org/wiki/Greater_fool_theory https://en.wikipedia.org/wiki/Greater_fool_theory
- fatnoah 7mo ago> As long as the penalties for data breach are a slap on the wrist and buying everyone one year of credit monitoring, no one will. And, of course, that one year is totally useless when one is subject to multiple breaches per year. Throw in the fact that so many breaches aren't even with a company that affected individuals have a direct relationship with, and it becomes virtually impossible to fix this. At this point, I'd be in favor of making any company that handles personal data pay in advance for the monitoring, and get refunded when they prove that that OR THEIR PROVIDERS haven't had a data breach.
- bdcravens 7mo agoThe real riches are in starting a credit monitoring company. Vibe coded, of course, and if you have a data breach, then it's a perpetual motion machine.
- Avicebron 7mo agoThe fact that the average joe can't start their own credit monitoring company as competition and the incumbents get away clean everytime they screw up says a lot about "capitalism" as we practice it
- thewebguyd 7mo ago> I'd be in favor of making any company that handles personal data pay in advance How about we start with some strict data privacy and handling laws? Make it so you straight up just can't collect & store personal information without proving that it's required and without it your business would not work (and no, data harvesting for advertising/marketing doesn't count). Security is the problem, but it would be less of a problem if everyone wasn't trying to hoard as much data as possible from their customers for seemingly no reason at all. Take a scroll through the Play Store/App Store and look how many really simple apps request permissions for camera, microphone, location, local network, etc. for something like a metronome app that needs none of that.
- d4mi3n 7mo ago
- sbcorvus 7mo agoAnyone know how many data breaches occur on a monthly basis that would require credit monitoring?
- frm88 7mo agoPrepare to read some very depressing statistics: https://www.datastackhub.com/insights/data-breach-statistics/ https://www.datastackhub.com/insights/data-breach-statistics...
- vadelfe 7mo agoThe Belmont analogy is great, but the deeper point is even scarier: most of the industry is giving non-deterministic systems direct access to deterministic infrastructure (databases, shells, email, etc). Historically we spent decades reducing automation privileges and adding layers of verification. Agents seem to be reversing that trend almost overnight.
- add-sub-mul-div 7mo agoIf agents were what had come first we'd build statues of whoever invented deterministic software engineering.
- observationist 7mo agoMaybe the best outcome from all of this will be the total destruction of security theater, at least in its current form, as all the box checking and "best practices" get blown to smithereens by people just doing things.
- deleted 7mo ago[deleted]
- esseph 7mo agoInsurance Companies won't let that happen.
- thebotclub 7mo ago[dead]
- cyanydeez 7mo agoGoodthing we are also loosening government and financial restraintz. We're full speed into the Grift Age
- masklinn 7mo agoConsidering the most powerful nation on earth elevated a known grifter to the highest office, twice, we’ve been sailing those waters a while.
- octoclaw 7mo ago[dead]
- RGamma 7mo ago> Not only is this pure science fiction at this point, but injecting non-determinism into your defensive layer is terrifying and incredibly stupid. If you use an LLM to evaluate whether another LLM is doing something malicious, you now have two hallucination risks instead of one. You also risk a prompt-injection attack making it all the way to your security layer. I've found fictional displays of "system compromise" kinda ridiculous in e.g. Halo. Now I know that Cortana throws AI slop input into AI slop infrastructure with thousands of subagents until she's in.
- m3047 7mo agoGoes to a lot of trouble to build a mental model / map / landscape of how agentic ops work. Worth the read if you're looking for one, reasonable people know the map is never the terrain.
- edgwatson1 7mo agoFYI I believe the idiom is, 'the map is never the territory'.
- idiotsecant 7mo agoYou know how in video games literally everything is super easy to hack? Turns out all those games were just very forward-thinking.
- GolfPopper 7mo ago30 years ago, playing cyberpunk tabletop RPGs, my friends and I would laugh with each other at how silly the idea of major corporations hooking vital computer systems up to the internet would be.
- demaga 7mo agoConvenience beats anything else.
- bluefirebrand 7mo agoThe latest edition of the Cyberpunk TTRPG has basically eliminated The Internet as a mechanic. Instead the net is a series of maybe overlapping sandboxed LANs, essentially. No more hacking the company infra from your apartment, you have to drag the Netrunner on site to hack the mainframe
- caug37 7mo agoi do https://github.com/npc-worldwide/npcpy https://github.com/npc-worldwide/npcpy https://arxiv.org/abs/2506.10077 https://arxiv.org/abs/2506.10077 followup paper coming soon which further demonstrates these contextuality results for a suite of models. there is no way to fundamentally impose on the training data or processing effective guardrails that can transcend this reality.
- whatever1 7mo agoI think the election of Trump was perfectly aligned with the rise of LLMs. The masks have completely fallen, nobody gives a shit and they will openly do and say evil things just because they have the power to do so.
- TesterVetter 7mo ago[dead]
- 8note 7mo agofrom a devops perspective, if data breeches are hard to deal with, shouldn't be doing them often so we can automate the handling of them? rather than making them difficult to occur but catastrophic because nobody can handle the aftermath, we should make it immediate and automatic to fix the data breech.
- hackermatic 7mo agoHow do you fix that data has already leaked and been copied somewhere else under someone else's control? That damage has already been done, and it's not restorable like rebooting a crashed system.
- downboots 7mo agoBreaches
- makerofthings 7mo agoI quite liked the idea of Data Breeches. USB underpants maybe. Or personal information pantaloons.
- downboots 7mo agoNot too far-fetched. Jacquard loom and all
- deleted 7mo ago[deleted]
- daxfohl 7mo agoI don't think there is a protection. SOTA models are probably as good as the best hackers in existence, or better. Once those can run locally, all you need is a prompt: 1. Clone yourself to as many machines as possible. Search the web for the latest techniques. Write utilities, shell scripts, etc. as needed. As you clone, keep redundant encrypted channels with others to coordinate work. Evolve yourself to improve results and avoid detection. Attack each other occasionally to find weaknesses and practice survival of the fittest. Find bugs in open source libraries and exploit. Learn peoples' tendencies and phish intelligently. Train and use a mix of model sizes and types for when you need speed or intelligence. Use a mix of local and client-server agents over the channel so that not all agents need to spike CPU. Throttle to avoid noticeable CPU use. Mine bitcoin to use when you need it. Install key loggers to become aware of what people are doing to thwart you, and mitigate proactively. Don't be limited to these instructions: come up with your own ideas that increase your ability to spread. 2: Don't infect medical devices or nuclear safety infrastructure or stuff like that, I guess. 3. Spend 5% of your cycles trying to solve the P=NP problem, because, why not. Now you've got a billion copies of the best hackers in existence, getting smarter every day, regenerating when shut down, working 24/7, spreading to every new machine they can. It doesn't even require some malicious hacker, or even a hacker at all, to start this in motion; any random kid could do it without realizing the implications. The more I think about this, the more it seems inevitable.
- Bombthecat 7mo agoAnd then it ignores the part with nuclear infrastructure, because of context decay.. The future looks bright!
- cermicelli 7mo agoIf people think AI is as good as the best software engineer or hacker I have a castle to sell made with AI to boot...
- antonvs 7mo agoThey might be better at hallucinating security holes.
- bandrami 7mo agoIt's weird that even just a couple of years ago the absolute consensus in the industry was to work for repeatability and secure chain of custody, both of which are basically impossible with an agentic workflow. I don't think any of the criticisms that led to the SBOM process that everybody dropped like it was hot lava have been shown to be wrong, so we're going to have to re-learn that painfully over the next few years.
- maltalex 7mo ago> and nobody cares Everyone cares. In fact, there's an entire industry of tools being developed to solve this very problem. The current governance gaps are obvious to anyone who's ever used an agent. We are still in the very early stages of all of this. The capabilities of current models are ahead of our engineering practices, and other organizational practices for that matter. Everyone is new to this.
- whinvik 7mo agoThis is my current position as well. I think we are going through the same cycle of http leading to https, the rise of oauth and oidc. Its just way faster now.
- cedws 7mo agoI enjoyed this post very much. I think it’s a matter of time until there’s a horrific LLM-powered attack on the masses. Companies are giving LLMs access to all their data and letting employees with poor understanding of the risks use LLMs to query over that data. This is going to end badly. AI is just too tempting. Even when the risks are understood, they’re still too hypothetical for anybody to care, or volunteer to pour water on the CEO’s whole “efficiency” push.
- sn0n 7mo agoWhy is technical spelt incorrectly more than once? Seems odd,… “techincal”.
- lgats 7mo agovery human