6 ms·

Will Cyber Security Be Replaced by AI?

• hackermatic 3y ago
  I wasn't expecting much based on the headline, but this is a great roundup of the pros and cons of applying AI to different technical facets of security. What it's missing is that the human factor is what makes the real difference to the security of a product or organization: changing or adapting to user behavior, convincing managers and developers to allocate time to security as well as features, or to alter or even forego a feature that carries more risk than it's worth. In some cases, AI can probably help explain these issues better or analyze factors to help determine how to balance security vs. revenue, but a lot of the work is really in building relationships and trust over time to educate and influence behavior and priorities.
• joejoesvk 3y ago
  of course it for sure maybe.
• beny23 3y ago
  Betteridge’s Law in action ;-)
  • tome 3y ago
    tome's law in action ;-) "In any discussion about an article whose title is a question, Betteridge's law is mentioned with probability 1." https://news.ycombinator.com/item?id=24545238 https://news.ycombinator.com/item?id=24545238
• ofjcihen 3y ago
  Sure, after literally everything else. Security remains top of the pile in terms of (targeted) creativity and foresight, things that separate “just AI” from “does everything imaginable for us” and indeed the threshold for what we can start to consider AGI. Once security is replaced by AI the system is essentially untethered.
  • c0pium 3y ago
    Most security jobs require almost no creativity or thought though, most of them are looking at the outputs of some commercially available AV/EDR/IDS in a commercially available SIEM/case management tool and doing the same thing hundreds of other people have done with those same tools. If you work in research maybe you’re on, but if you work tickets you should learn how to do something else.
    • ofjcihen 3y ago
      No, what you’re describing is what gets filtered to the people in charge of the SOAR who then automate those outcomes. Even with the top tools and detection you still absolutely need knowledgeable people to decide whether an alert is a false or true positive and then you absolutely need humans in the mix to decide what the next steps are. When you say “most security jobs” maybe you mean first tier SOC for a vendor? But I’ve never worked for one and as far as I know they’re universally seen as just above useless by anyone without a financial incentive to employ them.
      • c0pium 3y ago
        So you think that you know a lot about security but not only do you not know the difference between SIEM and SOAR you think that most security jobs aren’t dealing with incidents? Fascinating. It’s not an uncommon problem though I guess, I’ve worked with a lot of people who think they’re doing novel work discovering that you can use regex to find spam or inventing a DGA detector.
        • ofjcihen 3y ago
          I’m not sure why you seem so agitated suddenly. As you know I’m sure, a SOAR is meant for automation. The simple tasks with simple decision making are piped to the SOAR. Additionally there’s a lot of work that goes into the space between “something has alerted” and “this is an incident”. Also, just as an aside, I’m not sure where you’re getting mixed up about SIEM/SOAR?
• lifechoseme123 3y ago
  I hope AI brings us headlines that don't end in a question mark, because it's unoriginal and I'm sick of it?
  • beezlewax 3y ago
    I worked in a newspaper or two back in the day and the sub editors who wrote headlines were banned from putting a question mark in a headline.
• jmclnx 3y ago
  Glad I read the article, I think he is correct. I am sure people will try, but I think it will end being "border security" for Walled Garden Environments, which is were the Internet seems to be heading towards.
• GaggiX 3y ago
  I always disagree when I see something like: "5) Lack of Understanding When ChatGPT generates a human-like response, it's not a result of understanding the content in the way humans do. Instead, it's drawing upon patterns in the data it was trained on. It can mimic intelligence, but it doesn't genuinely comprehend the concepts it's discussing.". If I ask ChatGPT or GPT-4 to solve novel problems using a random library or programming language and it solves them then it is clear to me that it has understood the tasks, indifferently on how you paraphrase it if that's not understanding then we all lack understanding. But anyway, this part of the article feels more like a tangent, hopefully you're not going to use a LLM to analyze incoming traffic (maybe just for moderation).
  • constantly 3y ago
    Honestly I’m not sure I ever understand the content myself and am instead repeating patterns of what has worked in the past applied to new domains.
    • c0pium 3y ago
      You should be aware that not everyone is doing this. Contrary to what you might see on infosec twitter, not everyone is a trench coat covered pile of raccoons.
    • Retric 3y ago
      The issue is apparent when you consider the kind of mistakes LLM’s make. If I ask you to guess a blue whale’s body temperature you’re not going to save 500C because you understand if that were true various other things would be happening. An LLM is more likely to give the correct answer to the blue whale question, but when you greatly penalize wildly incorrect answers they do noticeably worse.
    • themulticaster 3y ago
      Yeah, I think coming up with definitions of "understanding" or "reasoning" that GPT-4 and friends supposedly don't fulfill is moving goalposts. To continue your line of thinking, when we add salt and pepper to a dish we've cooked, are we really doing it because we have a developed a thorough understanding of the human olfactory and gustatory systems [1] system, or because it tasted good previously when applied to similar recipes? And when it comes to understanding basic math, perhaps the patterns are still a tad to complicated for LLMs. Maybe there are too many surprising rules that appear out of nowhere and break the learnt patterns. But children struggle with those rules as well when they first come across them. Think about division through zero - when coming across the rule for the first time, you might wonder why it exists, instead of e.g. defining the result to be infinity. The answer to that question (not just "because that's the way things work!") is not obvious at all, to be honest I wouldn't be confident enough to attempt giving an explanation here. [1] I have to admit I had to look gustatory system up: it's the biological term for the system behind the sense of taste.
• nonrandomstring 3y ago
  For me I see IDS as a natural easy win; collating a shitload of sensor data, tripwires, logs... and providing carefully graded alerts and advice seems like something ANN trained on lots of good field data would excel at.
  • potatohead00 3y ago
    where will this good field data come from? using what context? a low risk alert for one org might be seen as something else in another.
    • asynchronous 3y ago
      If you look at the current major offerings like SentinelOne, they start off with a generic best practice baseline, then slowly “learn” the normal traffic on the network to be able to better define the abnormal incidents to the IDS.
    • nonrandomstring 3y ago
      You make a good point. What occurred to me is that some more standardised method for coding threat events is needed first. But that seems tractable given existing CVE taxonomies etc.
• marcosdumay 3y ago
  Well, looks like "AI" nowadays is officially another name for LLMs. I wonder how long it will take for people to talk about general stuff again, because those things are getting ridiculous. Security monitoring and classification tasks have been run by AI everywhere for a decade already. And AI will very clearly not take over any other part soon. At least from the "useful" cyber security. There is, of course, the entire charlatan cyber security, that is maybe an even larger market. And well, LLMs excel at charlatanism.
• holoduke 3y ago
  The last jobs on earth will be IT personel. In all different layers. Unless some skynet takes over.
  • c0pium 3y ago
    Every profession thinks this. Doctors, lawyers, plumbers, chefs, tour guides, carpenters. Some jobs are actually likely to be automated away, but most jobs have a last 5-10% that’s non-obvious and very hard to automate.
  • lxgr 3y ago
    Maybe so, but on the way there, 95% of IT personell will have been made redundant long before many manual labor jobs. Existing manual labor jobs are what’s left after industrialization and robotics, if you think about it. IT is full of low hanging fruit for automation.
    • shrimp_emoji 3y ago
      The ultimate post-modern irony. What we thought was hard is easy; what we thought was easy is hard. (When, really, there is no easy and hard; there's only optimization of a neural net to a task, and what we find hard are things we were never optimized for.) It's also karma in that the techies responsible for automating others' jobs away are the first to get hardcore automated. c':
• motohagiography 3y ago
  Surprisingly, no. The reason I got into security in the 90s was because I saw hackers as a check on the absolute dominion of tech enabled institutions, and a big part of that was always being the exception and finding a way to remain smarter than weaponized AI. We're going to see some truly awesome AI malware, and it will devalue some skills people and hackers have heavy sunk costs in, but we prevail. Short term, cyber security is governance with a high technical competency bar to entry at any serious level, and the reason we're still in demand and these problems aren't solved is because competence at managing dynamics and risk is still irreducible. Every model and formalization creates a dynamic with whatever it models and formalizes, which creates an opportunity to exploit and manage it. AI mainly creates manageable dynamics and demand for governance, which means no matter what you're going to need hackers unless we're all livestock. Long term, security as we understand it today will go the way of the Orange book, and it will evolve as a variation what it really is, a purer understanding and way of relating to tech. Fundamentally, hacking is the logic of an idea that we can apply to anything, where instead of goodguy/badguy, or oppressor/oppressed, we have true/false, where something is only true within the domain of its assumptions, and when you change those, you get understanding and leverage. It's all the things that work in practice just not in theory. AI is a figment of aggregated and nested assumptions, so all I see is opportunity. It will be as pernicious as human conceit, and I'm pretty sure it's going to kill a lot of people, but I'd say if you don't have the practice, you don't know what you don't know about our advantages over it. We're good. :)
  • c0pium 3y ago
    [flagged]
    • czbond 3y ago
      Maybe you're not experienced enough to understand it? I clearly understood what this parent post is saying.
      • c0pium 3y ago
        That’s not what word salad means. Don’t worry, one day you’ll have the experience to understand that.
• memorybadger 3y ago
  Imagine LLM do chip decapping and perform fault injection on hardware and extract firmware. I think Cyber securiry is bigger than that. I think that take another few decades until we get there and once we are there it is not anymore just cyber security jobs.
• faeriechangling 3y ago
  Security of out all the domains of computing seems to me like it will be amongst the last fields to be automated away. The lack of precision in AI systems is pretty damning because it only takes ONE mistake to cause a breach. As others have already mentioned, AI is already extensively used in security operations. I think we will see more and more people replaced by AI who spend their days crawling through logs since it's hard to imagine something AI is better at. This is a domain of security where sheer data crunching can make up for imprecision. When it comes to things like LLMs and secure application development, AI is happy to suggest code that it got from a stackoverflow thread with massive and glaring security holes. Or code which is not inherently insecure, but assumes the systems it connects to can be trusted, which is a dangerous assumption. It isn't even close to comprehending the broader social, business, and security context of an organisation and achieving a balance of those concerns. It needs an IMMENSE amount of handholding from a true expert to be useful. By the time AI can replace cybersecurity experts, it will be able to replace human IT and development experts altogether.
  • c048 3y ago
    How about AI being an additional security layer? Say I would look at security from a different point of view, namely behavior pattern matching. If your traceability is high enough, a trained AI might be able to sniff out potential breaches immediately. The same is true for cheating in AI. Once we've trained models and games log enough metrics, I'm quite sure that AI can sniff out cheaters with high precision by behavior alone. Even if they don't use aimbots, AI should be able to sniff out wallhackers or other minor cheaters by simply analyzing how players are observing their surroundings.
    • hsbauauvhabzb 3y ago
      There are plenty of tools that do ‘ai’ in security. Anomaly detection can be valuable, but for the most part they’re snake oil.
    • seabass-labrax 3y ago
      You might well be right about AI's efficacy at detecting video game cheaters, but I've never understood why this is an issue. If cheating can't be easily distinguished by a human from skilled (but legitimate) play, why would such cheating reduce the enjoyment of the game? With a well-functioning match-making lobby, the cheaters would simply class themselves out of all the normal games and end up playing other cheaters, leaving legitimate players well alone. As someone who enjoys casually playing online games, many of which have little or no cheating detection, I genuinely don't understand why many think the problem deserves stacks of 20GB VRAM cards running AI models, let alone rootkits client-side. Please enlighten me :)
• Barrin92 3y ago
  The article is decent because it debunks a lot of stuff but it is concerning it even needs to be written. What does someone even mean when they say a cyber security worker is going to replaced by AI? I fire Bob from the IT security department and step 2 is.. ? Like in concrete terms, what happens next? It's like asking "will debuggers replace cyber security?". A tool does not replace a worker, these aren't the same category of things.
• andrewprock 3y ago
  AI, as it exists today, is very good at interpolating lots of data. It is far less able to do a good job of extrapolating far outside the boundary of existing data. Given the cat-and-mouse nature of security, where the parties regularly move outside the bounds of past behaviors, it is very very unlikely that AI will be useful in security at the boundary. It will be very good at making sure known exploits are not available, but known unknowns and unknown unknowns will remain security blind spots.
• resharpe105 3y ago
  It will be taken to a next level by the AI, both in terms of defensive and offensive capabilities. We already have interesting startups and projects dealing with defence aspect. I’ll note glog.ai as something I’ve seen most recently in this domain - it entails a machine learning algorithm that classifies SAST output and assists the developer in triage of potential vulnerabilities.
• TheLoafOfBread 3y ago
  Hacking and exploitation is usually creative job of chaining unrelated errors into code executions. AI can currently badly repeat random learned facts but creativity - ability to come up with something new - isn't AI's strength.
• 29athrowaway 3y ago
  Security professionals that simply go through compliance routines may get replaced with AI first. The ones doing security research are often incredibly highly creative people that think out of the box and do things that have not been seen before, unlike LLMs mostly remix what they've seen before.
• kseifried 3y ago
  AI won’t replace humans. But a human with AI will replace a human without AI. AI will be a supplemental tool. Just a much higher level tool. That can help improve itself and the users of it.
• gxt 3y ago
  Better operating systems will greatly reduce the need for cyber security, up to 99%, by eradicating attack vectors. AI will play a role on the remaining 1% to help humans.