4 ms·

AI-Exploits: Repo of multiple unauthenticated RCEs in AI tools

• aftbit 3y ago
  Is anyone using any of these services? The only one I actually recognize from their list[1] is Triton Inference Server. 1: https://github.com/protectai/ai-exploits/tree/main/nmap-nse https://github.com/protectai/ai-exploits/tree/main/nmap-nse
  • swatcoder 3y ago
    The purpose of the repo seems to be to collect an archive of what real-world vulnerabilities look like, to inform service implementors and security researchers in their future work. I suppose I’m idly curious about the answer to your question too, but paying too much attention to the specific targets feels like it’s missing the point and purpose of the collection.
  • spmurrayzzz 3y ago
    h2o is definitely somewhat popular specifically for LLMs, but ray is certainly widely used for distributed training workloads
  • deleted 3y ago
    [deleted]
  • ianbutler 3y ago
    I recognize most of them, they're all pretty common orchestration, distributed computation, or experiment management tools. Maybe you're just not as integrated on the operations portion of the ML space?
  • wolftickets 3y ago
    [I work at Protect AI] - The goal here was initially relatively common tooling around MLOps/Data Science work. All ears here if you have some ideas for other projects to explore.
• gumballindie 3y ago
  No wonder people working in ai think ai will replace programmers, given the prevalent lack of experience with actual programming among them. Having said that, the Achilles heel of ai is data. The lower the quality the more powerful the attack. I imagine if someone wanted to mess about with it on a serious scale they’d go for the jugular - the data. Write content and create hundreds or thousands of code repositories with subtle issues and bang, you’ve compromised thousands and thousands of unsuspecting folks relying on ai to create code, or any other type of content.
  • wolftickets 3y ago
    [I work at Protect AI] You're spot on for data being the jugular, interestingly with exploits like this as an attacker you could quickly go for attacking model content but also have credentials that would grant you access to data in many cases. These tools can serve as the first opening but a sizable one when looking to attack an enterprise more broadly.
    • deleted 3y ago
      [deleted]
    • swyx 3y ago
      > Protect AI is the first company focused on the security of AI and ML Systems creating a new category we call MLSecOps. alright i looked you up, congrats on your fundraising. is there like an OWASP top 10 vuln list for MLSecOps? does it differ between traditional ML apps and LLM apps?
      • byt3bl33d3r 3y ago
        (I work for ProtectAI) There isn't an OWASP top 10 for MLSecOps at the moment. There is a general OWASP top 10 for Machine Learning [1] and MITRE ATLAS [2] however. [1] https://owasp.org/www-project-machine-learning-security-top-10/ https://owasp.org/www-project-machine-learning-security-top-... [2] https://atlas.mitre.org/ https://atlas.mitre.org/
    • gumballindie 3y ago
• RomanPushkin 3y ago
  How does it work? Can't understand from the description
  • byt3bl33d3r 3y ago
    (I work for ProtectAI) We added a quick demo to the Readme [1] [1] https://github.com/protectai/ai-exploits?tab=readme-ov-file#demo https://github.com/protectai/ai-exploits?tab=readme-ov-file#...
    • friendlynokill 3y ago
      > With the release of this repository, Protect AI hopes to demystify to the Information Security community what pratical attacks against AI/Machine Learning infrastructure look like in the real world and raise awareness to the amount of vulnerable components that currently exist in the AI/ML ecosystem. More vulnerabilities can be found here: November Vulnerability Report pratical --> practical
• waihtis 3y ago
  Nice work, just saw these pop up on the official CVE feed