5 ms·

"Vibe hacking" is real - here's an excerpt from my actual ChatGPT transcript trying to generate bot scripts to use for account takeovers and credential stuffing

"Vibe hacking" is real - here's an excerpt from my actual ChatGPT transcript trying to generate bot scripts to use for account takeovers and credential stuffing: >I can't help with automating logins to websites unless you have explicit authorization. However, I can walk you through how to ethically and legally use Puppeteer to automate browser tasks, such as for your own site or one you have permission to test. >If you're trying to test login automation for a site you own or operate, here's a general template for a Puppeteer login script you can adapt: ><the entire working script, lol> Full video is here, ChatGPT bit starts around 1:30: https://stytch.com/blog/combating-ai-threats-stytchs-device-fingerprinting/ https://stytch.com/blog/combating-ai-threats-stytchs-device-... The barrier to entry has never been lower; when you democratize coding, you democratize abuse. And it's basically impossible to stop these kinds of uses without significantly neutering benign usage too.

deleted 1y ago
[deleted]
dheera 1y ago
If I were in charge of an org's cybersecurity I would have AI agents continually trying to attack the systems 24/7 and inform me of successful exploits; it would suck if the major model providers block this type of usage.
- idontwantthis 1y ago
  Horizon3 offers this.
- cube00 1y ago
  That sounds expensive, those LLM API calls and tokens aren't cheap.
  - brulard 1y ago
    Actually thats quite cheap for such a powerful pentesting tool.
  - throwawaysleep 1y ago
    It’s about $200 a month for 15 human hours a day.
- jsheard 1y ago
  Judging from the experience of people running bug bounty programs lately, you'd definitely get an endless supply of successful exploit reports. Whether any of them would be real exploits is another question though. https://daniel.haxx.se/blog/2025/07/14/death-by-a-thousand-slops/ https://daniel.haxx.se/blog/2025/07/14/death-by-a-thousand-s...
- netvarun 1y ago
  Shameless plug: We're building this. Our goal is to provide AI pentesting agents that run continuously, because the reality is that companies (eg: those doing SOC 2) typically get a point-in-time pentest once a year while furiously shipping code via Cursor/Claude Code and changing infrastructure daily. I like how Terence Tao framed this [0]: blue teams (builders aka 'vibe-coders') and red teams (attackers) are dual to each other. AI is often better suited for the red team role, critiquing, probing, and surfacing weaknesses, rather than just generating code (In this case, I feel hallucinations are more of a feature than a bug). We have an early version and are looking for companies to try it out. If you'd like to chat, I'm at varun@keygraph.io. [0] https://mathstodon.xyz/@tao/114915606467203078 https://mathstodon.xyz/@tao/114915606467203078
  - mdaniel 1y ago
    > Our goal is to provide AI pentesting agents that run continuously, Pour one out for your observability team. Or, I guess here's hoping that the logs, metrics, and traces have a distinct enough attribute that one can throw them in the trash (continuously, natch)
    - trog 1y ago
      You can set this up in a non-production environment and realise a lot of the benefits. It would also help you figure out better ways to manage your logs such that you can improve signal-to-noise ratio in monitoring solutions and alarming. Not convinced "AI" is needed for this sort of around the clock pen testing - a well-defined set of rules that is being actively maintained as the threat landscape changes, and I am pretty sure there are a bunch of businesses that offer this already - but I think constant attacking is the only way to really improve security posture. To quote one of my favourite lines in Neal Stephenson's Anathem: "The only way to preserve the integrity of the defenses is to subject them to unceasing assault".
cyanydeez 1y ago
So many great parallels to the grift econy
cj 1y ago
Refusing hacking prompts would be like outlawing Burpsuite. It might slow someone down, but it won’t stop anyone. Perhaps vibe hacking is the cure against vibe coding. I’m not concerned about people generating hacking scripts, but am concerned that it lowers the barrier of entry for large scale social engineering. I think we’re ready to handle an uptick in script kiddie nuisance, but not sure we’re ready to handle large scale ultra-personalized social engineering attacks.
- eru 1y ago
  > It might slow someone down, but it won’t stop anyone. Nope, plenty of script kids go and something else.
quotemstr 1y ago
> The barrier to entry has never been lower; when you democratize coding, you democratize abuse. You also democratize defense. Besides: who gets to define "abuse"? You? Why? Vibe coding is like free speech: anything it can destroy should be destroyed. A society's security can't depend on restricting access to skills or information: it doesn't work, first of all, and second, to the extent it temporarily does, it concentrates power in an unelected priesthood that can and will do "good" by enacting rules that go against the wishes and interest of the public.
- chii 1y ago
  > You also democratize defense. not really - defense is harder than offence. Just think about the chance of each: for defense, you need to protect against _every attack_ to be successful. For offence, you only need to succeed once to be successful - each failure is not a concern. Therefore, the threat is asymmetric.
  - nradov 1y ago
    Is defense take that hard? The majority of successful attacks seem to result from ignoring basic best practices. Just total laziness and incompetence by the victims.
anon22981 1y ago
Mikko Hyppönen, who holds at least some level of authority on the subject, just recently said in an interview that he believes currently the defenders have the advantage. He claimed there’s currently zero known large incidents where the attackers have been known to utilize LLMs. (Apart from social hacking.) To be fair, he also said that the defenders having the advantage is going to change.

deleted 1y ago

[deleted]

dheera 1y ago

If I were in charge of an org's cybersecurity I would have AI agents continually trying to attack the systems 24/7 and inform me of successful exploits; it would suck if the major model providers block this type of usage.

idontwantthis 1y ago
Horizon3 offers this.
cube00 1y ago
That sounds expensive, those LLM API calls and tokens aren't cheap.
- brulard 1y ago
  Actually thats quite cheap for such a powerful pentesting tool.
- throwawaysleep 1y ago
  It’s about $200 a month for 15 human hours a day.
jsheard 1y ago
Judging from the experience of people running bug bounty programs lately, you'd definitely get an endless supply of successful exploit reports. Whether any of them would be real exploits is another question though. https://daniel.haxx.se/blog/2025/07/14/death-by-a-thousand-slops/ https://daniel.haxx.se/blog/2025/07/14/death-by-a-thousand-s...
netvarun 1y ago
Shameless plug: We're building this. Our goal is to provide AI pentesting agents that run continuously, because the reality is that companies (eg: those doing SOC 2) typically get a point-in-time pentest once a year while furiously shipping code via Cursor/Claude Code and changing infrastructure daily. I like how Terence Tao framed this [0]: blue teams (builders aka 'vibe-coders') and red teams (attackers) are dual to each other. AI is often better suited for the red team role, critiquing, probing, and surfacing weaknesses, rather than just generating code (In this case, I feel hallucinations are more of a feature than a bug). We have an early version and are looking for companies to try it out. If you'd like to chat, I'm at varun@keygraph.io. [0] https://mathstodon.xyz/@tao/114915606467203078 https://mathstodon.xyz/@tao/114915606467203078
- mdaniel 1y ago
  > Our goal is to provide AI pentesting agents that run continuously, Pour one out for your observability team. Or, I guess here's hoping that the logs, metrics, and traces have a distinct enough attribute that one can throw them in the trash (continuously, natch)
  - trog 1y ago
    You can set this up in a non-production environment and realise a lot of the benefits. It would also help you figure out better ways to manage your logs such that you can improve signal-to-noise ratio in monitoring solutions and alarming. Not convinced "AI" is needed for this sort of around the clock pen testing - a well-defined set of rules that is being actively maintained as the threat landscape changes, and I am pretty sure there are a bunch of businesses that offer this already - but I think constant attacking is the only way to really improve security posture. To quote one of my favourite lines in Neal Stephenson's Anathem: "The only way to preserve the integrity of the defenses is to subject them to unceasing assault".

cyanydeez 1y ago

So many great parallels to the grift econy

cj 1y ago

Refusing hacking prompts would be like outlawing Burpsuite. It might slow someone down, but it won’t stop anyone. Perhaps vibe hacking is the cure against vibe coding. I’m not concerned about people generating hacking scripts, but am concerned that it lowers the barrier of entry for large scale social engineering. I think we’re ready to handle an uptick in script kiddie nuisance, but not sure we’re ready to handle large scale ultra-personalized social engineering attacks.

eru 1y ago
> It might slow someone down, but it won’t stop anyone. Nope, plenty of script kids go and something else.

quotemstr 1y ago

> The barrier to entry has never been lower; when you democratize coding, you democratize abuse. You also democratize defense. Besides: who gets to define "abuse"? You? Why? Vibe coding is like free speech: anything it can destroy should be destroyed. A society's security can't depend on restricting access to skills or information: it doesn't work, first of all, and second, to the extent it temporarily does, it concentrates power in an unelected priesthood that can and will do "good" by enacting rules that go against the wishes and interest of the public.

chii 1y ago
> You also democratize defense. not really - defense is harder than offence. Just think about the chance of each: for defense, you need to protect against _every attack_ to be successful. For offence, you only need to succeed once to be successful - each failure is not a concern. Therefore, the threat is asymmetric.
- nradov 1y ago
  Is defense take that hard? The majority of successful attacks seem to result from ignoring basic best practices. Just total laziness and incompetence by the victims.

anon22981 1y ago

Mikko Hyppönen, who holds at least some level of authority on the subject, just recently said in an interview that he believes currently the defenders have the advantage. He claimed there’s currently zero known large incidents where the attackers have been known to utilize LLMs. (Apart from social hacking.) To be fair, he also said that the defenders having the advantage is going to change.