I know everybody's doing it because they don't know better, but it's a terrible idea to make the inductive leap from one successful sample to some abstract sens

I know everybody's doing it because they don't know better, but it's a terrible idea to make the inductive leap from one successful sample to some abstract sense of what a ML model is suited for. Especially for anything important. As a sibling comment noted, performance will almost certainly be sensitive to temperature (randomness), exact prompt phrasing, exact sequence of messages in a dialog, and the training-data frequency of both the site being analyzed and the phishing approach used. One could conceivably train a specialized ML model, perhaps with an LLM component, to detect sophsticated phishing attempts and I would assume this has even been done. But using a relying on generic "helpful chatbot" to do that reliably and sufficiently is a really bad idea. That's not what it's for, not what's good at, and not something its vendor promises for it to remain good at even if it happens to be today.