One thing that I keep thinking about are those "I make $XX/hour working from home" spam/scam messages that have been plaguing comment threads and message boards

One thing that I keep thinking about are those "I make $XX/hour working from home" spam/scam messages that have been plaguing comment threads and message boards for as long as I can remember. They are very transparently scams, yet they keep getting posted which means that they must be working on someone. The templated, borderline nonsense messages in these scams provide as a baseline "minimum viable scambot". Now think about how much better an LLM would be at generating those prompts -- the number of people that will get tricked will be orders of magnitude higher because the simple heuristics for spotting a scam are gone (look for weird formatting to circumvent filters, look for messages that completely ignore the context of the thread they're in, etc). This will suck for the people that get scammed, obviously. But there will also be second order effects once it's clear that the cost/benefit of running scambots has increased: spam filters will stop working, nobody will trust anybody in online fora, social platforms will adopt onerous verification rules, etc