It's also worth noting that to my knowledge, written CSAM content largely isn't illegal, it's just visual depictions that are. I'm not sure there's even any law

It's also worth noting that to my knowledge, written CSAM content largely isn't illegal, it's just visual depictions that are. I'm not sure there's even any law prohibiting profiting off of commercial, written depictions of CSAM, though I wouldn't swear to it, it's just that you'd have a very hard time with typical commercial infrastructure (web hosts, banks, payment processors, advertisement firms, publishers and distributors) because you'd be so poisonous to public opinion. So I'm not fully convinced that a LLM that generates CSAM text, even if it was interactive, would be in any way restricted by law. Image generation is, of course, a little different.