I was recently a defendant in a a set of lawsuits by a scammer seeking hundreds of billions of dollars in damages, using the courts as a weapon to shut down peo

I was recently a defendant in a a set of lawsuits by a scammer seeking hundreds of billions of dollars in damages, using the courts as a weapon to shut down people who were in the way of his confidence game. The cumulative costs incurred by his opponents ended up well over ten million £‎. After a five week trial the court found against him in every respect, concluding his evidence was forged, that his actions constituted a fraud upon the court, etc. Relevant to this thread is that partly through the process he began to use ChatGPT. He appeared to use it in four ways: To create forgeries that he didn't have the technical skills to create (at least not on a short timeframe) with e.g. ChatGPT walking him through the technical steps, to concoct excuses and explanations, and to flood out his opponents with hundreds of pages of ChatGPT authored testimony that expended our resources just reading the material. He also used it to suggest legal strategy, which didn't cause us any problems and probably worked in our favor as ChatGPT's suggestions on that front appear to have been uniformly idiotic. Though he concealed his earlier usage, claiming to have lost access to his prior account, all in all his ChatGPT activity from around September 2023 to January 2024 amounted to some 22 million lines of text. We were extremely fortunate that his usage of ChatGPT began after his litigation began and that he made numerous errors that made his usage unambiguous. Had he been using it all along and less obviously our cost of defending the claim may have been increased dramatically-- even at his current usage had we been funding our defense entirely out of our own pockets his abuse of chatgpt may well have been the straw that bankrupted us before we could secure our victory. I think the us of LLMs as a tool in crime is interesting. For many negative uses of LLMs like spamming you can counter that people could just hire humans for the task so the LLM is more a question of cost or degree. But when committing a crime anyone you hire is a party that could extort you or whistleblow. We suspect our con made as many errors as he did with his forgeries in part due to his limited ability to enlist third parties to vet them. The potential for productive, honest, usage of the technology in this case appeared pretty limited relative to the potential for abuse. Some of the abuse could be addressed through improvements in standard practices like filing length limits, but its less clear to me how abuse like using it as a confidential forgery aid can be addressed. I certainly don't think it can be solved with safety approaches, since the interaction never needs to tell the LLM the true purpose. The same doesn't work for a human forgery aid because their access to outside information and potential for being questioned would make it much harder to avoid the risk that they discover that they're being used as a tool for fraud.