As long as you only train the network and never use it for inference, I suppose that's a reasonable argument. But a person is restricted in action by copyright

As long as you only train the network and never use it for inference, I suppose that's a reasonable argument. But a person is restricted in action by copyright regardless of the writing implement they choose to use. Whether they use a pencil or an LLM, they cannot freely reproduce copyrighted works in whole or in part, excepting some narrow conditions.