Multiprocess isn't a panacea. Several frameworks get grumpy depending on the order of forking. E.g. Iirc, try to use grpc to feed data to a process using pytorc

Multiprocess isn't a panacea. Several frameworks get grumpy depending on the order of forking. E.g. Iirc, try to use grpc to feed data to a process using pytorch dataloader, and it'll straight up crash. Huggingface is at least polite enough to warn you, but performance degrades.