I think FTP the protocol would've been fine, and indeed is best suited for the job. The problems really stemmed from having to poll the filesystem because the F

I think FTP the protocol would've been fine, and indeed is best suited for the job. The problems really stemmed from having to poll the filesystem because the FTP server was also not under our control. If you embed the FTP server in your app (say, with IKVM ), you can keep the whole intermediate file where you want (or even in RAM if it fits) and this entire problem shrinks 10x.