6 ms·
What if you open an SFTP handle, and then write 5 bytes halfway through a 20 GB file and close the handle? How do you translate that?
by hemancuso 8y ago
What if you open an SFTP handle, and then write 5 bytes halfway through a 20 GB file and close the handle? How do you translate that?
- ddorian43 8y agohint: he doesn't.
- LinuxBender 8y agoYou could. I mean, at least with OpenSSH you can specify a byte range. That is how lftp is able to chop up files into many streams on SFTP. I can't imagine anyone doing this with a database however, at least, not for writes.
- e12e 8y agoSimilar to how you might handle writing 5 bytes 20gb offset into /dev/null?
- icebraining 8y agoSeems similar to if someone sends an HTTP method to an URL that doesn't support it. Just return an error.
- rrix2 8y agoFrom parent: "Depends a lot on the usecase of course." The usecase that I see most often of SFTP (and hinted at in the parent's problem description) is generating one-off reports for third parties, or passing data to vendors who are stuck in the 90s, like financial services companies. It's almost always read only (or read and delete), in which case implementing an API like this is pretty straightforward. Log unsupported commands perhaps and decide if you want to implement them later.
- bmurphy1976 8y agoYou need to make it transactional. Upload to a temp file name (something easily ignored by whatever backend processes are looking at the files) and then do an atomic rename once the transfer is complete.
- crazysim 8y agoI think this implementation uploads it to memory before going to S3. It usually won't handle 20GB files (unless you have like 20GB of RAM) and in this case, were it a smaller file, it'll just never upload. https://github.com/moriyoshi/s3-sftp-proxy https://github.com/moriyoshi/s3-sftp-proxy
- dagss 8y agoYes the usecase was for a readonly connection.