7 ms·
Show HN: Quickly copy a file between computers
Hey everyone, just wanted to share a quick project I built a bit ago and finally cleaned up enough to post.
The premise is simple, you have a file on one computer that you want on a different computer on the same network and don't want a lot of fuss.
For the file you want to copy, on the machine where the file is located, run:
./bcp filename
Now go to the computer (on the same network) where you want the file to be at, and run:
./bcp
The file will be copied over the network and you're done.
I created it as a nice way of not having to worry about ssh or anything else really. In fact, it is so simple, I use it on my local machine a lot when hopping around between terminals instead of remembering paths.
I have tested it on OSX and Linux.
Suggestions and comments welcomed.
- froseph 14y agoIt reminds me of http://www.fefe.de/ncp/ http://www.fefe.de/ncp/ It's always interesting when people come up with the same thing independently.
- diginux 14y agoAh yes, I take a slightly different approach, I don't do polling, only have a listener and a single request sent. Though, this still looks more robust, thanks for the link.
- RookToH7 14y agoI couldn't seem to get this compiled on OSX :( Do you (or anyone) know if there's a OSX version available? What I like about ncp is that it first requires an action at the sender (push) and then at the receiver afterwards (poll).
- midnightsine 14y ago$ nc -l 8888 > file $ cat file | nc host.example.com 8888 I don't think a dedicated utility is required. Edit: Sorry about my comment coming off as a bit hostile, I did not intend it to be.
- niggler 14y agoBetter to do $ <file nc host.example.com 8888 (performance benefits more apparent with larger files)
- HorizonXP 14y agoCare to explain why?
- niggler 14y ago(you can find more info in the manpage for your shell) $ cat foo | bar ends up creating two processes and a pipe. The foo file is first read by cat and then written onto the pipe (which bar then reads). $ <foo bar is an input redirection: foo is opened for reading and bar's standard input fd is set to that open file (so the file's data is only read once)
- blibble 14y agofor reading files and transferring over the network: the limit is always going to be the IO devices, not an extra context switch and a few extra copies...
- 1337p337 14y agos/always/usually/. Depending on the disks involved and your network hardware (either a super-fast link, or even a system with a slow CPU and cheap network hardware that expects the drivers to do the heavy lifting in software), it actually could end up CPU-constrained. But for the usual case where it is I/O-constrained, you can often get files across more quickly by throwing CPU at reducing the total amount of bandwidth required by, e.g., using something like bzip or pbzip: pbzip2 < file | nc $host $port And on the other side: nc -l $port | pbzip2 -d > file
- Udo 14y agoThere are a million ways to copy files, a lot of them are simpler to use than your example. But that's not the point here. Sometimes you just got to make something because you want to, because it fits a specific need that maybe not a lot of other people have, or because it's a learning experience, or just for the heck of it. So, no, it's not required, but that doesn't mean it's useless. It's cool that the author did something productive that works for him/her, and it's even cooler they shared it with everyone. I can say that's already infinitely more than what I made today. How about you?
- dclowd9901 14y agoI like how clever this is, but is there an option to name a bcp (or a planned option), so that I can send files among a big network with lots of people using this command concurrently?
- diginux 14y agoYes, I plan to do this as well as have the ability for the listener to stay active indefinitely instead of for just one transmission.
- dclowd9901 14y agoRad, especially the persistent mode.
- jamescun 14y agoDefinitely a interesting take on sharing data over a LAN but I would be worried about the repercussions of using broadcasts to move a large quantity of data on a larger network, cool for smaller/personal networks though.
- jbester 14y agoLooking at the code, UDP/Broadcast is used only for negotiation. After negotation, a TCP socket is used for the actual transfer.
- abcd_f 14y agoNice. Very Unix-ish in its spirit. The client-server arrangement is backwards though :) Typical arrangement is for the clients to do the "anyone there?" broadcast, for the servers to reply and then the client would select the server, connect to it and they would go about their business. In your case, the server connects to the client. If you re-arrange this to natural client-server order, you should be able to get rid of the fork() call and this will help with portability (not that you probably care at this point). Also, size = fread(&buf, 1, 100, ft) No harm in using chunks larger than 100, especially when dispensing larger files. Also, consider switching to multicast for discovery.
- diginux 14y agoI agree with you on it being backwards and for some reason I chose to do it this way, though I can not now remember why.. may have just been a flawed though, as I can see no reason not to do it your way at the moment. Good find on the fread, that is actually a "bug", should be MAXBUFLEN instead of 100. Agreed on multicast, I will add that to do the todo. Thanks for your feedback!
- pyre 14y agoThe example in the README could be better if the command prompt showed that you were on different hosts. As it stands, it seems that you sent it from host 'heisenberg' to host 'heisenberg'.
- dclowd9901 14y agoDifferent ports? Might as well be different hosts.
- pyre 14y agoActually, I didn't notice the port. I did notice that the IP and hostname were the same though.
- diginux 14y agoFair enough, I will change it :)
- huhsamovar 14y ago$ scp file host.example.com:
- niggler 14y agoSCP is significantly slower than FTP (and performance was an explicit goal of the project)
- misframer 14y agoThis method isn't as fast since it uses encryption.
- halayli 14y agoYeah few milliseconds are going to make a difference here.
- diginux 14y agoFor big files it would make a very noticeable difference actually, try scp with different encryption algorithms even and you will see how much it matters. With that said, by 'quickly', I was focusing more on easiness and simplicity.
- andrewcooke 14y agothere's a constant size overhead, and file copying is network, not cpu limited (isn't it?). so why would large files be significantly slower?
- diginux 14y agoI am not sure it is fair to say scp is cpu limited per se, but it is slow. A quick article that may help you out: http://intermediatesql.com/linux/scrap-the-scp-how-to-copy-data-fast-using-pigz-and-nc/ http://intermediatesql.com/linux/scrap-the-scp-how-to-copy-d...
- cowmix 14y agoThis reminds me of how you used to be able to disable the cypher on SSH and now you can't. I hate that. http://serverfault.com/questions/116875/how-can-i-disable-encryption-on-openssh http://serverfault.com/questions/116875/how-can-i-disable-en...
- ajross 14y agoUm... why? rsh still builds and works fine if that's what you want.
- jbert 14y agossh with no cipher still requires authentication, and can stil use pubkey auth. So "ssh with no cipher" means "people can snoop, but I can use proper auth". But rsh means "plaintext pw - or worse things if allowed".
- nbkolchin 14y agoYou can use High Performance patches for ssh: http://www.psc.edu/index.php/hpn-ssh http://www.psc.edu/index.php/hpn-ssh Cipher - none.
- jbester 14y agoIt's neat. I noticed a couple of subtle bugs. 1) The code assumes both the client and the server have the same endian. This can be issue since a uint32 is used for both the synchronization code (BCP_CODE) and the port. This can easily fixed with htonl/ntohl conversions. 2) it assumes both the client and the server define int to be the same size (may or may not be true based on compiler/processor/OS combinations). If you use stdint definitions, this typically won't be an issue.
- diginux 14y agoBoth great points. Will add these on my todo list. I appreciate you taking the time to look at the code.
- srl 14y agoAnd another, noted here: http://news.ycombinator.com/item?id=5232416 http://news.ycombinator.com/item?id=5232416 (Again, I don't mean to bash you or this project, just to say that it shouldn't be used in the "real-world" without being looked at rather more carefully. On a locally-secure network, or in an environment where security is unimportant, it's pretty cool.)
- diginux 14y agoCompletely agreed, I assumed the project would give the perception that it isn't a fully robust tool yet, but it is always worth to make it completely clear.
- xal 14y agoHehe, The first thing I wrote when I learned programming ( around 1990 ) was a tool called ipxcopy which did the same over IPX protocol that we always used at our Lan Parties.
- dschiptsov 14y agoscp?)
- donflamenco 14y agoZmodem (rz/sz) works if your terminal supports it. Though putty doesn't support it (wish it did.)
- dudurocha 14y agoOne of the ideas I have since I was a kid, was a graphical way to copy files between computers. Something like you just drag for the extreme left of right border, and sunddenly, the file is being transferred for the other computer.
- diginux 14y agoIf that doesn't already exist for synergy (http://synergy-foss.org/ http://synergy-foss.org/), that would be an awesome addition.
- readme 14y agoI think that was a planned feature for synergy. I don't think it was ever implemented.
- abcd_f 14y agoThis has some usability issues, e.g. you drag a file accidentally to another box, then change your mind and drag it back while it is still being copied or moved in the original direction. This sort of thing. It looks like a simple and natural extension to what Synergy has, but in reality it's just a can of worms.
- diginux 14y agoI wonder if there is a way they could utilize Dropbox or Dropbox-like functionality?
- deleted 14y ago[deleted]
- blt 14y agopython -m SimpleHTTPServer is another nice way to do this.
- diginux 14y agoI still have to know the IP of my machine, make a script wrap around python and wget or curl to achieve the bcp functionality. I actually use SimpleHTTPServer a lot for other things though, it is a great tip.
- dredmorbius 14y agoReminds me of http://www.advogato.org/article/555.html http://www.advogato.org/article/555.html
- diginux 14y agoGreat find! I added this link, as well as ncp to the README.
- deleted 14y ago[deleted]
- sebkomianos 14y agoTitle should be "between computers on a lan".
- latitude 14y agoTry and add support for detecting piped input on the server side: bcp < filename because this would allow for things like cat filename | gzip | bcp with the receiving end doing bcp | gunzip > filename Keep it as simple as it is now, but make it play nicely with other tools.
- lutusp 14y agoYour examples are all expected to sort out whether to accept an input stream or start an output stream, but without an explicit command, which poses a problem -- the app would have to determine that the input stream is empty before switching modes to streaming output. That's not so easy -- suppose the operator unintentionally pipes an empty input file? Will the app know what to do? Most (I won't say all) command-line apps must be explicitly told which mode to use. There's a good reason.
- huhtenberg 14y agoisatty() to the rescue, but that's not to say that there shouldn't be a -- option or defaulting to a different mode depending on argv[0], like gzip does. [0] http://pubs.opengroup.org/onlinepubs/007904875/functions/isatty.html http://pubs.opengroup.org/onlinepubs/007904875/functions/isa...
- diginux 14y agoThis would be a great addition, I will add it as an issue, thanks!
- kylecordes 14y agoUdpcast: http://www.udpcast.linux.lu/ http://www.udpcast.linux.lu/ Can do this and other related useful things, including multicast of a file to N machines in parallel. Don't let the UDP bother you: it implements retry/checksum/etc on top of datagrams. It's been around a few years, and is probably already in your distro. It can pipe, as well as copy files, I wrote about it some time ago: http://kylecordes.com/2008/multicast-your-db-backups-with-udpcast http://kylecordes.com/2008/multicast-your-db-backups-with-ud...
- diginux 14y agoThis is great tool and nice article. The only slightly additional requirement for this tool is you know the name of the file being sent. I actually was thinking of making this optional (for the same reasons they require it, to allow many different files). I've added a link for udpcast to my readme as an alternative. Thanks!
- julian37 14y agoThere's also UFTP, it doesn't require the receiver to know the file name. (But it is different from your implementation in a number of ways, most importantly it transfers all data via UDP.) http://www.tcnj.edu/~bush/uftp.html http://www.tcnj.edu/~bush/uftp.html
- songgao 14y agoAnd it's multicast, which saves a lot of bandwidth and makes it faster :-)
- f1codz 14y agoI use (and highly recommend) IP Messenger for this style of auto-discovery of other systems on your LAN and message/file transfer: http://ipmsg.org/index.html.en http://ipmsg.org/index.html.en (For Windows) http://ishwt.net/en/software/ipmsg/ http://ishwt.net/en/software/ipmsg/ (For Mac)