9 ms·

It really is a shame that there still isn't a easy way (A person whose computer knowledge extends to using facebook), that I know of, of sending arbitrarily lar

It really is a shame that there still isn't a easy way (A person whose computer knowledge extends to using facebook), that I know of, of sending arbitrarily large files that isn't tethered to a specific cloud service and is also reliable (can tolerate connection dropping). It seems that bitorrent protocols are pretty close, but I don't think there is a seamless client that allows for "magical" point to point transactions.

bubblethink 9y ago
Both parties would have to be online for any p2p solution to work. And if they are online, there are plenty of ways to create a p2p link. It is complicated by things like NAT though. However, that is the less common case. In most cases, the receiver isn't going to be online. So you need an intermediate server, and that's how you end up with third party solutions.
- wybiral 9y ago
  Yeah, the "being online" detail seems like the problem. It would be best if the encrypted data were distributed (something like IPFS) but then why would peers host files that aren't for them.
  - thatcat 9y ago
    You would need a seedbox to function as a cloud. Then when you send a file, send to seedbox simultaniously. This approach might also increase dl speed for the reciever.
  - C0DEHERO 9y ago
    They don't currently, but Filecoin is supposed to create an incentive for people to host other people's files. Obviously, you'll have to pay some amount to have your files hosted.
    - m-p-3 9y ago
      Storj is based on a similar concept.
  - akavel 9y ago
    AFAIR, there's already some service on IPFS which will pin files for you for money. I.e.: why would someone? Because you'd pay them for that - simple :)
- juliendorra 9y ago
  Smartphones in Europe are close to being online 24/24. Energy management by OSes might limit apps in what they can do, of course. In any case, 4G networks rise might lead to interesting new solutions.
- z3t4 9y ago
  If you can assemble IKEA furniture you can also run a Linux server.
AndrewDucker 9y ago
You're at a computer. You need to connect to another computer. First you need to know a route to that computer. If it has an externally reachable IP address and you know that, then great. If it has an externally reachable IP address and a DNS entry and you know _that_, then also great. If you don't know the IP address or domain name of the other computer then you'll have to do some kind of lookup/exchange to find it. That means some kind of centralised service to provide the lookup functionality. If the other computer doesn't even has an externally reachable IP address then a central service is going to have to act as a connection point which you can both connect to (or provide some other method of helping the two of you connect). I'm not aware of any entirely decentralised system which would allow two computers which are behind NAT to find and then talk to each other. Or any obvious design which would work there.
- jchw 9y ago
  Two thoughts: - IPv6 kind of helps here, at least if we hope that no NAT standard ever makes it into IPv6. Crossing my fingers. - There does exist at least one NAT hole-punching technique that can traverse two NATs with no central server using ICMP-based holepunching and UDP. Obviously, like all hole punching techniques, it only works on certain kinds of NATs, and firewalls can kill it.
  - dingaling 9y ago
    IPv6 doesn't help in corporate environments, where it's often firewalled harder than IPv4. The fact that it doesn't support NAT is seen as a risk in such environments ( which rightly or wrongly consider it as a second level of firewalling ). Some deployments that I've seen use only site-scoped with no globally-routed prefixes. Everything Internetty has to go over IPv4, which makes the Security team's job easier; drop all IPv6 at the DMZ and drop all non-NAT IPv4.
  - ac2u 9y ago
    > There does exist at least one NAT hole-punching technique that can traverse two NATs with no central server using ICMP-based holepunching and UDP. Obviously, like all hole punching techniques, it only works on certain kinds of NATs, and firewalls can kill it. I think you're referring to this? Clever hack indeed. https://github.com/samyk/pwnat https://github.com/samyk/pwnat
w4yai 9y ago
I'm not sure it tolerates connection dropping, but I've been using justbeamit for years and it never deceived me : https://www.justbeamit.com https://www.justbeamit.com
throw2016 9y ago
Technically a one to one connection to send files should be simple, just open a socket on both sides or use nc. Of course there is NAT but its not insurmountable. The sending or receiving party has to share their IP. And this can only be real time, there is no scope to store files anywhere. Aside I have recently noticed visits to ecommerce sites sit near permanently on top of my Firefox history than other sites, is Firefox doing deals with Amazon and others, and has this been been disclosed?
unlmtd1 9y ago
IPFS
peterwwillis 9y ago
Well, you can with FTP and UPnP. You the client gets the external IP of the machine, then use UPnP to open ports on the router/firewall, then uses PASV and connects to the destination. The destination runs an FTP server, that uses UPnP to open up the incoming port, and gets its external IP and uses it for PORT mode. The end result is two individuals behind NAT sharing arbitrary files in bulk. You can even tie this into FXP to do syncing with the cloud. Or run a local SFTP server with UPnP and avoid the extra complexity. In terms of finding each other, the client could get its external IP, open up a UPnP port, and provide the user with a QR code or brief snippet to paste into a chat conversation, which the other user would feed to their client to initiate the file transfer. You could do this commercially by providing an external website with which the clients could communicate out of band to set up their transfers (for free), and (for pay) optionally provide a cloud sync service (which would help person A "pre-send" a big file transfer until person B was available to receive it). While some of this is available today, you're right, I don't think there are accessible clients. ICQ, AIM and IRC used to be the solution, as they all did P2P file transfer. But now everything is on the web, so everything sucks.
pingiun 9y ago
> It seems that bitorrent protocols are pretty close, but I don't think there is a seamless client that allows for "magical" point to point transactions. instant.io works pretty well for me, it works on the bittorrent protocol, but over webrtc
hiq 9y ago
magic-wormhole ( https://github.com/warner/magic-wormhole https://github.com/warner/magic-wormhole ) is not so far from that, just add a simple GUI on top of it (or just use the terminal and convince the user that a terminal does not have to be complicated), and you're good to go. I don't know how reliable it is w.r.t. connection drops though.
csdreamer7 9y ago
Local-wise you can use Nitroshare. I use it all the time to quickly send files between my computers. https://nitroshare.net/ https://nitroshare.net/
danieldk 9y ago
Resilio Sync (formerly Bittorrent Sync) pretty much offers that. They use relays in case both peers cannot pierce NAT, but the relays only see ciphertext. They also have apps for Android and iOS, making transfers to mobile devices also possible. Since it is closed-source software, it's up to you to decide whether you trust the software. SyncThing is a free software alternative, but I wouldn't say that it is usable (yet) for the average user.
- mapgrep 9y ago
  I used BitTorrent Sync for a couple of years and loved it. Has it evolved much under the Resilio banner?
  - rodolphoarruda 9y ago
    Me too. I stopped using it after I read somewhere that it could be hacked to include new peers to your connection without your awareness, which is a serious concern. I don't know how legit the claim was... but I opted to avoid the risk.
  - ukyrgf 9y ago
    I keep mine updated to the latest version and still love it. The 'Ignore' file that is in the top-level of each share alone is worth the admission (similar to .gitignore). Dropbox/OneDrive/etc. all lose their damn minds if you try to sync a folder with 'node_modules' or 'vendor' to the share.
- ReAzem 9y ago
  Instead of Resilio Sync, I would suggest that people take a look at Syncthing. They have similar features but Syncthing is open source.
  - QasimK 9y ago
    There is no iOS app for Syncthing as far as I can tell
mwest 9y ago
I've found https://github.com/warner/magic-wormhole https://github.com/warner/magic-wormhole really useful, although it only meets 1/3 of your requirements. I'm hoping someone will add a nice GUI and re-transmission. Previously on HN: https://news.ycombinator.com/item?id=14649727 https://news.ycombinator.com/item?id=14649727
- staz 9y ago
  On the principle it look really nice, but with a quick glance at the code I'm not too sure I would trust it... https://github.com/warner/magic-wormhole/blob/master/src/wormhole/_boss.py#L62 https://github.com/warner/magic-wormhole/blob/master/src/wor...
drzaiusapelord 9y ago
I just want to be able to right-click somewhere and have the system bring up a a simple FTPS or SFTP daemon and have it try UPnP to open a firewall/NAT transversal. This daemon would spin up just for the transfer then shut itself down once done. Even if both parties are behind a firewall that won't respect UPnP, there's UDP hole punching which should usually work. Mozilla would just need to host a server to handle the port number handoffs. I imagine this would handle 90% of the use cases. The rest would use traditional 3rd party hosting, IM transfers, email, etc.
thatcat 9y ago
file.pizza is pretty solid, but uses webrtc so ive had failures when sending to some iOS users. Apparently webrtc is included in safari 11 though.
- sunsetMurk 9y ago
  thanks for sharing this - very cool.
kyrra 9y ago
https://upspin.io/ https://upspin.io/ is in super early concept stage, but trying to solve file sharing issues. It allows for hosting on multiple platforms and trying to keep control in the hands of the users. https://upspin.io/doc/overview.md https://upspin.io/doc/overview.md
alexnewman 9y ago
Magic wormhole
tjoff 9y ago
And this isn't? (Honest question, hasn't looked at it hard enough yet) I don't disagree with you but I kind of expected a note as to why this wasn't a solution in this context.
- hardwaresofton 9y ago
  I think he considers this as "tethered to a cloud service". I think a solution to this problem that wasn't tethered would be something like in an IPV6 world where everyone has an IP address, if I could just put in your address, send you a file, and as long as our computers were on/connected to the world wide web, it'd get to you.
stephenboyd 9y ago
This was once a common use of AOL Instant Messenger.
wodenokoto 9y ago
File.pizza is very seamless point to point transfer of arbitrary file sizes. Doesn't work in safari though (haven't tested ie) so maybe it is not general enough for your use case.
- api 9y ago
  Does it work in Safari in High Sierra? I thought they had WebRTC now.
  - wodenokoto 9y ago
    That's a good question. My box is too old for High Sierra, so I can't check and didn't think that my browser is now outdated feature-wise, before writing the comment.
api 9y ago
There are lots of them with varying trade-offs between ease of use and power. The problem is that there are none with enough of a user base to have a network effect, and users hate installing new apps. It's 2017 and it's still hard and somewhat dangerous to install apps.
vel0city 9y ago
With the recent shuttering of AIM, I'm reminded of the old client's Direct IM feature which would allow you to transfer arbitrarily large files through the chat session. That tool could not have been simpler to the user, as they just had to click "Direct IM" and could then drag and drop files into the chat. However, that system still relied on the cloud service (AIM) being a broker of connecting a handle to an IP address, and that there was little guarantee of the identity of the person on the other end of the IM.

bubblethink 9y ago

Both parties would have to be online for any p2p solution to work. And if they are online, there are plenty of ways to create a p2p link. It is complicated by things like NAT though. However, that is the less common case. In most cases, the receiver isn't going to be online. So you need an intermediate server, and that's how you end up with third party solutions.

wybiral 9y ago
Yeah, the "being online" detail seems like the problem. It would be best if the encrypted data were distributed (something like IPFS) but then why would peers host files that aren't for them.
- thatcat 9y ago
  You would need a seedbox to function as a cloud. Then when you send a file, send to seedbox simultaniously. This approach might also increase dl speed for the reciever.
- C0DEHERO 9y ago
  They don't currently, but Filecoin is supposed to create an incentive for people to host other people's files. Obviously, you'll have to pay some amount to have your files hosted.
  - m-p-3 9y ago
    Storj is based on a similar concept.
- akavel 9y ago
  AFAIR, there's already some service on IPFS which will pin files for you for money. I.e.: why would someone? Because you'd pay them for that - simple :)
juliendorra 9y ago
Smartphones in Europe are close to being online 24/24. Energy management by OSes might limit apps in what they can do, of course. In any case, 4G networks rise might lead to interesting new solutions.
z3t4 9y ago
If you can assemble IKEA furniture you can also run a Linux server.

AndrewDucker 9y ago

You're at a computer. You need to connect to another computer. First you need to know a route to that computer. If it has an externally reachable IP address and you know that, then great. If it has an externally reachable IP address and a DNS entry and you know _that_, then also great. If you don't know the IP address or domain name of the other computer then you'll have to do some kind of lookup/exchange to find it. That means some kind of centralised service to provide the lookup functionality. If the other computer doesn't even has an externally reachable IP address then a central service is going to have to act as a connection point which you can both connect to (or provide some other method of helping the two of you connect). I'm not aware of any entirely decentralised system which would allow two computers which are behind NAT to find and then talk to each other. Or any obvious design which would work there.

jchw 9y ago
Two thoughts: - IPv6 kind of helps here, at least if we hope that no NAT standard ever makes it into IPv6. Crossing my fingers. - There does exist at least one NAT hole-punching technique that can traverse two NATs with no central server using ICMP-based holepunching and UDP. Obviously, like all hole punching techniques, it only works on certain kinds of NATs, and firewalls can kill it.
- dingaling 9y ago
  IPv6 doesn't help in corporate environments, where it's often firewalled harder than IPv4. The fact that it doesn't support NAT is seen as a risk in such environments ( which rightly or wrongly consider it as a second level of firewalling ). Some deployments that I've seen use only site-scoped with no globally-routed prefixes. Everything Internetty has to go over IPv4, which makes the Security team's job easier; drop all IPv6 at the DMZ and drop all non-NAT IPv4.
- ac2u 9y ago
  > There does exist at least one NAT hole-punching technique that can traverse two NATs with no central server using ICMP-based holepunching and UDP. Obviously, like all hole punching techniques, it only works on certain kinds of NATs, and firewalls can kill it. I think you're referring to this? Clever hack indeed. https://github.com/samyk/pwnat https://github.com/samyk/pwnat

w4yai 9y ago

I'm not sure it tolerates connection dropping, but I've been using justbeamit for years and it never deceived me : https://www.justbeamit.com https://www.justbeamit.com

throw2016 9y ago

Technically a one to one connection to send files should be simple, just open a socket on both sides or use nc. Of course there is NAT but its not insurmountable. The sending or receiving party has to share their IP. And this can only be real time, there is no scope to store files anywhere. Aside I have recently noticed visits to ecommerce sites sit near permanently on top of my Firefox history than other sites, is Firefox doing deals with Amazon and others, and has this been been disclosed?

unlmtd1 9y ago

IPFS

peterwwillis 9y ago

Well, you can with FTP and UPnP. You the client gets the external IP of the machine, then use UPnP to open ports on the router/firewall, then uses PASV and connects to the destination. The destination runs an FTP server, that uses UPnP to open up the incoming port, and gets its external IP and uses it for PORT mode. The end result is two individuals behind NAT sharing arbitrary files in bulk. You can even tie this into FXP to do syncing with the cloud. Or run a local SFTP server with UPnP and avoid the extra complexity. In terms of finding each other, the client could get its external IP, open up a UPnP port, and provide the user with a QR code or brief snippet to paste into a chat conversation, which the other user would feed to their client to initiate the file transfer. You could do this commercially by providing an external website with which the clients could communicate out of band to set up their transfers (for free), and (for pay) optionally provide a cloud sync service (which would help person A "pre-send" a big file transfer until person B was available to receive it). While some of this is available today, you're right, I don't think there are accessible clients. ICQ, AIM and IRC used to be the solution, as they all did P2P file transfer. But now everything is on the web, so everything sucks.

pingiun 9y ago

> It seems that bitorrent protocols are pretty close, but I don't think there is a seamless client that allows for "magical" point to point transactions. instant.io works pretty well for me, it works on the bittorrent protocol, but over webrtc

hiq 9y ago

magic-wormhole ( https://github.com/warner/magic-wormhole https://github.com/warner/magic-wormhole ) is not so far from that, just add a simple GUI on top of it (or just use the terminal and convince the user that a terminal does not have to be complicated), and you're good to go. I don't know how reliable it is w.r.t. connection drops though.

csdreamer7 9y ago

Local-wise you can use Nitroshare. I use it all the time to quickly send files between my computers. https://nitroshare.net/ https://nitroshare.net/

danieldk 9y ago

Resilio Sync (formerly Bittorrent Sync) pretty much offers that. They use relays in case both peers cannot pierce NAT, but the relays only see ciphertext. They also have apps for Android and iOS, making transfers to mobile devices also possible. Since it is closed-source software, it's up to you to decide whether you trust the software. SyncThing is a free software alternative, but I wouldn't say that it is usable (yet) for the average user.

mapgrep 9y ago
I used BitTorrent Sync for a couple of years and loved it. Has it evolved much under the Resilio banner?
- rodolphoarruda 9y ago
  Me too. I stopped using it after I read somewhere that it could be hacked to include new peers to your connection without your awareness, which is a serious concern. I don't know how legit the claim was... but I opted to avoid the risk.
- ukyrgf 9y ago
  I keep mine updated to the latest version and still love it. The 'Ignore' file that is in the top-level of each share alone is worth the admission (similar to .gitignore). Dropbox/OneDrive/etc. all lose their damn minds if you try to sync a folder with 'node_modules' or 'vendor' to the share.
ReAzem 9y ago
Instead of Resilio Sync, I would suggest that people take a look at Syncthing. They have similar features but Syncthing is open source.
- QasimK 9y ago
  There is no iOS app for Syncthing as far as I can tell

mwest 9y ago

I've found https://github.com/warner/magic-wormhole https://github.com/warner/magic-wormhole really useful, although it only meets 1/3 of your requirements. I'm hoping someone will add a nice GUI and re-transmission. Previously on HN: https://news.ycombinator.com/item?id=14649727 https://news.ycombinator.com/item?id=14649727

staz 9y ago
On the principle it look really nice, but with a quick glance at the code I'm not too sure I would trust it... https://github.com/warner/magic-wormhole/blob/master/src/wormhole/_boss.py#L62 https://github.com/warner/magic-wormhole/blob/master/src/wor...

drzaiusapelord 9y ago

I just want to be able to right-click somewhere and have the system bring up a a simple FTPS or SFTP daemon and have it try UPnP to open a firewall/NAT transversal. This daemon would spin up just for the transfer then shut itself down once done. Even if both parties are behind a firewall that won't respect UPnP, there's UDP hole punching which should usually work. Mozilla would just need to host a server to handle the port number handoffs. I imagine this would handle 90% of the use cases. The rest would use traditional 3rd party hosting, IM transfers, email, etc.

thatcat 9y ago

file.pizza is pretty solid, but uses webrtc so ive had failures when sending to some iOS users. Apparently webrtc is included in safari 11 though.

sunsetMurk 9y ago
thanks for sharing this - very cool.

kyrra 9y ago

https://upspin.io/ https://upspin.io/ is in super early concept stage, but trying to solve file sharing issues. It allows for hosting on multiple platforms and trying to keep control in the hands of the users. https://upspin.io/doc/overview.md https://upspin.io/doc/overview.md

alexnewman 9y ago

Magic wormhole

tjoff 9y ago

And this isn't? (Honest question, hasn't looked at it hard enough yet) I don't disagree with you but I kind of expected a note as to why this wasn't a solution in this context.

hardwaresofton 9y ago
I think he considers this as "tethered to a cloud service". I think a solution to this problem that wasn't tethered would be something like in an IPV6 world where everyone has an IP address, if I could just put in your address, send you a file, and as long as our computers were on/connected to the world wide web, it'd get to you.

stephenboyd 9y ago

This was once a common use of AOL Instant Messenger.

wodenokoto 9y ago

File.pizza is very seamless point to point transfer of arbitrary file sizes. Doesn't work in safari though (haven't tested ie) so maybe it is not general enough for your use case.

api 9y ago
Does it work in Safari in High Sierra? I thought they had WebRTC now.
- wodenokoto 9y ago
  That's a good question. My box is too old for High Sierra, so I can't check and didn't think that my browser is now outdated feature-wise, before writing the comment.

api 9y ago

There are lots of them with varying trade-offs between ease of use and power. The problem is that there are none with enough of a user base to have a network effect, and users hate installing new apps. It's 2017 and it's still hard and somewhat dangerous to install apps.

vel0city 9y ago

With the recent shuttering of AIM, I'm reminded of the old client's Direct IM feature which would allow you to transfer arbitrarily large files through the chat session. That tool could not have been simpler to the user, as they just had to click "Direct IM" and could then drag and drop files into the chat. However, that system still relied on the cloud service (AIM) being a broker of connecting a handle to an IP address, and that there was little guarantee of the identity of the person on the other end of the IM.