6 ms·
Combine your Wi-Fi, Ethernet and 3G connections to increase your download speed
- Morhaus 13y agoDispatch-proxy lets you boot a proxy on your machine that dispatches every incoming connection to a different network interface. The possibilities are endless: * download a file through threaded downloading, effectively combining multiple connections' speed in one single file download, * create two proxies, assign to each its own interface, and run two apps simultaneously that use a different interface, * create a hotspot proxy at home that connects through Ethernet and your 4G card for all your mobile devices, * etc. Here are some instructions to quickly get started with using the proxy: 1) INSTALLATION * Install Node.JS (http://nodejs.org/ http://nodejs.org/). * Launch a terminal. * Run the command `npm install -g dispatch-proxy` as a power user and press Enter. It will install the `dispatch` command-line utility. 2) STARTING AN HTTP PROXY * To quickly start an HTTP proxy, run `dispatch start --http`. * The program will print the proxy address and port (by default localhost:8080) as well as the list of local addresses the program will be dispatching requests to. * Configure your system to use the combination address:port as an HTTP proxy. 3) LISTING ALL AVAILABLE NETWORK INTERFACES * To list all available network interfaces, run `dispatch list` * Each available network interface will be printed with a list of addresses. * Right now, the dispatch utility only supports IPv4 addresses. * If an address is marked to be "internal", then you won't be able to use it to connect to the internet. I encourage you to read the docs https://github.com/Morhaus/dispatch-proxy https://github.com/Morhaus/dispatch-proxy.
- cypherpunks01 13y agoDoes this do bonding at the packet level, or at the TCP level? Outcome being, if I download one particular file over HTTP will it run N times as fast, or are you just increasing the total speed available for many connections?
- Morhaus 13y agoAt the TCP level.
- Joona 13y agoI'm guessing this is Linux-only? Would love to use my tablet's connection aswell.
- Morhaus 13y agoWindows, Mac OS X and Linux. Every platform which runs NodeJS in fact.
- Joona 13y agoAh, got confused as you said terminal.
- asmosoinio 13y agoIs that in the readme?
- Morhaus 13y agoAdding it now.
- Jhsto 13y agoSince uncapped 3G is pretty common where I live, I've always thought to build something similar but without never finding the time to do it. This will also be great for load balancing my network the next time I gather to play some online-games with my friends. Any idea will the proxy still be available if one of the connections drops? Like if my broadband cuts out (like it does on a daily basis) will the cellular data automatically take over?
- Morhaus 13y agoFor now, the proxy will still try to dispatch sockets to every network interfaces specified at launch. But such a feature could perfectly be implemented.
- jervisfm 13y agoThis is a neat idea. Does anyone know if it's possible to achieve a similar effect natively on Linux ? i.e. make the linux network stack load-balance all outgoing connections to all available interfaces ?
- mike-cardwell 13y agoI'm pretty sure you can do this sort of load balancing on Linux without having to install any extra software, at the IP level, with the "ip rule" and "iptables" commands. EDIT: Yep: https://home.regit.org/netfilter-en/links-load-balancing/ https://home.regit.org/netfilter-en/links-load-balancing/
- anoncow16 13y agoTL;DR just buy a faster internet connection. It is possible to do per-packet load sharing in Linux as the link above suggests. Both links need to be able to source traffic using the same IP address, i.e. you can't mix comcast and dsl. The way to get around this is to have an upstream router that will source your traffic from the same IP address: client - routerD --upstream1---routerU -- the internets |-------upstream2-----| The iptables statistic module with random mode seems to work better than nth packet. Keep in mind PPS load sharing is not the most effective way to utilize links as retransmissions will slow things down. Now if you have two links to the same provider it's probably OK. You can even buy a bonded connection from most providers where they do all of this for you. You are much better off doing flow based ECMP routing: http://lkml.indiana.edu/hypermail/linux/net/0207.2/0047.html http://lkml.indiana.edu/hypermail/linux/net/0207.2/0047.html ECMP is only useful if you have a large number of clients. So you can do PPS load sharing, knowing full well that it's going to be slower than a single link some percentage of the time or just buy a faster connection. TL;DR again, just buy a faster connection.
- shocks 13y ago> just spend more money Mmmhmm, that's the true hacker spirit right there...
- kclay 13y agoI use Super Download on Android that does the same thing.. Always wondered how they pulled it off.
- bburky 13y agoNice. Interesting that you're doing it at the userland level and balancing across IP address and not interfaces. I may find a use for this. (On a network rate limited by IP address, not per physical connection or per MAC address, and allows me to use multiple IPs. Go figure.) I haven't set this up myself, but I think the iptables method for doing this is `nexthop via`. Never found any great documentation, but this seems to describe it: http://lartc.org/howto/lartc.rpdb.multiple-links.html http://lartc.org/howto/lartc.rpdb.multiple-links.html I think the iptables method balances across interfaces, not IPs. But I want to balance across IP aliases (multiple IPs, same interface). And I'm usually on a Mac. So I may see if dispatch-proxy is a better choice.
- bburky 13y agoIf anyone is interested in my particularly strange use case, this seems to work: $ sudo ifconfig en0 alias 192.168.2.210 255.255.255.0 $ sudo ifconfig en0 alias 192.168.2.211 255.255.255.0 $ sudo ifconfig en0 alias 192.168.2.212 255.255.255.0 $ dispatch start SOCKS5 server started on localhost:1080 Dispatching to addresses 192.168.2.28@1, 192.168.2.210@1, 192.168.2.211@1, 192.168.2.213@1 Also the link I mentioned is different from what other people have mentioned in this thread. It's not interface bonding or TCP multipath. It is actually similar to dispatch-proxy, it balances TCP connections across interfaces as far as I can tell.
- edsiper2 13y agomoderns systems comes with TCP Multipath enabled.
- chacham15 13y agoCan anyone explain how this works? From previous experience I always thought that this was not possible without some cooperation on the other end because the problem is you want to split the data being sent to you to different ips. If the other end sends all the data to the same ip, you cannot redirect/load balance to other ips before the data actually gets to you. What am I missing?
- wmf 13y agoProbably each TCP connection only goes over one IP address, so the remote end doesn't get confused. All the data coming back on that connection will have to come in on the same path the connection started on.
- chacham15 13y agoIf that were the case, the download would be no faster than using a single connection. I guess it would help if you download multiple things, but that is a different claim than what is claimed in the title.
- mikeash 13y agoYou can use multiple connections for a single download (e.g. HTTP range requests).
- guyinblackshirt 13y agois this the same as connectify dispatch? http://www.connectify.me/dispatch/ http://www.connectify.me/dispatch/
- pyed 13y agonow i can use dispatch proxy on application bases like firefox or aria2c, simply by adding the proxy "localhost:8080" to the proxy section on these applications but how to use it system wide ? so apps that don't support the use of proxies will be able to benefit from all this ? edit: I'm using Mac
- bburky 13y agoOn a Mac set it in System Preferences > Network > Advanced... > Proxies > SOCKS Proxy. Many Mac programs will automatically pick it up.
- kevrone 13y agoGod I've been wanting to do something like this for a long time, but just didn't have the networking chops to get started. Glad someone beat me to it! I imagine this would work well, not just on networks you own, but with cooperative neighbors as well. What if an entire floor of an apartment building used the same proxy? Would that work?
- songgao 13y agoWhy not just use Multipath TCP?
- mvzink 13y agoIdeally, in the future, yeah. But for now, two reasons I can think of: first, Multipath TCP doesn't cover the same use cases as dispatch. For example, having one proxy only route to low-latency connections (e.g. for gaming) and another to all connections (e.g. for downloads). Second, Multipath TCP isn't widely implemented and would require application developers to use it, whereas this can give many of the same benefits without waiting for your applications to support it.
- gte910h 13y agoI have used dual wan routers that do this: While speed can go up, and bandwidth certainly does, the latency variance makes many games unplayable, and really throws some video and audio streaming apps for a loop. I'd love to see a write up how well this does stuff that is sensitive to latency and changing network conditions, beyond simple web surfing.
- Morhaus 13y agoOne of the advantages of using a proxy is that you can configure it per app. For example, let's say I have 3 interfaces. Interfaces (1) and (2) are low-latency and interface (3) is high latency. I can start a proxy on localhost:1080 that redirects to interfaces (1), (2) and (3), and configure my download manager to use it. I can then start another proxy on localhost:1081 that redirects to interfaces (1) and (2), and configure my game client to use it.
- codex 13y agoiPhone's Siri does something similar, but for a different reason: reliability. It multipaths each query.
- songgao 13y agoSiri does this in a more systematical way. It uses Multipath TCP, which is in transport layer. This is an application layer hack. This one is actually less convenient and less efficient (tunnelling in proxy adds extra headers and reduces available data space in each packet). EDIT: typos
- MasterScrat 13y agoSo if I am in a residence where I can connect to 3 independant WiFi networks, and have 3 wireless network adapters, I can combine their bandwidth?! because that would be freaking awesome.
- Morhaus 13y agoYup.
- MasterScrat 13y agoYou should totally add this use case to your README. That's the very first things my fellow student friends ask when I show them this.
- Morhaus 13y agoI just did, thanks :)
- OoTheNigerian 13y agoOP, Great job. I am not too technical so pardon my question. Can multiple 3G connections be combined or have a 'random selector'/failover. So where I live (you can make a guess), mobile connectivity is not the most reliable in the world. So for instance, POSs are not reliable since one network will always fail. With this, could we use multiple SIMs thereby making them more reliable? BTW, have you seen BRCK http://www.brck.com/specifications/ http://www.brck.com/specifications/? Seems to have the sort of the same concept although I think yours is combining speed vs. having a fail-over system for internet access.
- Morhaus 13y agoThere is no built-in failover mechanism atm. However I guess you could use another program on top of dispatch to implement such a feature.
- sharjeel 13y agoTake a look at Horde: http://nms.csail.mit.edu/projects/horde/ http://nms.csail.mit.edu/projects/horde/ http://nms.lcs.mit.edu/papers/horde-mobisys-2005.pdf http://nms.lcs.mit.edu/papers/horde-mobisys-2005.pdf
- qq66 13y agoI think Justin.TV used a setup like this in the early days to stream 24/7 live video -- several mobile hotspots from different carriers all working on sending data to the remote server, ensuring that the data got there somehow. The difference was that they controlled the server side.
- jakejake 13y agoAs far as I know this is a DIY solution for what companies sell as a "bonded" internet connection..? I spent weeks trying to get a reliable setup for streaming concerts in HD from all varieties of spotty WiFi and 4G hotspots. The streaming server wouldn't accept packets that seemed to come from totally different IP addresses so there had to be an intermediate proxy server in the cloud that would re-assemble the data from my various connections. I was able to get it working and it would have been great for normal Internet usage, but I never found a reliable system for streaming. That was about a year and a half ago. I would love to see this technology mature to the point where it worked well for streaming.
- drdaeman 13y agoFrom a quick glance over the code, it just seem to proxy TCP streams over one channel or another, probabilistically. Most trivial form of load balancing possible. This approach certainly won't work for typical media streaming, as it usually uses a single connection. Particularly, because streaming over multiple connections is tricky because as they don't guarantee delivery order, so you'll probably get your packets "scrambled" and will have to reorder them by yourself (implementing your own flow control).
- evolve2k 13y agoWould using this improve the connection for Skype/Hangout video calls?
- pertsix 13y agoNo. Simultaneous (locally bonded) connections do not have great enough threading integrity to bridge the two streams together.
- sturmeh 13y agoIs it possible to use one for uploads and one for downloads?
- jon49 13y agoOpen Garden already does this. Plug and chug. Pretty straight forward.