8 ms·
Tunnel TCP Through WebSockets (CLI Tool)
- toomim 10y agoWoohoo! Someone built TCP built on WebSockets built on HTTP built on TCP This is like where newborn babies and the elderly have a lot in common.
- slau 10y agoWebSockets aren't built on top of HTTP. The initial handshake resembles that of HTTP, but once the connection is upgraded, all compatibility is lost. In other words, WebSockets is a TCP protocol that mimicks HTTP during the handshake in order to be left alone by proxies and whatnot.
- peterwwillis 10y ago(edit) WebSockets isn't built on top of HTTP, but it is effectively an extension of HTTP. Not officially of course - then they would have had to get people to agree to an extension, so instead they called it a new protocol. But basically it's an HTTP request that turns into an obfuscated socket.
- toast0 10y agoWhen WebSockets is negotiated on an HTTP connection, it ceases to be an HTTP connection; there is no way back to HTTP without closing the connection. It's kind of like when you start ppp, slip, or slirp on a shell you dialed into -- it's no longer a shell with a virtual terminal, it's changed into an internet connection, and you (usually) can't go back without dropping carrier and reconnecting.
- peterwwillis 10y agoThere is really no such thing as an HTTP connection. There are request-response transactions, and these can happen without needing to break down the stateful connection they're transmitted over. Ahem: https://en.wikipedia.org/wiki/HTTP_tunnel#HTTP_CONNECT_tunneling https://en.wikipedia.org/wiki/HTTP_tunnel#HTTP_CONNECT_tunne... In this mechanism, the client asks an HTTP Proxy server to forward the TCP connection to the desired destination. The server then proceeds to make the connection on behalf of the client. Once the connection has been established by the server, the Proxy server continues to proxy the TCP stream to and from the client. Note that only the initial connection request is HTTP - after that, the server simply proxies the established TCP connection. Both HTTP CONNECT and WS are designed to work over http proxies, both are designed to tunnel arbitrary application data, both use http to initiate a connection, both use http application ports for these connections, and neither of them return to http request-responses once they're set up. But somehow, WS is a separate protocol and stops being http, while HTTP CONNECT is just an HTTP extension and becomes some other application proxied over http. I guess WS is just special.
- pjmlp 10y agoI see some potential to work around corporate firewalls.
- pekk 10y agoIf you had made this comment to the introduction of HTTP itself, I would today be hailing you as a visionary
- bitexploder 10y agoFirst, as always hats off to the authors for publishing code and hacking on stuff for us all to use. That said...You could just use corkscrew. https://wiki.archlinux.org/index.php/HTTP_tunneling https://wiki.archlinux.org/index.php/HTTP_tunneling Corkscrew + SSH has been helping individuals escape restrictive corporate networks for years. Pretty sure SSH for remote server on port 443 plus corkscrew offers more than this solution. (Socks proxy, etc. Very flexible. No need for DNS muckery you would need with this solutions example etc.)
- paulddraper 10y agoWeb browser JS can't use HTTP tunneling, unfortunately. And it's all about the web browser, so...
- derhuerst 10y agoThanks for the tip! I stumbled upon corkscrew when quickly researching possible solutions, but eventually, I didn't have a second look at it.
- Buge 10y agoInstead of stuff inside websockets inside TLS, wouldn't it be more efficient to just do stuff inside TLS? It's encrypted so firewalls shouldn't be able to tell the difference.
- pyvpx 10y agoyou'd be surprised what a "deep packet inspection" (better known as DPI or L7-aware) firewall can tell. some are quite sophisticated, and there are a surprising number of techniques to identify even encrypted traffic (not the precise payload, per se)
- Buge 10y agoWell there's ALPN/NPN, but you could lie about those. I would indeed find it surprising if any commercial firewalls distinguish websockets over TLS versus raw bytes over TLS. Not that it's impossible, but I wouldn't think it would be done commercially.
- peterwwillis 10y agoFirewall packet inspection is basically just unpacking protocols layer by layer. Once you find the TCP or HTTP layer, you see if what follows looks like a WebSocket, which has a unique fingerprint and predictable elements. You can also look at streams and find patterns, like how one packet looks like HTTP, and then the following ones on the same stream don't look like HTTP. Initial versions may not be very accurate, but on "extremely paranoid" settings you can simply reject traffic that seems suspicious. Also, having the DoD as a client will buy you some serious R&D time.
- Buge 10y agoYou can't unpack protocols layer by layer when they're encrypted.
- peterwwillis 10y ago
- Lerc 10y agoMy first thought when I saw this was "Hey! I made one of these". Turns out it goes the other way. From websocket to TCP. The thing I made was something where you connect to a socket(typically a unix domain socket in /tmp) and it connects to a web page via websockets.
- chrisallick 10y agoBefore I ask my question, let me ask a "am i right or wrong." WebSockets is a small layer on top of HTTP. And HTTP is a protocol on top of TCP. Right or wrong? So then why would do this? To get TCP running over port 80 to get through a firewall? Why not just do TCP over port 80? Like this project https://github.com/jpillora/chisel https://github.com/jpillora/chisel Edit: above project is also using websockets. okay, i think i understand why you would do this.
- icebraining 10y agoWebSockets is not really a layer over HTTP as such; it uses an HTTP request format to set up the connection, but then it's mostly just a raw socket (with message framing, like in UDP). Your question is still valid, I'm just dispelling the myth that WS is like TCP-over-DNS or something similarly inefficient.
- peterwwillis 10y agoIt's almost exactly like TCP-over-DNS. Both are stateful protocols on top of stateless ones. The only difference is HTTP responses can essentially go on forever, whereas DNS ones are typically UDP with an upper bound on size, and TCP ones have a timeout on the order of seconds, and thus need to re-encapsulate further messages. The biggest difference there is that while TCP-over-DNS just has TCP->DNS->UDP as overhead, the WS method has TCP->WS->TCP. Because the connection is stateful and the WS encapsulation is minimal, it's more efficient (and HTTP tunneling is almost always more functional than DNS or ICMP, anyway) It's still inefficient. And it's not a raw socket, it's an application layer socket, essentially.
- icebraining 10y agoAre you talking about WS itself, or this tunnel thing? Because I was just talking about WS itself, which doesn't have any TCP over it, it's just framed data over regular TCP. Anyway, looking at the code of this tool, it doesn't seem like there's any actual TCP being tunneled; from what I can tell, the raw TCP terminates in the application, and the data is piped directly into the WS connection. That's why you have to choose a static target, whereas with a TCP tunneling system like iodine, you can use it as an open gateway. Fair enough on the raw socket, I meant a TCP socket.
- deleted 10y ago[deleted]
- wfunction 10y agoNot related to the post, just a general question about WebSockets: can someone explain why long-polling was insufficient?
- Filligree 10y agoAt a minimum, long-polling requires a server round-trip per message. Among other problems, this limits throughput to (message size / latency).
- wfunction 10y ago> At a minimum, long-polling requires a server round-trip per message. Among other problems, this limits throughput to (message size / latency). I don't see why? Just have N multiple outstanding requests to the server (with N being large enough to achieve whatever communication rate you can accept). When the server wants to send you something, it picks one and replies to it. While you're busy processing it the server can still reply to anther one and send more data. When you're done you send more outstanding requests to the server to reach N again. And if you ever run out of outstanding requests, then that's equivalent to your socket not being able to accept a new connection because there are too many pending already, which can always happen.
- jkarneges 10y agoYeah, you could have multiple requests hanging open with the server to enable faster server sends (and if a client wants to send data rapidly it could make multiple parallel POST requests). The downside is you end up with a complex stateful protocol, with multiple sockets, and need the ability to handle out-of-order data. If you're targeting a platform that can't do HTTP streaming, then sure, but otherwise this is a pretty gnarly road to go down. My personal preference for long-polling is to use it for stateless / RESTful APIs that have infrequent updates. Super clean, easy to use from the client's perspective, and efficient enough. I would not use it for stateful communication (IMO SockJS and Engine.IO are gross but I understand why they exist).
- jkarneges 10y ago
- BrandiATMuhkuh 10y agoI use since a year https://ondevice.io https://ondevice.io which does ssh and others via websockets. I run it in all my devices an I'm very happy. I was told by the developer that he uses it also to maintain hundreds of machines which don't have a public IP.
- derhuerst 10y agowell the idea is not to trust any company or service or even reveal my password/key to it. my use case was only escaping the restricted wifi, not managing machines.
- jwilk 10y agoSetting a wss:// URL as homepage was a clever idea, but GitHub truncated it without making it clickable. :-\
- simonjgreen 10y agoHow is this any better than the plethora of HTTPS VPNs which have been around for nearly two decades?
- gbuk2013 10y agoApples and oranges. This tool is a proxy not a VPN.
- peterwwillis 10y agoNot really better, just an alternative. Chisel is more than likely much faster anyway and does the same thing. HTTPS VPNs require TLS, and some proxies don't let you use TLS, because you might tunnel with it. It's pretty easy to block most HTTP tunneling because they typically use the CONNECT method. Ones that simply do an HTTP handshake and then pass the socket to a tunneling program can be detected or simply terminated after the session has gone on for a while or transferred a certain number of bytes. BOSH connections are a little more reliable, and I don't know off the top of my head if any tunneling apps support chunked encoding/multipart/etc, but it's possible to implement a tunnel using only valid HTTP methods, but it would be inefficient. Basically, most tunnels can be defeated without negative impact to regular users because HTTP isn't supposed to be used that way. In comes WebSockets. Since it's a part of the modern web, and disabling them would negatively affect user experiences on the web, you can abuse them to tunnel random crap and proxies can't really do anything about it. The protocol even allows for obfuscation to make it difficult to see what's going over the socket (though it was intended to prevent cache poisoning attacks). So in theory, if you can reach a target HTTP server, WebSockets may be the most reliable method to tunnel a connection over HTTP. The fact that it also supports multiple streams may make it faster than alternatives. And the fact that you don't need to use TLS may make it even faster, assuming the app you're tunneling does its own encryption.
- partycoder 10y agoDo not mute errors. It is a bad practice. https://github.com/derhuerst/tcp-over-websockets/blob/master/server.js#L42-L44 https://github.com/derhuerst/tcp-over-websockets/blob/master...
- derhuerst 10y agoi'm aware that it usually is a bad practice. but mostly it is about handling errors as a valid state of the application, whereas in this case i'm not interested in any connection errors whatsoever.
- api 10y agoTCP over web sockets over TCP. Sigh. Of course I develop network virtualization software so I'm just as guilty of it.
- NetStrikeForce 10y agoBig fan of ZeroTier here :) (I also run a semi-related project, without the decentralized magic of ZeroTier)
- dom0 10y agoActually TCP over web sockets [over TLS] over TCP. How's your buffer bloat? — Fine, how's yours? — Great, thanks for asking!
- dreamlayers 10y agoThis is just one TCP connection though, not like a connection to the Internet which you can use to make various TCP connections. You can pass PPP or SLIP via WebSockets and connect to the Internet that way. That's already used for connecting DOSBox running in a web browser to the Internet: http://blog.vrcade.io/2017/03/setting-up-a-visp-using-pppow/ http://blog.vrcade.io/2017/03/setting-up-a-visp-using-pppow/