9 ms·
Socat – A utility for data transfer between two addresses (2018)
- vesinisa 6y agoThis tool is like netcat on steroids. Definitely part of my stable toolbelt.
- rwmj 6y agoWhenever someone mentions netcat or nc I always ask "which one?". There are at last count 3 different implementations, all subtly incompatible. Which is another reason why socat is better since there's only one to deal with.
- cogburnd02 6y agoHow do you refer to each implementation & what are some of the subtle incompatibilities?
- vesinisa 6y agoThe original one is the "TCP/IP Swiss army knife" released in 1995: https://nc110.sourceforge.io/ https://nc110.sourceforge.io/ (original release: https://seclists.org/bugtraq/1995/Oct/28 https://seclists.org/bugtraq/1995/Oct/28) Although it is "freely given away to the Internet community" with "an obligation to give credit where due", at least OpenBSD and GNU have seen the need to write their own versions under their project licenses: GNU netcat: http://netcat.sourceforge.net/ http://netcat.sourceforge.net/ OpenBSD nc: https://man.openbsd.org/nc.1 https://man.openbsd.org/nc.1 (The OpenBSD version has been ported to at least FreeBSD and Apple Macintosh OS.) All of them have the same basic telnet's `host port` syntax for outbound TCP connections, but annoyingly the syntax for opening a local listening TCP socket varies. Say, you want to open a TCP socket listening on port 1234 (local), and a confirmation when it is ready: The original and GNU netcats: netcat -v -l -p 1234 BSD netcat: nc -v -l 1234
- rwmj 6y agoThat makes 4 implementations, because nmap also has one (https://nmap.org/ncat/ https://nmap.org/ncat/).
- vesinisa 6y agoDammit, they have of course gone with the BSD syntax for opening a local listen socket (ncat -l 1234): https://nmap.org/book/ncat-man-examples.html https://nmap.org/book/ncat-man-examples.html Means half of implemations use `-l -p 1234` and half `-l 1234`.
- cogburnd02 6y agoDoes BSD netcat give an error if you include the -p?
- vesinisa 6y agoYes, the manual explicitly notes that -p can not be used with -l: https://man.openbsd.org/nc.1 https://man.openbsd.org/nc.1 And this is indeed enforced, see line 424: http://cvsweb.openbsd.org/cgi-bin/cvsweb/src/usr.bin/nc/netcat.c?annotate=1.217 http://cvsweb.openbsd.org/cgi-bin/cvsweb/src/usr.bin/nc/netc... However, at least Debian ships a ported version of BSD's netcat. They actually apply a patch to allow using -p and -s with -l for consistency with traditional netcat: https://sources.debian.org/patches/netcat-openbsd/1.217-3/use-flags-to-specify-listen-address.patch/ https://sources.debian.org/patches/netcat-openbsd/1.217-3/us...
- tyingq 6y agoIndeed. There are examples for making local/remote shells. It can make a pty, use setsid() to be a daemon, reset the terminal state, then listen(), etc, all in a one liner. Or route a serial port over ip via ptys in raw mode, again, with a one-liner.
- Datagenerator 6y agoShameless plug [1], socat combined with WireGuard [1] https://github.com/lrvl/tunnel-wireguard-udp2tcp https://github.com/lrvl/tunnel-wireguard-udp2tcp
- maddyboo 6y agoExcellent article! I’ve tried to grok socat a few times but gave up quickly upon being faced with its screenfuls of dense help text. This post sheds light on the core concepts behind the tool in a way that the man page and help text just didn’t do for me. Now that I ‘get’ it, I’m actually excited to find an opportunity to use socat! I can think of so many interesting use cases.
- je42 6y agovery versatile - socat PTY,Link is awesome.
- prussian 6y agovery underrated. I've used this trick before to mock serial ports on some Linux-running industrial devices so I could debug them.
- inivekin 6y agoI made a small utility that'll replay socat log files captured by the -lf -x -v flags with the timing it collects if used in this way. It is very simple and is basically only there if you can't be bothered to roll your own: https://gitlab.com/inivekin/socatplayer/-/releases https://gitlab.com/inivekin/socatplayer/-/releases So if you're commonly doing a socat process like: socat -x OPEN:/dev/ttyS0,b115200,echo=0,icanon=0 PTY,link=/dev/ttyO0,rawer 2> socat_log.log then it may be of interest to you.
- oofnik 6y agoI've used socat in the past to debug all kinds of odd network behavior. It helped me discover a networking issue with some code by comparing performance of piping a Unix socket over TCP using socat vs. the native TCP implementation, which turned out to be caused by an overly conservative default send buffer limit.
- tssva 6y agoI use socat along with npiperelay to allow the ssh client in my WSL sessions to access the Windows ssh-agent.
- strictfp 6y agoInteresting! Do you mind elaborating a bit? I ssh into WSL for this reason.
- nullify88 6y agoLikely something like this https://github.com/rupor-github/wsl-ssh-agent#wsl-2-compatibility https://github.com/rupor-github/wsl-ssh-agent#wsl-2-compatib... Similarly, you can also do this with KeepassXC
- tssva 6y agoI followed the directions at https://stuartleeks.com/posts/wsl-ssh-key-forward-to-windows/ https://stuartleeks.com/posts/wsl-ssh-key-forward-to-windows... to get the combination of socat and npiperelay to work for this purpose. I did slightly deviate from his final script by using pgrep instead of the combination of ps and grep to detect if npiperelay was already running. Edit: These directions are very similar to those linked in the comment this is in reply to and either will work.
- dataflow 6y agoI haven't tried on WSL2, but on WSL1 this script also lets you integrate with KeePass's KeeAgent: https://gist.github.com/duebbert/4298b5f4eb7cc064b09e9d865dd490c9 https://gist.github.com/duebbert/4298b5f4eb7cc064b09e9d865dd...
- shawnz 6y agoIn the past I also used https://github.com/benpye/wsl-ssh-pageant https://github.com/benpye/wsl-ssh-pageant to go from WSL to Pageant (which I believe KeeAgent is also compatible with)
- jonenst 6y agoWe use socat to buffer logs sent from containers (docker-compose) during the short window at startup where logstash and elasticsearch are not ready yet. Works great!
- nwjsmith 6y agoHave you happened to blog about this? I'd love to be able to fix our setups with this trick!
- deleted 6y ago[deleted]
- mrspuratic 6y agoSimilar: using it to get syslog out of self-chrooting ssh with minimal trauma: socat -u UNIX-RECV:/home/sftponly/user/dev/log,mode=666 UNIX-SENDTO:/dev/log It's also an CLI IMAPS client, with history: socat READLINE,history=$HOME/.imaps_history EXEC:'"openssl s_client -connect mailserver:993"' or a CLI web browser with history: socat -d -d READLINE,history=$HOME/.http_history TCP4:www.domain.com:www,crnl or, if you have to interact with something that has no readline, say sendmail: socat READLINE EXEC:"sendmail -bt"
- sedachv 6y ago> if you have to interact with something that has no readline See also https://github.com/hanslub42/rlwrap https://github.com/hanslub42/rlwrap
- lukax 6y agoSocat is the default implementation for Kubernetes port-forwarding [1] when using dockershim for the kubelet runtime. Similar to how tar is used as the default Kubernetes cp command implementation. [1] https://github.com/kubernetes/kubernetes/blob/770d3f181c5d7ed100d1ba43760a74093fc9d9ef/pkg/kubelet/dockershim/docker_streaming_others.go#L42 https://github.com/kubernetes/kubernetes/blob/770d3f181c5d7e...
- Bluecobra 6y agoGreat article! I use socat all the time to test multicast subscriptions in Linux.
- xrd 6y agoDo you have any links/writeups to how you do this?
- justin_oaks 6y agoI often use a socat docker container to proxy network traffic to docker containers. It effectively allows creating dynamic docker port mappings. For example, I have a database with no docker port mappings (only a web application container in the same docker network accesses it directly). I can use a socat container with port mappings to proxy network traffic to the database.
- barbazoo 6y agoInteresting, what's the benefit over exposing the port through Docker?
- throwaway888abc 6y agoSecurity https://en.wikipedia.org/wiki/DMZ_(computing) https://en.wikipedia.org/wiki/DMZ_(computing)
- justin_oaks 6y agoIt is exposed through docker, just via a separate container. The reason for doing so is that a container can't have new port mappings added after it is created. Using the socat container allows me to access a port on my original container that I wouldn't be able to access unless I recreated that container. As for why the database didn't have port mappings originally, it is best not to allow network access to things unless it's necessary. It's the principle of least privilege.
- fguerraz 6y agoI use socat a lot, especially when wanting to transfer files without incurring the ssh encryption overhead or rsync daemon setup. Just tar | scocat and voilà, Superfast whole directory tree transfer. Sprinkle a bit of mbuffer or openssl on top and you can do a lot!
- ldiracdelta 6y agoI use these two socat commands for MITM all the time on my own stuff so I can see the traffic: socat -v TCP-LISTEN:666,forever,reuseaddr,fork TCP:example.com:8080 socat -v TCP-LISTEN:8080,forever,reuseaddr,fork ssl:example.com:443
- jorl17 6y agosocat is an invaluable tool. I first came about it at my first job, where we (just two clueless dudes) had to migrate the mostly undocumented infrastructure of about 15 machines at Digital Ocean to Linode, adding replication and failover on top of that (making it ~35 or so machines). During that migration, we tried to minimize downtime, so we made sure that requests made to the old infrastructure were aptly forwarded to the new infrastructure wherever possible, until DNS properly propagated. I found out about socat and we went for it. It worked great, and I marveled at the simplicity of the solution! Ever since then, I've used socat both as an emergency saviour and as an actual service. I seem to recall using it once to go around some docker bug or shenanigans.
- bombcar 6y agoThis is a brilliant idea and I’ve never even considered doing that form of port forwarding. I’ll have to remember it for the next time I migrate.
- cowmix 6y agoFinally socat is getting the luv it deserves.
- prhn 6y agoI use socat daily. Nearly all of the microservices we develop have a way to be run locally with all dependent services using docker-compose. The problem is most teams don't provide a way to do efficient local development for their particular service. Some do remote debugging into the built/running container (slow and requires rebuilding docker image for every change). Some inspect logs to debug (yikes!). When I want to debug a particular service in IntelliJ, I simply remove it from the docker network and replace it with a container running socat, forwarding the appropriate port to host.docker.internal (the host machine). This is on a MAC, of course, where Docker runs in a VM. This allows me to do in-IDE debugging of any service that runs in the docker network without having to rebuild the image every time I make a change. I know there are IntelliJ plugins that provide a similar workflow, but I believe they all build an image and deploy a container to the network for every change. I could be wrong.
- tomerbd 6y agoInteresting .. Can you provide an example?
- robbles 6y agoHow do you swap out the container in docker-compose? Do you just have a service defined for each, and only start one of them?
- e12e 6y ago> remove it from the docker network and replace it with a container running socat, forwarding the appropriate port to host.docker.internal (the host machine Would you happen to have a fully worked example of this? We to have some services that we can and do run via docker compose - but it's often a cumbersome process to move one and one service out of the docker compose setup for debug/devel.
- cperciva 6y agoA similar tool: spiped[0]. Connects two arbitrary TCP4, TCP6, or UNIX socket addresses, encrypting + authenticating data using a preshared key. For many applications it's basically socat + security. [0] https://www.tarsnap.com/spiped.html https://www.tarsnap.com/spiped.html
- qbasic_forever 6y agoHave they done any audits or had critical analysis of their security code? These days I'm super hesitant to consider anything that hasn't had a lot of eyes on it. It's just too easy for a tiny screwup in protocol design/implementation, random number generation, etc. that puts everything at risk. Why risk it when I can just use wireguard.
- boulos 6y agocperciva is the author of spiped and a security expert [0]. But your question about an audit is still a good one! Tarsnap has bug bounties, but the top hit for “spiped security audit” led to a sort of amusing flame war on HN [1]. [0] https://www.tarsnap.com/about.html https://www.tarsnap.com/about.html [1] https://news.ycombinator.com/item?id=7539903 https://news.ycombinator.com/item?id=7539903
- cperciva 6y agoLots of people have looked at the spiped source code; it has been around since 2011 and I've even handed out a few bug bounties. It's also considerably simpler than wireguard, which reduces the potential for vulnerabilities; and operates on a per-connection basis, which means you can do things like establishing a secure connection to another host but have a socket endpoint which can only be accessed by certain users.
- nirav 6y ago"lots of people looking at source code" is contemporarily a weak argument against audit and review by experts, I really love spiped.
- AnotherGoodName 6y agoIt talks serial too. I've used socat to send out data to the network from a set of serial connected devices for environmental modelling. Ended up being one of those jobs I got done from end to end in an afternoon with the socat part taking 10mins
- megous 6y agoOh! I might as well use it for line editing on serial line with the readline local and serial remote addresses, for example when talking AT to a modem. Great! :) And with history and all that stuff, too! I guess it can be used to forward serial port over network too.
- rgovostes 6y agoI'll join the chorus praising socat, but also share my gratitude for Gerhard, the developer / maintainer. We exchanged emails back and forth about a use case I had, which was to split a serial device into two PTYs, allowing a process to control one while I monitor the other with picocom. We couldn't get it to work, so I gave up, but then Gerhard wrote back a few months later saying he added it as a feature. It can be frustrating to figure out the right incantation for some advanced use cases, but this is more a complaint about esoteric corners of the operating system than socat which exposes the full complexity to you. One of the invocations I recently worked out was to wrap an interactive process with a PTY, so that I could make a mock serial device that emulates a piece of hardware. socat PTY,link=/tmp/dev,rawer,wait-slave \ EXEC:"./emulator",pty,setsid,ctty,echo=0
- iJohnDoe 6y agoIs it possible to use Socat (or any utility) to redirect a UDP connection on a local Windows machine to TCP port 443/80 to a remote system listening on 443/80 that will connect to the original intended UDP connection? I can also do port redirects with iptables that can assist with solving the problem. I need to host a microservice that uses UDP which won’t connect out on most corporate networks due to firewalls. I need to get the connection out via a standard TCP port. Nothing malicious. It’s a basic corporate client/server app, but it’s bound to a UDP port that I can’t change.
- cyberpunk 6y agoI think your best bet is to setup a simple WireGuard for this, also udp but put the server on 53 or something it’s unlikely to be blocked.
- cyberpunk 6y agoOh or skip that abd just use pen ( http://siag.nu/pen/ http://siag.nu/pen/ ) to get the traffic from 53->whatever port (can’t remember if haproxy supports udp)
- alexeldeib 6y agohuh. I haven't heard of this one. Seems like it's been around a while? Why might I use this compared to something more well known? haproxy doesn't seem to do generic udp, but maybe linux virtual server or something like that?
- cyberpunk 6y agoshrug -- it's just something I've used to do it, if it's a single server then I guess a simple iptables rule would also do the trick :}
- axiolite 6y agoSocat is terrible. When used to establish a persistent connection to a remote listening (TCP) port, it has NO WAY to detect that the remote end has dropped the connect, and automatically reconnect. It just does not work. Yes, I know all about all the keep-alive and retry options in the manual. Try it yourself. It's one of those horrible tools that you can get up and running quickly, but can never make work reliably.
- bandie91 6y agoi doubt that socat did not handle TCP RST. is not it the network environtment which eats TCP RST in some cases (like AWS NLB).
- axiolite 6y agoThat's why I said try it yourself... It's no good for serious use. Here's the first search result: https://stackoverflow.com/questions/9341254/reconnect-socat-connection-after-network-failure https://stackoverflow.com/questions/9341254/reconnect-socat-...
- sundbry 6y agoI don't think you understand how TCP works. A TCP connection can stay alive through an unplugged Ethernet cable if it hasn't sent any packets.
- axiolite 6y agoI've worked as a senior network admin for some very large organizations. Safe to say I understand TCP.