8 ms·

Fun with BPF, or, shutting down a TCP listening socket the hard way

26 points by ScottWRobinson 11y ago

majke 11y ago
Wouldn't a temporary "iptables -j REJECT" had been easier? Or even maybe "iptables -m condition --condition <> -j REJECT", which would mean you could control the reject by "echo 1 > file". Then a basic suid program (or sudo rule) that writes zero or one to the file, and you can control everything without a root. Like from the upgrade scripts.
- bartbes 11y ago
  The rest of the blog post seems to imply the application needs to be aware this is happening anyway, so it can clear the backlog, so it may as well be in charge of this. Conveniently, this also means (setuid) root is never involved at any point, whereas it would be for your iptables solution.
sargun 11y ago
Where is the documentation for BPF? For example, let's say I want to mangle packets and change the encapsulation on them. How would I go about doing this? It seems like it's a full-on VM. Is there any description of the language - and a compiler?
- _wmd 11y ago
  The canonical documentation is in the kernel source tree: https://www.kernel.org/doc/Documentation/networking/filter.txt https://www.kernel.org/doc/Documentation/networking/filter.t... You can use tcpdump as a makeshift compiler using it's '-d', '-dd', and '-ddd' options: ~] sudo tcpdump -i en1 -d 'tcp port 13' (000) ldh [12] (001) jeq #0x86dd jt 2 jf 8 (002) ldb [20] (003) jeq #0x6 jt 4 jf 19 (004) ldh [54] (005) jeq #0xd jt 18 jf 6 (006) ldh [56] (007) jeq #0xd jt 18 jf 19 (008) jeq #0x800 jt 9 jf 19 (009) ldb [23] (010) jeq #0x6 jt 11 jf 19 (011) ldh [20] (012) jset #0x1fff jt 19 jf 13 (013) ldxb 4*([14]&0xf) (014) ldh [x + 14] (015) jeq #0xd jt 18 jf 16 (016) ldh [x + 16] (017) jeq #0xd jt 18 jf 19 (018) ret #65535 (019) ret #0 While BPF has STORE instructions, I'm not sure in which context they can be used. It's definitely not intended to be a generic system for rewriting packets
  - duskwuff 11y ago
    If I recall correctly, BPF also explicitly lacks any looping instructions, or instructions that can be used to create a loop. Forward jumps only. :)
smegel 11y ago
Why can't they just use a lock around the accept() like the thundering herd solution, that way the thread shutting down the socket can do so knowing nothing else is accepting new connections at the time?
- _wmd 11y ago
  The process accepting the connections (the kernel) runs entirely asynchronously to the application, and there is no way to signal it to stop accepting new connections except by calling close(), which has the dual effect of triggering the connection resets.

majke 11y ago

Wouldn't a temporary "iptables -j REJECT" had been easier? Or even maybe "iptables -m condition --condition <> -j REJECT", which would mean you could control the reject by "echo 1 > file". Then a basic suid program (or sudo rule) that writes zero or one to the file, and you can control everything without a root. Like from the upgrade scripts.

bartbes 11y ago
The rest of the blog post seems to imply the application needs to be aware this is happening anyway, so it can clear the backlog, so it may as well be in charge of this. Conveniently, this also means (setuid) root is never involved at any point, whereas it would be for your iptables solution.

sargun 11y ago

Where is the documentation for BPF? For example, let's say I want to mangle packets and change the encapsulation on them. How would I go about doing this? It seems like it's a full-on VM. Is there any description of the language - and a compiler?

_wmd 11y ago
The canonical documentation is in the kernel source tree: https://www.kernel.org/doc/Documentation/networking/filter.txt https://www.kernel.org/doc/Documentation/networking/filter.t... You can use tcpdump as a makeshift compiler using it's '-d', '-dd', and '-ddd' options: ~] sudo tcpdump -i en1 -d 'tcp port 13' (000) ldh [12] (001) jeq #0x86dd jt 2 jf 8 (002) ldb [20] (003) jeq #0x6 jt 4 jf 19 (004) ldh [54] (005) jeq #0xd jt 18 jf 6 (006) ldh [56] (007) jeq #0xd jt 18 jf 19 (008) jeq #0x800 jt 9 jf 19 (009) ldb [23] (010) jeq #0x6 jt 11 jf 19 (011) ldh [20] (012) jset #0x1fff jt 19 jf 13 (013) ldxb 4*([14]&0xf) (014) ldh [x + 14] (015) jeq #0xd jt 18 jf 16 (016) ldh [x + 16] (017) jeq #0xd jt 18 jf 19 (018) ret #65535 (019) ret #0 While BPF has STORE instructions, I'm not sure in which context they can be used. It's definitely not intended to be a generic system for rewriting packets
- duskwuff 11y ago
  If I recall correctly, BPF also explicitly lacks any looping instructions, or instructions that can be used to create a loop. Forward jumps only. :)

smegel 11y ago

Why can't they just use a lock around the accept() like the thundering herd solution, that way the thread shutting down the socket can do so knowing nothing else is accepting new connections at the time?

_wmd 11y ago
The process accepting the connections (the kernel) runs entirely asynchronously to the application, and there is no way to signal it to stop accepting new connections except by calling close(), which has the dual effect of triggering the connection resets.