5 ms·

> Linux is still dicey to correlate traffic with a particular app, especially if the connection is/was shortlived. This is has become a lot easier and more rel

> Linux is still dicey to correlate traffic with a particular app, especially if the connection is/was shortlived. This is has become a lot easier and more reliable to do now with BPF [0]. I also used the same approach to create a somewhat user-friendly TUI and web dashboard for it [1]. It is also able to hash the executable (even if it was shortlived). [0] https://www.gcardone.net/2020-07-31-per-process-bandwidth-monitoring-on-Linux-with-bpftrace/ https://www.gcardone.net/2020-07-31-per-process-bandwidth-mo... [1] https://github.com/elesiuta/picosnitch https://github.com/elesiuta/picosnitch

nathants 4y ago
ebpf used in this way is typically nonblocking. it drops data instead of blocking. for a firewall, that’s potentially bad. can ebpf operate in a blocking manner while making drop/allow decisions on packets WITH reliable access to the callers pid and argv?
- elesiuta 4y ago
  > for a firewall, that’s potentially bad I wasn't talking about using it as a firewall, just a connection/bandwidth monitor that correlates traffic with a particular app.
  - nathants 4y ago
    bandwidth monitor use case seems like a perfect fit, and the occasional missed packet wouldn’t be an issue. picosnitch looks really cool! i’ve rss subscribed to its github commits.
    - elesiuta 4y ago
      Thanks! Also I used lost_cb [0] to detect if a packet or connection (with security_socket_connect) was missed between the BPF and Python parts, but is it possible for the BPF program to miss either entirely without triggering that callback? If so (without a kernel vulnerability which should be a given) I'd like to have it mentioned under the limitations section for picosnitch so others can be aware as well. [0] https://github.com/iovisor/bcc/blob/master/docs/reference_guide.md#2-open_perf_buffer https://github.com/iovisor/bcc/blob/master/docs/reference_gu...
      - nathants 4y ago
        i don’t think so. i think exactly what you’ve documented is the case. if the callback can’t keep up with the data before the ringbuffer overflows, data is lost. in that case, the solution is to increase the size of the ringbuffer, giving the callback a larger window to keep up with incoming data bursts. in the end there are only two ways to handle this: drop data or block. for a bandwidth monitor, i’d choose drop. for a firewall, i’d choose block. i use bpftrace to monitor docker filesystem access in a similar way[1]. i also increase the ringbuffer size until i stop seeing lost data. 1. https://github.com/nathants/docker-trace#files https://github.com/nathants/docker-trace#files
- metadat 4y ago
  Very cool! I also found it recently submitted to HN: https://news.ycombinator.com/item?id=31160863 https://news.ycombinator.com/item?id=31160863
- yencabulator 4y ago
  I think OpenSnitch just uses eBPF to capture metadata about the new connection, the "hold for user confirmation" does not happen inside eBPF. The eBPF does not need to block waiting for user input. https://github.com/evilsocket/opensnitch/blob/4ce8b0e57cfb2587bd7cf85c275de40e4c2478c9/daemon/conman/connection.go#L88-L97 https://github.com/evilsocket/opensnitch/blob/4ce8b0e57cfb25... https://github.com/evilsocket/opensnitch/blob/d9e0c59158ddf6ba7346bcbc984ee4243857668b/daemon/firewall/iptables/rules.go#L56-L59 https://github.com/evilsocket/opensnitch/blob/d9e0c59158ddf6...
  - nathants 4y ago
    by the time a user prompt is displayed, all metadata has been joined to the packet, and libnetfilterqueue is waiting for a decision. my understanding of what is happening is as follows: - 1: libnetfilterqueue gets a packet. - 2: lookup packet via ebpf to get pid and argv. - 3: lookup rule, on miss prompt user to allow/deny. - 4: libnetfilterqueue allows/denies the packet. the tricky part is step 2. afaik there are 3 possibilities: - ebpf already knows about about this packet, return pid and argv. - ebpf will know about this packet shortly, wait, then return pid and argv. - ebpf dropped the data because of ringbuffer overflow, wait, then give up.
    - yencabulator 4y ago
      My understanding: ebpf runs at connect(2) time, stores metadata in a map; later, an NFQUEUE userspace helper fetches the metadata from the map.
      - nathants 4y ago
        this is correct. it's the datarace between these components, and the possibility of ebpf dropping data because of a slow userspace callback, that makes this flaky.

nathants 4y ago

ebpf used in this way is typically nonblocking. it drops data instead of blocking. for a firewall, that’s potentially bad. can ebpf operate in a blocking manner while making drop/allow decisions on packets WITH reliable access to the callers pid and argv?

elesiuta 4y ago
> for a firewall, that’s potentially bad I wasn't talking about using it as a firewall, just a connection/bandwidth monitor that correlates traffic with a particular app.
- nathants 4y ago
  bandwidth monitor use case seems like a perfect fit, and the occasional missed packet wouldn’t be an issue. picosnitch looks really cool! i’ve rss subscribed to its github commits.
  - elesiuta 4y ago
    Thanks! Also I used lost_cb [0] to detect if a packet or connection (with security_socket_connect) was missed between the BPF and Python parts, but is it possible for the BPF program to miss either entirely without triggering that callback? If so (without a kernel vulnerability which should be a given) I'd like to have it mentioned under the limitations section for picosnitch so others can be aware as well. [0] https://github.com/iovisor/bcc/blob/master/docs/reference_guide.md#2-open_perf_buffer https://github.com/iovisor/bcc/blob/master/docs/reference_gu...
    - nathants 4y ago
      i don’t think so. i think exactly what you’ve documented is the case. if the callback can’t keep up with the data before the ringbuffer overflows, data is lost. in that case, the solution is to increase the size of the ringbuffer, giving the callback a larger window to keep up with incoming data bursts. in the end there are only two ways to handle this: drop data or block. for a bandwidth monitor, i’d choose drop. for a firewall, i’d choose block. i use bpftrace to monitor docker filesystem access in a similar way[1]. i also increase the ringbuffer size until i stop seeing lost data. 1. https://github.com/nathants/docker-trace#files https://github.com/nathants/docker-trace#files
metadat 4y ago
Very cool! I also found it recently submitted to HN: https://news.ycombinator.com/item?id=31160863 https://news.ycombinator.com/item?id=31160863
yencabulator 4y ago
I think OpenSnitch just uses eBPF to capture metadata about the new connection, the "hold for user confirmation" does not happen inside eBPF. The eBPF does not need to block waiting for user input. https://github.com/evilsocket/opensnitch/blob/4ce8b0e57cfb2587bd7cf85c275de40e4c2478c9/daemon/conman/connection.go#L88-L97 https://github.com/evilsocket/opensnitch/blob/4ce8b0e57cfb25... https://github.com/evilsocket/opensnitch/blob/d9e0c59158ddf6ba7346bcbc984ee4243857668b/daemon/firewall/iptables/rules.go#L56-L59 https://github.com/evilsocket/opensnitch/blob/d9e0c59158ddf6...
- nathants 4y ago
  by the time a user prompt is displayed, all metadata has been joined to the packet, and libnetfilterqueue is waiting for a decision. my understanding of what is happening is as follows: - 1: libnetfilterqueue gets a packet. - 2: lookup packet via ebpf to get pid and argv. - 3: lookup rule, on miss prompt user to allow/deny. - 4: libnetfilterqueue allows/denies the packet. the tricky part is step 2. afaik there are 3 possibilities: - ebpf already knows about about this packet, return pid and argv. - ebpf will know about this packet shortly, wait, then return pid and argv. - ebpf dropped the data because of ringbuffer overflow, wait, then give up.
  - yencabulator 4y ago
    My understanding: ebpf runs at connect(2) time, stores metadata in a map; later, an NFQUEUE userspace helper fetches the metadata from the map.
    - nathants 4y ago
      this is correct. it's the datarace between these components, and the possibility of ebpf dropping data because of a slow userspace callback, that makes this flaky.