22 ms·

Programming with pcap (2002)

48 points by poindontcare 12y ago

tcas 12y ago
I've found libpcap to be a very simple API to use given the power of what it's able to do. However, if you're trying to work with things like TCP streams, writing the reordering logic can be a huge timesink. The only non GPL solutions I've found are libntoh[1] and libtins[2]. Both are nice, but libtins is much more powerful out of the box, taking care of the pcap code as well. There's one or two things I don't like with the API of libtins, but overall it's a pretty nice library if you want to capture / inject packets. [1] https://github.com/sch3m4/libntoh https://github.com/sch3m4/libntoh [2] http://libtins.github.io/ http://libtins.github.io/
- tptacek 12y ago
  Vern Paxson taught me a very simple trick to implementing TCP reordering: For each TCP flow (each side of an open connection), open a file. Remember the initial sequence number (ISN) for the flow. Now, when a new segment comes in, use its seqno delta the ISN to seek to the correct offset in the file; write the segment contents to that location. This is, like, 10 lines of code. Naturally, in a production application, you won't really use FILE-stars or fds to track connections. But it's very straightforward to build an array-based ADT that provides the open/close/read/write/seek semantics of a file --- much simpler than TCP. Meanwhile, you get to start with working code to test against.
  - theoh 12y ago
    This reminds me of David Clark's interesting algorithm for datagram reassembly. It puts information about the remaining holes in the datagram into the spaces themselves. http://tools.ietf.org/html/rfc815 http://tools.ietf.org/html/rfc815
jgable 12y ago
I was recently trying to write my own TCP stack to solidify my networking knowledge, similar to Julia Evans's efforts [1]. I got frustrated working with raw sockets on my mac, because they don't provide the same level of control as raw sockets on linux [2]. Is pcap a better level of abstraction that is cross-platform? I don't really care about windows, but it would be nice to work on linux and osx. [1] http://jvns.ca/blog/2014/08/12/what-happens-if-you-write-a-tcp-stack-in-python/ http://jvns.ca/blog/2014/08/12/what-happens-if-you-write-a-t... [2] http://stackoverflow.com/questions/6878603/strange-raw-socket-on-mac-os-x http://stackoverflow.com/questions/6878603/strange-raw-socke...
- tptacek 12y ago
  Yes, pcap is the right abstraction to program bare-metal network code to. It's how most non-kernel-resident tools that implement their own TCP actually work.
- tpush 12y ago
  If you only care about OS X, bpf(4) provides a very similar but nicer API for intercepting and sending raw packets. You could even go zero-copy without the hell that is Linux's 3 differently broken AF_PACKET mechanisms. I made a small example program[0] which just intercepts all packets and prints out statistics. The pthread stuff is only there to asynchroniusly handle signals without having to check every syscall for EINTR. Makes it possible to get statistics at run time by pressing control+T, however. [0] https://github.com/thasenpusch/bpf-example/blob/master/main.c https://github.com/thasenpusch/bpf-example/blob/master/main....
  - tptacek 12y ago
    pcap is an abstraction over bpf; in fact, the two were originally developed in tandem.
mattgreenrocks 12y ago
libpcap is a lot of fun to play with. Writing a simple protocol dissector, starting with IP, then TCP, then app-level protocols is a good project. I had to do something similar many moons ago. You'll get a sense of what's actually being put on the wire, and what your networking stack is doing for you behind the scenes.
- peterwwillis 12y ago
  With the exception that you don't learn about the OS glue that's used behind the scenes by libpcap. Working with the sockets themselves will get you more intimate with the systems you're programming for versus just the protocols. (For embedded systems without libpcap this may be a requirement)
  - mattgreenrocks 12y ago
    Sometimes the trick to pet projects is scoping them just so. You can always start with pcap and then peel it off later.

48 points by poindontcare 12y ago

tcas 12y ago

I've found libpcap to be a very simple API to use given the power of what it's able to do. However, if you're trying to work with things like TCP streams, writing the reordering logic can be a huge timesink. The only non GPL solutions I've found are libntoh[1] and libtins[2]. Both are nice, but libtins is much more powerful out of the box, taking care of the pcap code as well. There's one or two things I don't like with the API of libtins, but overall it's a pretty nice library if you want to capture / inject packets. [1] https://github.com/sch3m4/libntoh https://github.com/sch3m4/libntoh [2] http://libtins.github.io/ http://libtins.github.io/

tptacek 12y ago
Vern Paxson taught me a very simple trick to implementing TCP reordering: For each TCP flow (each side of an open connection), open a file. Remember the initial sequence number (ISN) for the flow. Now, when a new segment comes in, use its seqno delta the ISN to seek to the correct offset in the file; write the segment contents to that location. This is, like, 10 lines of code. Naturally, in a production application, you won't really use FILE-stars or fds to track connections. But it's very straightforward to build an array-based ADT that provides the open/close/read/write/seek semantics of a file --- much simpler than TCP. Meanwhile, you get to start with working code to test against.
- theoh 12y ago
  This reminds me of David Clark's interesting algorithm for datagram reassembly. It puts information about the remaining holes in the datagram into the spaces themselves. http://tools.ietf.org/html/rfc815 http://tools.ietf.org/html/rfc815

jgable 12y ago

I was recently trying to write my own TCP stack to solidify my networking knowledge, similar to Julia Evans's efforts [1]. I got frustrated working with raw sockets on my mac, because they don't provide the same level of control as raw sockets on linux [2]. Is pcap a better level of abstraction that is cross-platform? I don't really care about windows, but it would be nice to work on linux and osx. [1] http://jvns.ca/blog/2014/08/12/what-happens-if-you-write-a-tcp-stack-in-python/ http://jvns.ca/blog/2014/08/12/what-happens-if-you-write-a-t... [2] http://stackoverflow.com/questions/6878603/strange-raw-socket-on-mac-os-x http://stackoverflow.com/questions/6878603/strange-raw-socke...

tptacek 12y ago
Yes, pcap is the right abstraction to program bare-metal network code to. It's how most non-kernel-resident tools that implement their own TCP actually work.
tpush 12y ago
If you only care about OS X, bpf(4) provides a very similar but nicer API for intercepting and sending raw packets. You could even go zero-copy without the hell that is Linux's 3 differently broken AF_PACKET mechanisms. I made a small example program[0] which just intercepts all packets and prints out statistics. The pthread stuff is only there to asynchroniusly handle signals without having to check every syscall for EINTR. Makes it possible to get statistics at run time by pressing control+T, however. [0] https://github.com/thasenpusch/bpf-example/blob/master/main.c https://github.com/thasenpusch/bpf-example/blob/master/main....
- tptacek 12y ago
  pcap is an abstraction over bpf; in fact, the two were originally developed in tandem.

mattgreenrocks 12y ago

libpcap is a lot of fun to play with. Writing a simple protocol dissector, starting with IP, then TCP, then app-level protocols is a good project. I had to do something similar many moons ago. You'll get a sense of what's actually being put on the wire, and what your networking stack is doing for you behind the scenes.

peterwwillis 12y ago
With the exception that you don't learn about the OS glue that's used behind the scenes by libpcap. Working with the sockets themselves will get you more intimate with the systems you're programming for versus just the protocols. (For embedded systems without libpcap this may be a requirement)
- mattgreenrocks 12y ago
  Sometimes the trick to pet projects is scoping them just so. You can always start with pcap and then peel it off later.