2 ms·
If you browse some of the _example.txt files in https://github.com/iovisor/bcc/tree/master/tools https://github.com/iovisor/bcc/tree/master/tools , you'll see i
by brendangregg 11y ago
If you browse some of the _example.txt files in https://github.com/iovisor/bcc/tree/master/tools https://github.com/iovisor/bcc/tree/master/tools , you'll see it's solving the same problems we used to solve with DTrace, plus a few extra. Here's a couple of the ZFS examples (since we're talking ZFS):
# ./zfsslower
Tracing ZFS operations slower than 10 ms
TIME COMM PID T BYTES OFF_KB LAT(ms) FILENAME
06:31:28 dd 25570 W 131072 38784 303.92 data1
06:31:34 dd 25686 W 131072 38784 388.28 data1
06:31:35 dd 25686 W 131072 78720 519.66 data1
06:31:35 dd 25686 W 131072 116992 405.94 data1
06:31:35 dd 25686 W 131072 153600 433.52 data1
[...]
# ./zfsdist
Tracing ZFS operation latency... Hit Ctrl-C to end.
^C
operation = 'read'
usecs : count distribution
0 -> 1 : 0 | |
2 -> 3 : 0 | |
4 -> 7 : 4479 |****************************************|
8 -> 15 : 1028 |********* |
16 -> 31 : 14 | |
32 -> 63 : 1 | |
64 -> 127 : 2 | |
128 -> 255 : 6 | |
256 -> 511 : 1 | |
512 -> 1023 : 1256 |*********** |
1024 -> 2047 : 9 | |
2048 -> 4095 : 1 | |
4096 -> 8191 : 2 | |
[...]
The current BPF interface we're using (bcc) is Python for the frontend, and C for the backend. It's currently much more verbose than DTrace, and involves writing 10x the lines of code. For some immediate use cases at Netflix, that's not a big problem, as staff will be using BPF via a GUI (Vector), not writing these tools directly.
There's also high level features it's still missing (like tracepoints and sampling), so what will be in Ubuntu 14.04 won't do everything, but it will do a fair amount: most of those _example.txt's. Some use a newer BPF interface (linux 4.5), and we've been putting the legacy versions in an /old directory specifically for Ubuntu 16.04 users.
- cthalupa 11y agoDoes the current publicly released version of Vector support BPF? Or is there perhaps a PMDA that allows BPF support? I'm following along with all of this pretty excitedly, and crossing my fingers for a Linux tracing book with BPF, ftrace, perf, etc. to read through and keep on my shelf next to your performance and dtrace books ;)
- brendangregg 11y agoVector doesn't have BPF support yet. When it does we should be open sourcing it. As for a book, I'll try to get it done. :)