6 ms·

Wireshark is the proverbial hammer that makes all networking problems look like nails. Even if there's a more specific tool available there's a good chance I c

by wolrah 3y ago

Wireshark is the proverbial hammer that makes all networking problems look like nails. Even if there's a more specific tool available there's a good chance I can swing Wireshark at the problem and figure it out. It continues to blow my mind how many people there are in the world who consider themselves networking professionals but have never used or do not understand Wireshark. It is possibly the most important tool for actually understanding what's really happening in your network, without it you're effectively blind to so many things. Just yesterday I used it to troubleshoot a weird behavior in a recently upgraded Asterisk/FreePBX system which would have probably taken me days to guess my way through without packet captures, but with them I was able to see clearly what was happening on the network and then track that back from there. Congrats to everyone on the Wireshark team on 25 years of making network troubleshooting infinitely easier! I would 100% not be where I am today without it.

hartator 3y ago
Fiddle is awesome too.
- protonbob 3y ago
  Fiddler classic is what I always go for.
flashgordon 3y ago
Forget something as specific and hardcore as networking. I had a week to build a nodejs poc of a legacy spring/java app/service in Amazon that was doing a bunch of service to service auth with some Tibco messaging. I couldn't find any open implementations of Tibco clients (around 2013) and the frugality leadership principle meant getting an official spec would be almost impossible. I just needed a few details of the packet structure on a couple of requests. You can guess which tool saved the day for me here! Principle Eng at the time was surprised such a tool existed!
el_benhameen 3y ago
Do you have any favorite sources for really understanding Wireshark? I’m not a networking professional per se, but I’m network-adjacent and I’ve dabbled in Wireshark from time to time. I can see the power, but it’s also one of those tools that’s totally overwhelming when I first approach it unless I have a very small, very specific problem. Or is it one of those tools that you learn as you need it?
- baby_souffle 3y ago
  There’s a million little features and tricks you can do but you’ll never stumble into them unless you’re actively googleing “how do I …”. You might look for some pcap based CTFs with walkthroughs to get exposure to some of the more unique things you can do. Just letting it run for a few min on your router and then powering a device up can also yield some interesting captures…
- wolrah 3y ago
  Unfortunately I can't really help there, I'm a "learn by doing" type of person who just jumps in the deep end and hopes he figures out how to swim. Most of my learning was just "capture the problem happening, capture what happens when it works right if possible, open up the relevant RFCs, then try to understand what's different and why. I work in the VoIP industry so I'm dealing with a lot of NAT problems (insert rant here about lazy ISPs that still haven't enabled IPv6 on their networks) and my main protocol (SIP) is heavily inspired by HTTP and as a result is more or less human readable plaintext, so it was a relatively easy learning curve to just have Wireshark open on one side of the screen and the relevant RFCs on the other side. All I can really say is have a problem you want to solve and start from there.
  - el_benhameen 3y ago
    Sounds reasonable to me, thanks. That’s how I always end up learning, but sometimes I wonder if there’s a better way.
seiferteric 3y ago
It's like a debugger for networking, and surprisingly many programmers don't know how to use debuggers either.
- x86x87 3y ago
  It's more like strace but yeah.
  - auguzanellato 3y ago
    And a lot of programmers don’t know about strace either
    - maximus-decimus 3y ago
      I can confirm, I didn't know about strace until this very moment. Looking at it, it basically only intercepts system calls? How often is that useful? What do people use it for?
      - mitchs 3y ago
        The most fun I've had with strace was debugging a 3-process deadlock. An snmp daemon was blocked waiting for a cli child process to finish, the cli was waiting for a response to a message on a socket it had open with a routing protocol daemon, which was waiting for a response from the snmp daemon. It is also a great way to figure out why programs without useful debug output die. Ie. after a program opens and reads a config file it doesn't like, it starts cleaning up and exits.
      - jeffreygoesto 3y ago
        I recently fired it up to quickly check which headers a crosscompiler used on a specific compilation unit. strace, grep, sort, done. I also use it as first check if something seems to hang. Sometimes you can see lock files trying to be acquired or access to wrong paths.
        3y ago
zvmaz 3y ago
> Just yesterday I used it to troubleshoot a weird behavior in a recently upgraded Asterisk/FreePBX system which would have probably taken me days to guess my way through without packet captures Do you mind sharing with us what was the problem and how you solved it with packet captures, if you have time? A blog post would be very interesting too.
- toast0 3y ago
  For something like this firefox bug [1], getting down to pcaps helps determine where the problem is. Client is spinning on a request and server doesn't know about it could be a server problem or a client problem or a network in the middle problem. In this case, the problem was the client wasn't actually sending the request, and with a sizable request that's visible even without decoding the https; although to be totally clear on what was happening, decoding was needed. I've also debugged issued in remote networks where iirc, connections were being reset by some equipment local to the user. Seq/ack sequencing showed the resets were in response to a specific client sent packet and the timestamps showed it was impossible for that to have come from anywhere but equipment near the user. For this bug [2], it took a lot of luck and patience to get a good capture, but once I did, the immediate problem became obvious: the machine I controlled was getting an icmp needs frag but DF set at the same mtu it was already using, and responding by sending the whole sendqueue at once, packetized to the new MTU that was the same as the old one. There's actually three problems here: a) there's no reason for the other side to send this packet (I found this is an already fixed linux bug with forwarding and large receive offload, but no way to contact the administrator of that router), b) our side shouldn't resend the whole sendqueue when the mtu changes, c) if the mtu didn't change, then there's no need to take any action. We only fixed c, but that solved the major problem: these resends would trigger more resends and we'd have periods of unavailability as the network was really busy. This is pretty common when looking at wireshark; unless you work somewhere with full control of all clients and servers and a very network aware developer team, you're going to find lots of non-optimal or semi-broken stuff, and you've got to ignore it and focus on the majorly broken bit. [1] https://bugzilla.mozilla.org/show_bug.cgi?id=1740856 https://bugzilla.mozilla.org/show_bug.cgi?id=1740856 [2] https://reviews.freebsd.org/rS288412 https://reviews.freebsd.org/rS288412
- 3y ago

by wolrah 3y ago

hartator 3y ago

Fiddle is awesome too.

protonbob 3y ago
Fiddler classic is what I always go for.

flashgordon 3y ago

Forget something as specific and hardcore as networking. I had a week to build a nodejs poc of a legacy spring/java app/service in Amazon that was doing a bunch of service to service auth with some Tibco messaging. I couldn't find any open implementations of Tibco clients (around 2013) and the frugality leadership principle meant getting an official spec would be almost impossible. I just needed a few details of the packet structure on a couple of requests. You can guess which tool saved the day for me here! Principle Eng at the time was surprised such a tool existed!

el_benhameen 3y ago

Do you have any favorite sources for really understanding Wireshark? I’m not a networking professional per se, but I’m network-adjacent and I’ve dabbled in Wireshark from time to time. I can see the power, but it’s also one of those tools that’s totally overwhelming when I first approach it unless I have a very small, very specific problem. Or is it one of those tools that you learn as you need it?

baby_souffle 3y ago
There’s a million little features and tricks you can do but you’ll never stumble into them unless you’re actively googleing “how do I …”. You might look for some pcap based CTFs with walkthroughs to get exposure to some of the more unique things you can do. Just letting it run for a few min on your router and then powering a device up can also yield some interesting captures…
wolrah 3y ago
Unfortunately I can't really help there, I'm a "learn by doing" type of person who just jumps in the deep end and hopes he figures out how to swim. Most of my learning was just "capture the problem happening, capture what happens when it works right if possible, open up the relevant RFCs, then try to understand what's different and why. I work in the VoIP industry so I'm dealing with a lot of NAT problems (insert rant here about lazy ISPs that still haven't enabled IPv6 on their networks) and my main protocol (SIP) is heavily inspired by HTTP and as a result is more or less human readable plaintext, so it was a relatively easy learning curve to just have Wireshark open on one side of the screen and the relevant RFCs on the other side. All I can really say is have a problem you want to solve and start from there.
- el_benhameen 3y ago
  Sounds reasonable to me, thanks. That’s how I always end up learning, but sometimes I wonder if there’s a better way.

seiferteric 3y ago

It's like a debugger for networking, and surprisingly many programmers don't know how to use debuggers either.

x86x87 3y ago
It's more like strace but yeah.
- auguzanellato 3y ago
  And a lot of programmers don’t know about strace either
  - maximus-decimus 3y ago
    I can confirm, I didn't know about strace until this very moment. Looking at it, it basically only intercepts system calls? How often is that useful? What do people use it for?
    - mitchs 3y ago
      The most fun I've had with strace was debugging a 3-process deadlock. An snmp daemon was blocked waiting for a cli child process to finish, the cli was waiting for a response to a message on a socket it had open with a routing protocol daemon, which was waiting for a response from the snmp daemon. It is also a great way to figure out why programs without useful debug output die. Ie. after a program opens and reads a config file it doesn't like, it starts cleaning up and exits.
    - jeffreygoesto 3y ago
      I recently fired it up to quickly check which headers a crosscompiler used on a specific compilation unit. strace, grep, sort, done. I also use it as first check if something seems to hang. Sometimes you can see lock files trying to be acquired or access to wrong paths.
      - 3y ago

zvmaz 3y ago

> Just yesterday I used it to troubleshoot a weird behavior in a recently upgraded Asterisk/FreePBX system which would have probably taken me days to guess my way through without packet captures Do you mind sharing with us what was the problem and how you solved it with packet captures, if you have time? A blog post would be very interesting too.

toast0 3y ago
For something like this firefox bug [1], getting down to pcaps helps determine where the problem is. Client is spinning on a request and server doesn't know about it could be a server problem or a client problem or a network in the middle problem. In this case, the problem was the client wasn't actually sending the request, and with a sizable request that's visible even without decoding the https; although to be totally clear on what was happening, decoding was needed. I've also debugged issued in remote networks where iirc, connections were being reset by some equipment local to the user. Seq/ack sequencing showed the resets were in response to a specific client sent packet and the timestamps showed it was impossible for that to have come from anywhere but equipment near the user. For this bug [2], it took a lot of luck and patience to get a good capture, but once I did, the immediate problem became obvious: the machine I controlled was getting an icmp needs frag but DF set at the same mtu it was already using, and responding by sending the whole sendqueue at once, packetized to the new MTU that was the same as the old one. There's actually three problems here: a) there's no reason for the other side to send this packet (I found this is an already fixed linux bug with forwarding and large receive offload, but no way to contact the administrator of that router), b) our side shouldn't resend the whole sendqueue when the mtu changes, c) if the mtu didn't change, then there's no need to take any action. We only fixed c, but that solved the major problem: these resends would trigger more resends and we'd have periods of unavailability as the network was really busy. This is pretty common when looking at wireshark; unless you work somewhere with full control of all clients and servers and a very network aware developer team, you're going to find lots of non-optimal or semi-broken stuff, and you've got to ignore it and focus on the majorly broken bit. [1] https://bugzilla.mozilla.org/show_bug.cgi?id=1740856 https://bugzilla.mozilla.org/show_bug.cgi?id=1740856 [2] https://reviews.freebsd.org/rS288412 https://reviews.freebsd.org/rS288412
3y ago