9 ms·

The part that resonated most with me is "Show things that are normally hidden". Tools that do this make things clearer almost immediately. Consider the develop

The part that resonated most with me is "Show things that are normally hidden". Tools that do this make things clearer almost immediately. Consider the developer tools in a web browser. Do you remember the "dark ages" before such things existed? It was awful because you had to guess instead of seeing what was going on. Tools like Wireshark that show you every last byte of network packets that it has access to AND parses it to help you see the structure. This isn't just for debugging networking data; it's hugely beneficial in teaching networking concepts because nothing is hidden. This is also one of my favorite things about open source software. I can view the source to understand what's causing a bug, to fill in knowledge gaps left by the documentation, or just learn more about programming concepts. Nothing is hidden.

jzwinck 3y ago
Wireshark is great but it does not show you every byte the network carried. For example it never shows Ethernet preambles, only sometimes shows Ethernet frame checksums, and never shows interpacket gaps (which are a required part of the Ethernet protocol). So yes it comes close but it just goes to show you, there is always more detail hiding somewhere!
- 5pales 3y ago
  To be fair, the preamble can be easily considered just an electrical signal due to its objective (sync) which doesn't affect how the network works.
- justin_oaks 3y ago
  Yes, the toughest "hidden things" problems are pulling together data that is related, but not part of the same system. In this case, Wireshark can only show you what the OS gives to it. In the article, it was pointed out that DNS caches can be hidden. They're especially hidden when they're upstream and in another computer!
- AceJohnny2 3y ago
  Are the Ethernet frame checksums even visible to Wireshark, which hooks into the IP layer? would some of the ethernet stuff be only visible within the ethernet card itself, not to the software stack?
  - jzwinck 3y ago
    Sometimes they are. It depends on how the capture was generated. If you look in the options of Wireshark there is one to detect bad checksums, so clearly there is a way to capture them. Here is one such way: https://stackoverflow.com/questions/22101650/how-can-i-receive-the-wrong-ethernet-frames-and-disable-the-crc-fcs-calcul https://stackoverflow.com/questions/22101650/how-can-i-recei... This can be used to detect partially-bad network cables, because there is no reason you should ever receive a bad FCS.
  - thelittleone 3y ago
    Wireshark works at the data link layer (L2).
    - vitus 3y ago
      Well... ethhdr only has 14 bytes: 6 for dest mac address, 6 for source mac address, 2 for ethertype (e.g. IPv4 vs IPv6). Any bad checksums that Wireshark can detect are predominantly at the transport layer (L4; TCP / UDP). You'd have to explicitly turn on FCS, which I don't know if you can even do on, say, Windows.
  - fulafel 3y ago
    Wireshark has support for lots of interfaces, it's just a consumer for OS/driver supplied data in this aspect. See eg this bit about wlan for some of the complexity: https://wiki.wireshark.org/CaptureSetup/WLAN#link-layer-radio-packet-headers https://wiki.wireshark.org/CaptureSetup/WLAN#link-layer-radi...
- kortilla 3y ago
  It doesn’t show electrical signals either. That’s a pointless nitpick if you’re developing anything beyond an Ethernet implementation.
- torginus 3y ago
  My biggest problem with Wireshark is that it can't do anything with HTTPS traffic - which is most of the traffic I'd be interested in. I understand that's kind of the point of HTTPS, and a MITM proxy with cert replacement is somewhat out of the scope of Wireshark, it still limits the usefulness of the program.
  - dv35z 3y ago
    Check out Charles Proxy for mac. It’s great.
  - pl4nty 3y ago
    it can, just not as easily as MITM proxies: https://wiki.wireshark.org/TLS https://wiki.wireshark.org/TLS
  - kalmi10 3y ago
    This method is really simple for Chrome, Firefox and other OpenSSL apps: https://wiki.wireshark.org/TLS#using-the-pre-master-secret https://wiki.wireshark.org/TLS#using-the-pre-master-secret You set an environment variable to instruct the app the write a file that Wireshark can use to decrypt its traffic, and change a setting in Wireshark to use that file, and that's it. You will even be able to see decrypted WebRTC traffic.
- gizmo686 3y ago
  Wireshark will also frequently show the wrong checksums for IP, UDP, and TCP. Specifically when computing them is ofloaded to the network card.
swayvil 3y ago
Sector editor! (Or back in the day, looking at the source code because we ran uncompiled stuff in Basic and whatever and that was pretty cool)
- justin_oaks 3y ago
  That's one of the advantages of using programming languages where source code is distributed (e.g. Python, JavaScript, PHP), not compiled binary artifacts (C/C++, Java). You can see the source. You can even modify it and run the modified version without compilation. It's also awesome to use Java IDEs that can show both the bytecode of .class files and also perform decompilation.
vaughan 3y ago
My dream is to make everything visualizable at runtime. I think all of computing becomes very simple and much less complex if we can do this. We are visualizing things in our head already. And any explanation of anything in computing is a diagram. But we have zero diagrams when coding. Just dynamically instrument all code to send messages to a GUI.
- the-alchemist 3y ago
  Clojure does pretty well. See https://github.com/nubank/morse https://github.com/nubank/morse, https://docs.datomic.com/cloud/other-tools/REBL.html https://docs.datomic.com/cloud/other-tools/REBL.html, and https://vlaaad.github.io/reveal/ https://vlaaad.github.io/reveal/. It's one of the areas that homoiconicity helps: code is data, data is code, so visualization tools can work on both sides.
- yourapostasy 3y ago
  >My dream is to make everything visualizable at runtime. Check out demos of the old Lisp Machines, [1] is a brief overview demo, [2] links to a timestamp with a view of some simple diagramming, but I’ve seen TI-Symbolics beasts routinely display complex relationships in Lisp code on their massive (for the time) bitmapped screens. The limitation was the end user managing the visualization complexity. With open source llvm, clang and similar making available abstract syntax trees and even semantic analysis and type checking results, LLM’s assisting with decompiling binary blobs, and modern hardware (goggles, graphics cards, and so on), I sometimes wonder how close we can come to reproducing that aspect of the Lisp Machine experience on open source operating systems. [1] https://youtu.be/o4-YnLpLgtk https://youtu.be/o4-YnLpLgtk [2] https://youtu.be/jACcgLfyiyM?t=43m52s https://youtu.be/jACcgLfyiyM?t=43m52s
Natsu 3y ago
Man, having spent waaaay too long troubleshooting errors where it's not remotely clear what part of the config the program is even consulting makes this hit home.
raincole 3y ago
The game development equivalent of these is renderDoc. I was stunned when I learned its existance for the first time.
- carterschonwald 3y ago
  I’m about to embark on my first real and truly massive graphics project. Thx for mentioning such a cool tool.
dclowd9901 3y ago
Yep. Mozilla developer tools basically launched my career.
userbinator 3y ago
This is also one of my favorite things about open source software. I can view the source to understand what's causing a bug, to fill in knowledge gaps left by the documentation, or just learn more about programming concepts. Nothing is hidden. On the contrary, you don't need the source, and in some cases it may even be misleading in many ways, when you can look directly at the instructions the machine executes. Tools like disassemblers and decompilers would be equivalent to what you speak of.
- saghm 3y ago
  Nothing stops you from doing this with open source software either, though; making something open source is strictly an increase in the information available, and at least to me that seems in the spirit of "show me things that are normally hidden"
wilkystyle 3y ago
This is one thing I love about Magit for Emacs. The UI is really clever and slick—maybe the best Git frontend that I've ever used—but the way you interact with UI is by toggling flags and options that actually map to the underlying command line Git arguments. I can seamlessly hop into the command line and feel right at home using it directly.
- TeMPOraL 3y ago
  On top of the UI having very tight and visible relationship to git CLI commands, there's also a log buffer showing you exact commands executed and their outputs, and it's only a single $ press away.
- ParetoOptimal 3y ago
  Yes, text based UI's like magit offer this kind of simplicity without sacrificing cli speed.
giantg2 3y ago
'"Show things that are normally hidden". Tools that do this make things clearer almost immediately.' At least with all the DevOps shit going on now, it seems that many of the tools increasingly hide things. And the gurus who had that knowledge and could teach you are now concentrated in those tool companies instead of in your org.
agent281 3y ago
This is what I like about SQL databases. They frequently have tables that you can query to find system information. This makes it easy to explore the system within the system. The proc filesystem on Linux is philosophically similar. It allows you to understand processes by working with files.
csours 3y ago
https://news.ycombinator.com/item?id=37467078 https://news.ycombinator.com/item?id=37467078 Is there a way to get Awk to emit a non-terse version of the script passed in? ie awk '/test/' -> '{ if($0~/test/){print $0} }'

jzwinck 3y ago

Wireshark is great but it does not show you every byte the network carried. For example it never shows Ethernet preambles, only sometimes shows Ethernet frame checksums, and never shows interpacket gaps (which are a required part of the Ethernet protocol). So yes it comes close but it just goes to show you, there is always more detail hiding somewhere!

5pales 3y ago
To be fair, the preamble can be easily considered just an electrical signal due to its objective (sync) which doesn't affect how the network works.
justin_oaks 3y ago
Yes, the toughest "hidden things" problems are pulling together data that is related, but not part of the same system. In this case, Wireshark can only show you what the OS gives to it. In the article, it was pointed out that DNS caches can be hidden. They're especially hidden when they're upstream and in another computer!
AceJohnny2 3y ago
Are the Ethernet frame checksums even visible to Wireshark, which hooks into the IP layer? would some of the ethernet stuff be only visible within the ethernet card itself, not to the software stack?
- jzwinck 3y ago
  Sometimes they are. It depends on how the capture was generated. If you look in the options of Wireshark there is one to detect bad checksums, so clearly there is a way to capture them. Here is one such way: https://stackoverflow.com/questions/22101650/how-can-i-receive-the-wrong-ethernet-frames-and-disable-the-crc-fcs-calcul https://stackoverflow.com/questions/22101650/how-can-i-recei... This can be used to detect partially-bad network cables, because there is no reason you should ever receive a bad FCS.
- thelittleone 3y ago
  Wireshark works at the data link layer (L2).
  - vitus 3y ago
    Well... ethhdr only has 14 bytes: 6 for dest mac address, 6 for source mac address, 2 for ethertype (e.g. IPv4 vs IPv6). Any bad checksums that Wireshark can detect are predominantly at the transport layer (L4; TCP / UDP). You'd have to explicitly turn on FCS, which I don't know if you can even do on, say, Windows.
- fulafel 3y ago
  Wireshark has support for lots of interfaces, it's just a consumer for OS/driver supplied data in this aspect. See eg this bit about wlan for some of the complexity: https://wiki.wireshark.org/CaptureSetup/WLAN#link-layer-radio-packet-headers https://wiki.wireshark.org/CaptureSetup/WLAN#link-layer-radi...
kortilla 3y ago
It doesn’t show electrical signals either. That’s a pointless nitpick if you’re developing anything beyond an Ethernet implementation.
torginus 3y ago
My biggest problem with Wireshark is that it can't do anything with HTTPS traffic - which is most of the traffic I'd be interested in. I understand that's kind of the point of HTTPS, and a MITM proxy with cert replacement is somewhat out of the scope of Wireshark, it still limits the usefulness of the program.
- dv35z 3y ago
  Check out Charles Proxy for mac. It’s great.
- pl4nty 3y ago
  it can, just not as easily as MITM proxies: https://wiki.wireshark.org/TLS https://wiki.wireshark.org/TLS
- kalmi10 3y ago
  This method is really simple for Chrome, Firefox and other OpenSSL apps: https://wiki.wireshark.org/TLS#using-the-pre-master-secret https://wiki.wireshark.org/TLS#using-the-pre-master-secret You set an environment variable to instruct the app the write a file that Wireshark can use to decrypt its traffic, and change a setting in Wireshark to use that file, and that's it. You will even be able to see decrypted WebRTC traffic.
gizmo686 3y ago
Wireshark will also frequently show the wrong checksums for IP, UDP, and TCP. Specifically when computing them is ofloaded to the network card.

swayvil 3y ago

Sector editor! (Or back in the day, looking at the source code because we ran uncompiled stuff in Basic and whatever and that was pretty cool)

justin_oaks 3y ago
That's one of the advantages of using programming languages where source code is distributed (e.g. Python, JavaScript, PHP), not compiled binary artifacts (C/C++, Java). You can see the source. You can even modify it and run the modified version without compilation. It's also awesome to use Java IDEs that can show both the bytecode of .class files and also perform decompilation.

vaughan 3y ago

My dream is to make everything visualizable at runtime. I think all of computing becomes very simple and much less complex if we can do this. We are visualizing things in our head already. And any explanation of anything in computing is a diagram. But we have zero diagrams when coding. Just dynamically instrument all code to send messages to a GUI.

the-alchemist 3y ago
Clojure does pretty well. See https://github.com/nubank/morse https://github.com/nubank/morse, https://docs.datomic.com/cloud/other-tools/REBL.html https://docs.datomic.com/cloud/other-tools/REBL.html, and https://vlaaad.github.io/reveal/ https://vlaaad.github.io/reveal/. It's one of the areas that homoiconicity helps: code is data, data is code, so visualization tools can work on both sides.
yourapostasy 3y ago
>My dream is to make everything visualizable at runtime. Check out demos of the old Lisp Machines, [1] is a brief overview demo, [2] links to a timestamp with a view of some simple diagramming, but I’ve seen TI-Symbolics beasts routinely display complex relationships in Lisp code on their massive (for the time) bitmapped screens. The limitation was the end user managing the visualization complexity. With open source llvm, clang and similar making available abstract syntax trees and even semantic analysis and type checking results, LLM’s assisting with decompiling binary blobs, and modern hardware (goggles, graphics cards, and so on), I sometimes wonder how close we can come to reproducing that aspect of the Lisp Machine experience on open source operating systems. [1] https://youtu.be/o4-YnLpLgtk https://youtu.be/o4-YnLpLgtk [2] https://youtu.be/jACcgLfyiyM?t=43m52s https://youtu.be/jACcgLfyiyM?t=43m52s

Natsu 3y ago

Man, having spent waaaay too long troubleshooting errors where it's not remotely clear what part of the config the program is even consulting makes this hit home.

raincole 3y ago

The game development equivalent of these is renderDoc. I was stunned when I learned its existance for the first time.

carterschonwald 3y ago
I’m about to embark on my first real and truly massive graphics project. Thx for mentioning such a cool tool.

dclowd9901 3y ago

Yep. Mozilla developer tools basically launched my career.

userbinator 3y ago

This is also one of my favorite things about open source software. I can view the source to understand what's causing a bug, to fill in knowledge gaps left by the documentation, or just learn more about programming concepts. Nothing is hidden. On the contrary, you don't need the source, and in some cases it may even be misleading in many ways, when you can look directly at the instructions the machine executes. Tools like disassemblers and decompilers would be equivalent to what you speak of.

saghm 3y ago
Nothing stops you from doing this with open source software either, though; making something open source is strictly an increase in the information available, and at least to me that seems in the spirit of "show me things that are normally hidden"

wilkystyle 3y ago

This is one thing I love about Magit for Emacs. The UI is really clever and slick—maybe the best Git frontend that I've ever used—but the way you interact with UI is by toggling flags and options that actually map to the underlying command line Git arguments. I can seamlessly hop into the command line and feel right at home using it directly.

TeMPOraL 3y ago
On top of the UI having very tight and visible relationship to git CLI commands, there's also a log buffer showing you exact commands executed and their outputs, and it's only a single $ press away.
ParetoOptimal 3y ago
Yes, text based UI's like magit offer this kind of simplicity without sacrificing cli speed.

giantg2 3y ago

'"Show things that are normally hidden". Tools that do this make things clearer almost immediately.' At least with all the DevOps shit going on now, it seems that many of the tools increasingly hide things. And the gurus who had that knowledge and could teach you are now concentrated in those tool companies instead of in your org.

agent281 3y ago

This is what I like about SQL databases. They frequently have tables that you can query to find system information. This makes it easy to explore the system within the system. The proc filesystem on Linux is philosophically similar. It allows you to understand processes by working with files.

csours 3y ago

https://news.ycombinator.com/item?id=37467078 https://news.ycombinator.com/item?id=37467078 Is there a way to get Awk to emit a non-terse version of the script passed in? ie awk '/test/' -> '{ if($0~/test/){print $0} }'