4 ms·

Visualizing Binary Files

135 points by zdw 2mo ago

lbhdc 2mo ago
https://github.com/wader/fq https://github.com/wader/fq I have really liked fq for binary visualization.
nvalis 2mo ago
I always liked the 3D visualization in Veles[0]. It's very intuitive to make out structured ASCII or other non-compressed sections in binary data. [0] https://github.com/codilime/veles https://github.com/codilime/veles
st_goliath 2mo ago
You might be interested in GNU poke[1][2], a hex-editor type program that can actually parse and visualize the structure of binary data (using a DSL for describing formats). [1] https://jemarch.net/poke https://jemarch.net/poke [2] https://news.ycombinator.com/item?id=34986042 https://news.ycombinator.com/item?id=34986042
- pdpi 2mo ago
  I've been using ImHex[0] for this purpose, to pretty good effect. [0]: https://github.com/werwolv/imhex https://github.com/werwolv/imhex
1970-01-01 2mo ago
And binwalk if you want to find the juice within the file. https://github.com/ReFirmLabs/binwalk/wiki/Supported-Signatures https://github.com/ReFirmLabs/binwalk/wiki/Supported-Signatu...
deleted 2mo ago
[deleted]
EvanAnderson 2mo ago
Christopher Domas gave a nice presentation[0] at DerbyCon 2012 about software that ended up becoming CantorDust[1]. I was pretty jazzed about after the presentation but didn't keep up with where it went. [0] https://www.irongeek.com/i.php?page=videos/derbycon2/4-2-1-christopher-domas-the-future-of-re-dynamic-binary-visulization https://www.irongeek.com/i.php?page=videos/derbycon2/4-2-1-c... [1] https://github.com/Battelle/cantordust https://github.com/Battelle/cantordust
xvilka 2mo ago
Rizin[1][2] has the byte-level highlight (enter visual mode with `V` key then rotate through modes using `p` or `P`)[3], and there is an issue[4] for Cutter to add binary visualization as a plugin, if anyone is up to the task. [1] https://rizin.re https://rizin.re [2] https://github.com/rizinorg/rizin https://github.com/rizinorg/rizin [3] https://book.rizin.re/src/visual_mode/intro.html https://book.rizin.re/src/visual_mode/intro.html [4] https://github.com/rizinorg/cutter-plugins/issues/3 https://github.com/rizinorg/cutter-plugins/issues/3
Piraty 2mo ago
arcan [1] has senseye [2][3]. [1] https://arcan-fe.com https://arcan-fe.com [2] youtu.be/WBsv9IJpkDw [3] youtu.be/ALA1W-BL8fA
zX41ZdbW 2mo ago
ClickHouse can read and visualize its own binary: https://play.clickhouse.com/binary?user=binary https://play.clickhouse.com/binary?user=binary It is useful to understand which dependencies, libraries, and template instantiations contribute to the size.
WalterGR 2mo ago
Related: "Your hex editor should color-code bytes" (simonomi.dev) 614 points | 4 months ago | 159 comments https://news.ycombinator.com/item?id=47846688 https://news.ycombinator.com/item?id=47846688
trevor-e 2mo ago
Very nice, I did something similar a while back and was going to make a poster of interesting ones to hang in my office: https://www.telkins.com/blog/visualizing-file-formats https://www.telkins.com/blog/visualizing-file-formats Never got around to that poster though. :( If anyone is interested we open-sourced https://github.com/getsentry/launchpad https://github.com/getsentry/launchpad which can produce a treemap visualization for Mach-O. I've been working on ELF support in my free time too but it's not finished.
yeehawjared 2mo ago
surprised to not see hilbert curve mentioned here https://corte.si/posts/visualisation/binvis/ https://corte.si/posts/visualisation/binvis/
- fussylogic 2mo ago
  Cool tool from the above blog post. Worth a look. https://binvis.io/ https://binvis.io/ Ps: I had the exact same thought! Also: can programs be encoded into lossy image formats? Would be cool to see what image compression does to program functionality
  - Retr0id 2mo ago
    I use binvis quite a lot for getting a quick overview of an unknown file format, using the entropy mode in particular. However, I almost always switch it into the linear view. Hilbert curves are cool but I've never really found any benefit, and it makes it harder to intuit things like how far a particular feature is into the file, or whether two features are actually nearby (while near things in linear-space are typically near in hilbert-space, there are discontinuities).
nyx 2mo ago
I've been using https://binvis.io https://binvis.io for this when I just want quick-and-dirty color coding to see what the heck I'm looking at.
deleted 2mo ago
[deleted]
wseqyrku 2mo ago
I would like to see the CFG visualized for executable files. I imagine a recursive descent parser would resemble a spiral?
edpot1 2mo ago
A different approach I’ve been working on: https://github.com/ufex/Ufex https://github.com/ufex/Ufex Detect the file format and display a detailed view of the structures within. Also has the hex editor with coloring based on bytes and sequences.
0xad 2mo ago
Binvisio from cortesi and cantordust from domas being already mentioned, I’m surprised I haven’t seen this yet: https://lcamtuf.coredump.cx/newtcp/ https://lcamtuf.coredump.cx/newtcp/

135 points by zdw 2mo ago

lbhdc 2mo ago

https://github.com/wader/fq https://github.com/wader/fq I have really liked fq for binary visualization.

I always liked the 3D visualization in Veles[0]. It's very intuitive to make out structured ASCII or other non-compressed sections in binary data. [0] https://github.com/codilime/veles https://github.com/codilime/veles

st_goliath 2mo ago

You might be interested in GNU poke[1][2], a hex-editor type program that can actually parse and visualize the structure of binary data (using a DSL for describing formats). [1] https://jemarch.net/poke https://jemarch.net/poke [2] https://news.ycombinator.com/item?id=34986042 https://news.ycombinator.com/item?id=34986042

pdpi 2mo ago
I've been using ImHex[0] for this purpose, to pretty good effect. [0]: https://github.com/werwolv/imhex https://github.com/werwolv/imhex

1970-01-01 2mo ago

And binwalk if you want to find the juice within the file. https://github.com/ReFirmLabs/binwalk/wiki/Supported-Signatures https://github.com/ReFirmLabs/binwalk/wiki/Supported-Signatu...

deleted 2mo ago

[deleted]

EvanAnderson 2mo ago

Christopher Domas gave a nice presentation[0] at DerbyCon 2012 about software that ended up becoming CantorDust[1]. I was pretty jazzed about after the presentation but didn't keep up with where it went. [0] https://www.irongeek.com/i.php?page=videos/derbycon2/4-2-1-christopher-domas-the-future-of-re-dynamic-binary-visulization https://www.irongeek.com/i.php?page=videos/derbycon2/4-2-1-c... [1] https://github.com/Battelle/cantordust https://github.com/Battelle/cantordust

xvilka 2mo ago

Rizin[1][2] has the byte-level highlight (enter visual mode with `V` key then rotate through modes using `p` or `P`)[3], and there is an issue[4] for Cutter to add binary visualization as a plugin, if anyone is up to the task. [1] https://rizin.re https://rizin.re [2] https://github.com/rizinorg/rizin https://github.com/rizinorg/rizin [3] https://book.rizin.re/src/visual_mode/intro.html https://book.rizin.re/src/visual_mode/intro.html [4] https://github.com/rizinorg/cutter-plugins/issues/3 https://github.com/rizinorg/cutter-plugins/issues/3

Piraty 2mo ago

arcan [1] has senseye [2][3]. [1] https://arcan-fe.com https://arcan-fe.com [2] youtu.be/WBsv9IJpkDw [3] youtu.be/ALA1W-BL8fA

zX41ZdbW 2mo ago

ClickHouse can read and visualize its own binary: https://play.clickhouse.com/binary?user=binary https://play.clickhouse.com/binary?user=binary It is useful to understand which dependencies, libraries, and template instantiations contribute to the size.

WalterGR 2mo ago

Related: "Your hex editor should color-code bytes" (simonomi.dev) 614 points | 4 months ago | 159 comments https://news.ycombinator.com/item?id=47846688 https://news.ycombinator.com/item?id=47846688

trevor-e 2mo ago

Very nice, I did something similar a while back and was going to make a poster of interesting ones to hang in my office: https://www.telkins.com/blog/visualizing-file-formats https://www.telkins.com/blog/visualizing-file-formats Never got around to that poster though. :( If anyone is interested we open-sourced https://github.com/getsentry/launchpad https://github.com/getsentry/launchpad which can produce a treemap visualization for Mach-O. I've been working on ELF support in my free time too but it's not finished.

yeehawjared 2mo ago

surprised to not see hilbert curve mentioned here https://corte.si/posts/visualisation/binvis/ https://corte.si/posts/visualisation/binvis/

fussylogic 2mo ago
Cool tool from the above blog post. Worth a look. https://binvis.io/ https://binvis.io/ Ps: I had the exact same thought! Also: can programs be encoded into lossy image formats? Would be cool to see what image compression does to program functionality
- Retr0id 2mo ago
  I use binvis quite a lot for getting a quick overview of an unknown file format, using the entropy mode in particular. However, I almost always switch it into the linear view. Hilbert curves are cool but I've never really found any benefit, and it makes it harder to intuit things like how far a particular feature is into the file, or whether two features are actually nearby (while near things in linear-space are typically near in hilbert-space, there are discontinuities).

nyx 2mo ago

I've been using https://binvis.io https://binvis.io for this when I just want quick-and-dirty color coding to see what the heck I'm looking at.

deleted 2mo ago

[deleted]

wseqyrku 2mo ago

I would like to see the CFG visualized for executable files. I imagine a recursive descent parser would resemble a spiral?

edpot1 2mo ago

A different approach I’ve been working on: https://github.com/ufex/Ufex https://github.com/ufex/Ufex Detect the file format and display a detailed view of the structures within. Also has the hex editor with coloring based on bytes and sequences.

0xad 2mo ago

Binvisio from cortesi and cantordust from domas being already mentioned, I’m surprised I haven’t seen this yet: https://lcamtuf.coredump.cx/newtcp/ https://lcamtuf.coredump.cx/newtcp/