6 ms·

Binary data visualization

196 points by dmit 9y ago

miduil 9y ago
Really cool, I really like that binary data, with all it's abstractions, still creates visual meaning. Also reminds me of pixid [1], which sadly never got any traction on HN. Plus the common example, in which CBC encryption fails [2], is definitively something to throw in. [1]: https://github.com/FireyFly/pixd https://github.com/FireyFly/pixd [2]: https://en.wikipedia.org/wiki/Block_cipher_mode_of_operation#/media/File:Tux_ecb.jpg https://en.wikipedia.org/wiki/Block_cipher_mode_of_operation...
- armitron 9y ago
  It's not CBC but ECB.
  - miduil 9y ago
    Uh, missed that - Thank's for pointing out.
- deleted 9y ago
  [deleted]
gue5t 9y ago
Is this using the same techniques as the old mostly-vaporware "cantor dust" project? It looks very similar. https://www.reddit.com/r/ReverseEngineering/comments/2apw6l/what_happened_to_cantor_dust/ https://www.reddit.com/r/ReverseEngineering/comments/2apw6l/...
- tekacs 9y ago
  Yes. If you follow the link at the top of the page to the explanation post[1], it says at the bottom: > Veles visualizations were inspired by Christopher Domas’ talk from Derbycon 2012. > Check out the video of his presentation https://www.youtube.com/watch?v=4bM3Gut1hIk https://www.youtube.com/watch?v=4bM3Gut1hIk. [1]: https://codisec.com/binary-visualization-explained/ https://codisec.com/binary-visualization-explained/
PartyDonkey 9y ago
Reminds me of a talk I saw at a conference a while back about doing it with fractals. http://ieeexplore.ieee.org/document/6694490/ http://ieeexplore.ieee.org/document/6694490/
olejorgenb 9y ago
Seems to be open source: https://github.com/codilime/veles https://github.com/codilime/veles A nice summary of other simialr tools: https://reverseengineering.stackexchange.com/questions/6003/visualizing-elf-binaries https://reverseengineering.stackexchange.com/questions/6003/...
kuwze 9y ago
Reminds me of svforth[0]. [0] https://github.com/ephsec/svforth/blob/master/README.md https://github.com/ephsec/svforth/blob/master/README.md
userbinator 9y ago
Relatedly, a similar technique is used to assess the quality of random number generators: https://en.wikipedia.org/wiki/Spectral_test https://en.wikipedia.org/wiki/Spectral_test
uoaei 9y ago
That's not binary data at all! That is data stored in a binary, but the data itself is hexadecimal.
- peteretep 9y ago
  Binary like "this file may be binary doyou still wish to view it" I guess
- jacobolus 9y ago
  Actually they explicitly pull out bytes (each byte is 8 bits, or 2 hexadecimal digits). So really it’s ducentahexaquinquagesimal (if Wikipedia can be trusted about the word for base 256). https://codisec.com/binary-visualization-explained/ https://codisec.com/binary-visualization-explained/ But the files themselves mostly consist of data elements (like numbers) which are larger than that, e.g. 8-byte double-precision floating point numbers. When people say “binary data” what they mean is that it’s a machine-readable data format, not a human-readable text-based file.
  - uoaei 9y ago
    "Binary data" means, in actuality, data that are only expressed using two bits. In common parlance, this means 1 and 0, or presence and absence. Using anything larger requires a different label to remain accurate.
    - jacobolus 9y ago
      https://en.wikipedia.org/wiki/Binary_data#In_computer_science https://en.wikipedia.org/wiki/Binary_data#In_computer_scienc...
    - marvin 9y ago
      Isn't this just arguing semantics? A byte written in hexadecimal notation is just a convenient human-readable(ish) way to represent a sequence of 8 bits of binary data.
      - 9y ago
hyperpallium 9y ago
\tangent (properly) Compressed data is indistinguishable from noise. This is why the universe's background radiation is actually alien communication from all over the place.
- mkow 9y ago
  It's not (at least in most cases): https://codisec.com/compression-vs-encryption-visual-recognition/ https://codisec.com/compression-vs-encryption-visual-recogni...
- jchw 9y ago
  LZ-based compression won't look like noise.
thanatropism 9y ago
They keep spelling "Cartesian" as "Carthesian".
- bahjoite 9y ago
  Perhaps they're getting it mixed-up with Carthusian.
dahoramanodoceu 9y ago
There was no way to argue with Gentry; the juice was his, because he was the one who fiddled it out of the Fission Authority; without Gentry's monthly passes on the console, the ritual moves that kept the Authority convinced Factory was somewhere else, some place that paid its bill, there wouldn't be any electricity. And Gentry was so strange anyway, he thought, feeling his knees creak as he stood up and took the Judge's control unit from his jacket pocket. Gentry was convinced that cyberspace had a Shape, an overall total form. Not that that was the weirdest idea Slick had ever run across, but Gentry had this obsessive conviction that the Shape mattered totally . The apprehension of the Shape was Gentry's grail. Slick had once stimmed a Net/Knowledge sequence about what shape the universe was; Slick figured the universe was everything there was, so how could it have a shape? If it had a shape, then there was something around it for it to have a shape in , wasn't there? And if that something was something, then wasn't that part of the universe too? This was exactly the kind of thing you didn't want to get into with Gentry, because Gentry could tie your head in knots. But Slick didn't think cyberspace was anything like the universe anyway; it was just a way of representing data. The Fission Authority had always looked like a big red Aztec pyramid, but it didn't have to; if the FA wanted it to, they could have it look like anything. Big companies had copyrights on how their stuff looked. So how could you figure the whole matrix had a particular shape? And why should it mean anything if it did? He touched the unit's power stud; the Judge, ten meters away, hummed and trembled.
- tda- 9y ago
  I saw this and instantly thought of Gibson.. glad I wasn't the only one ;) We need more of this kinda thing..
  - dahoramanodoceu 9y ago
    Totally agree
dahoramanodoceu 9y ago
[Tim Borgmann - Mind Over Eye][https://vimeo.com/145264053 https://vimeo.com/145264053] This is beautiful and fascinating! It Reminds me a lot of this. Also, can this be used in any functionally meaningful way for code-analysis and/or conceptualizing? Animating a sequence of code execution?
- mkow 9y ago
  There's probably a potential in visualizing code execution, see e.g. this: https://twitter.com/doegox/status/811934079620878336 https://twitter.com/doegox/status/811934079620878336

196 points by dmit 9y ago

Really cool, I really like that binary data, with all it's abstractions, still creates visual meaning. Also reminds me of pixid [1], which sadly never got any traction on HN. Plus the common example, in which CBC encryption fails [2], is definitively something to throw in. [1]: https://github.com/FireyFly/pixd https://github.com/FireyFly/pixd [2]: https://en.wikipedia.org/wiki/Block_cipher_mode_of_operation#/media/File:Tux_ecb.jpg https://en.wikipedia.org/wiki/Block_cipher_mode_of_operation...

armitron 9y ago
It's not CBC but ECB.
- miduil 9y ago
  Uh, missed that - Thank's for pointing out.
deleted 9y ago
[deleted]

gue5t 9y ago

Is this using the same techniques as the old mostly-vaporware "cantor dust" project? It looks very similar. https://www.reddit.com/r/ReverseEngineering/comments/2apw6l/what_happened_to_cantor_dust/ https://www.reddit.com/r/ReverseEngineering/comments/2apw6l/...

tekacs 9y ago
Yes. If you follow the link at the top of the page to the explanation post[1], it says at the bottom: > Veles visualizations were inspired by Christopher Domas’ talk from Derbycon 2012. > Check out the video of his presentation https://www.youtube.com/watch?v=4bM3Gut1hIk https://www.youtube.com/watch?v=4bM3Gut1hIk. [1]: https://codisec.com/binary-visualization-explained/ https://codisec.com/binary-visualization-explained/

PartyDonkey 9y ago

Reminds me of a talk I saw at a conference a while back about doing it with fractals. http://ieeexplore.ieee.org/document/6694490/ http://ieeexplore.ieee.org/document/6694490/

olejorgenb 9y ago

Seems to be open source: https://github.com/codilime/veles https://github.com/codilime/veles A nice summary of other simialr tools: https://reverseengineering.stackexchange.com/questions/6003/visualizing-elf-binaries https://reverseengineering.stackexchange.com/questions/6003/...

kuwze 9y ago

Reminds me of svforth[0]. [0] https://github.com/ephsec/svforth/blob/master/README.md https://github.com/ephsec/svforth/blob/master/README.md

userbinator 9y ago

Relatedly, a similar technique is used to assess the quality of random number generators: https://en.wikipedia.org/wiki/Spectral_test https://en.wikipedia.org/wiki/Spectral_test

uoaei 9y ago

That's not binary data at all! That is data stored in a binary, but the data itself is hexadecimal.

peteretep 9y ago
Binary like "this file may be binary doyou still wish to view it" I guess
jacobolus 9y ago
Actually they explicitly pull out bytes (each byte is 8 bits, or 2 hexadecimal digits). So really it’s ducentahexaquinquagesimal (if Wikipedia can be trusted about the word for base 256). https://codisec.com/binary-visualization-explained/ https://codisec.com/binary-visualization-explained/ But the files themselves mostly consist of data elements (like numbers) which are larger than that, e.g. 8-byte double-precision floating point numbers. When people say “binary data” what they mean is that it’s a machine-readable data format, not a human-readable text-based file.
- uoaei 9y ago
  "Binary data" means, in actuality, data that are only expressed using two bits. In common parlance, this means 1 and 0, or presence and absence. Using anything larger requires a different label to remain accurate.
  - jacobolus 9y ago
    https://en.wikipedia.org/wiki/Binary_data#In_computer_science https://en.wikipedia.org/wiki/Binary_data#In_computer_scienc...
  - marvin 9y ago
    Isn't this just arguing semantics? A byte written in hexadecimal notation is just a convenient human-readable(ish) way to represent a sequence of 8 bits of binary data.
    - 9y ago

hyperpallium 9y ago

\tangent (properly) Compressed data is indistinguishable from noise. This is why the universe's background radiation is actually alien communication from all over the place.

mkow 9y ago
It's not (at least in most cases): https://codisec.com/compression-vs-encryption-visual-recognition/ https://codisec.com/compression-vs-encryption-visual-recogni...
jchw 9y ago
LZ-based compression won't look like noise.

thanatropism 9y ago

They keep spelling "Cartesian" as "Carthesian".

bahjoite 9y ago
Perhaps they're getting it mixed-up with Carthusian.

dahoramanodoceu 9y ago

There was no way to argue with Gentry; the juice was his, because he was the one who fiddled it out of the Fission Authority; without Gentry's monthly passes on the console, the ritual moves that kept the Authority convinced Factory was somewhere else, some place that paid its bill, there wouldn't be any electricity. And Gentry was so strange anyway, he thought, feeling his knees creak as he stood up and took the Judge's control unit from his jacket pocket. Gentry was convinced that cyberspace had a Shape, an overall total form. Not that that was the weirdest idea Slick had ever run across, but Gentry had this obsessive conviction that the Shape mattered totally . The apprehension of the Shape was Gentry's grail. Slick had once stimmed a Net/Knowledge sequence about what shape the universe was; Slick figured the universe was everything there was, so how could it have a shape? If it had a shape, then there was something around it for it to have a shape in , wasn't there? And if that something was something, then wasn't that part of the universe too? This was exactly the kind of thing you didn't want to get into with Gentry, because Gentry could tie your head in knots. But Slick didn't think cyberspace was anything like the universe anyway; it was just a way of representing data. The Fission Authority had always looked like a big red Aztec pyramid, but it didn't have to; if the FA wanted it to, they could have it look like anything. Big companies had copyrights on how their stuff looked. So how could you figure the whole matrix had a particular shape? And why should it mean anything if it did? He touched the unit's power stud; the Judge, ten meters away, hummed and trembled.

tda- 9y ago
I saw this and instantly thought of Gibson.. glad I wasn't the only one ;) We need more of this kinda thing..
- dahoramanodoceu 9y ago
  Totally agree

dahoramanodoceu 9y ago

[Tim Borgmann - Mind Over Eye][https://vimeo.com/145264053 https://vimeo.com/145264053] This is beautiful and fascinating! It Reminds me a lot of this. Also, can this be used in any functionally meaningful way for code-analysis and/or conceptualizing? Animating a sequence of code execution?

mkow 9y ago
There's probably a potential in visualizing code execution, see e.g. this: https://twitter.com/doegox/status/811934079620878336 https://twitter.com/doegox/status/811934079620878336