4 ms·

DEFCON30, Mayhem CTF. We were given a file full of random bytes. The flag was in there somewhere. It was too random to be encrypted, there wasn't any structure

by bwiggs 6mo ago

DEFCON30, Mayhem CTF. We were given a file full of random bytes. The flag was in there somewhere. It was too random to be encrypted, there wasn't any structure. `file` didn't return anything, truly just a bag of bytes. I had decided to install `hexyl` as an alternative option to some of the other hex editors installed o my linux machine. All the bytes were colored grey. I scrolled the file and noticed a blip of yellow. A random golden `{` amongst all the noise. Weird. The next colored byte was a `C`, then `T`, `F`. --- At that time, I was mostly using HexFiend to look at raw files, which didn't have byte coloring. For DEFCON I had decided to drive my linux machine. I had ghex installed, but i had also decided to install and try `hexyl` via cli. So seeing bytes in color was purely by chance that I had installed it. I eventually posted an issue to ghex to add color support. https://gitlab.gnome.org/GNOME/ghex/-/issues/60 https://gitlab.gnome.org/GNOME/ghex/-/issues/60 I need to see if I can find the file and post it on that blog post. https://bwiggs.com/posts/2023-08-31-hacking-in-color/ https://bwiggs.com/posts/2023-08-31-hacking-in-color/

abcd_f 6mo ago
> It was too random to be encrypted That's a rather odd remark.
- bwiggs 6mo ago
  You would still expect some amount of file structure, ex byte headers or something at the beginning/end of file. no?
  - Crestwave 6mo ago
    That would be expected for encodings or container file formats. Straight-up encryption like AES produces results that are visually indistinguishable from random data.
  - throwawaysoxjje 6mo ago
    I’d expect the greater length of the encrypted data (which should look random) vs the structured header/footer to rapidly push the Shannon entropy to the maximum
  - wang_li 6mo ago
    No. Why would you? Encrypted data should look no different than random. The app figures out if it is the app's data after it attempts to decode it.
- justsomehnguy 6mo ago
  It's not? Compare a random data from a pseudo gen, a really random data and some encrypted data. They are all different.
  - BenjiWiebe 6mo ago
    If it's a CSPRNG, and decent encryption, they'll be indistinguishable from random noise.
  - fc417fc802 6mo ago
    Either you messed up somehow or you've made a massive discovery. I assume it's the former.
Crestwave 6mo ago
Wouldn't strings(1) have worked for this?
- ksherlock 6mo ago
  By default, strings needs a run of 4+ (printable|ascii) characters. This sounds like it was 1 ascii character at a time in a sea of random data (with other alpha chars removed).
  - Crestwave 6mo ago
    Ah, right. It would have to be `strings -n 1`, then.
masfuerte 6mo ago
I don't get it. If you were looking at random data, why did hexyl apply colour to only the brace, C, T and F?
- Crestwave 6mo ago
  The implication is that the random-looking data did not contain ASCII characters other than the parts of the flag

by bwiggs 6mo ago

abcd_f 6mo ago

> It was too random to be encrypted That's a rather odd remark.

bwiggs 6mo ago
You would still expect some amount of file structure, ex byte headers or something at the beginning/end of file. no?
- Crestwave 6mo ago
  That would be expected for encodings or container file formats. Straight-up encryption like AES produces results that are visually indistinguishable from random data.
- throwawaysoxjje 6mo ago
  I’d expect the greater length of the encrypted data (which should look random) vs the structured header/footer to rapidly push the Shannon entropy to the maximum
- wang_li 6mo ago
  No. Why would you? Encrypted data should look no different than random. The app figures out if it is the app's data after it attempts to decode it.
justsomehnguy 6mo ago
It's not? Compare a random data from a pseudo gen, a really random data and some encrypted data. They are all different.
- BenjiWiebe 6mo ago
  If it's a CSPRNG, and decent encryption, they'll be indistinguishable from random noise.
- fc417fc802 6mo ago
  Either you messed up somehow or you've made a massive discovery. I assume it's the former.

Crestwave 6mo ago

Wouldn't strings(1) have worked for this?

ksherlock 6mo ago
By default, strings needs a run of 4+ (printable|ascii) characters. This sounds like it was 1 ascii character at a time in a sea of random data (with other alpha chars removed).
- Crestwave 6mo ago
  Ah, right. It would have to be `strings -n 1`, then.

masfuerte 6mo ago

I don't get it. If you were looking at random data, why did hexyl apply colour to only the brace, C, T and F?

Crestwave 6mo ago
The implication is that the random-looking data did not contain ASCII characters other than the parts of the flag