4 ms·

Does anyone know why the binary blobs cannot be reverse engineered in the age of AI and recompiled to closely match the original source? Is it for legal reasons

Does anyone know why the binary blobs cannot be reverse engineered in the age of AI and recompiled to closely match the original source? Is it for legal reasons? Is it firmware signatures?

x-complexity 5mo ago
The capability isn't there yet. Some of it is there, but not to the level of reliable reverse engineering. https://programbench.com/ https://programbench.com/
- himata4113 5mo ago
  I beg to differ I've done this already. This is a harness issue not a model issue. It won't be identical, but as long as the A->B test loop can be closed I've had 100% success rate.
sschueller 5mo ago
They probably can many things but I think things like memory timing is something you can't just easily reverse engineer from a blob. You need to test every state that the device can be in and see how the blob responds which is quite difficult.
- himata4113 5mo ago
  Why not? Those timings and general initialization are inside the blobs?
bradfa 5mo ago
Many silicon vendors, when providing said binary blobs to a device OEM or even just documentation or source code for the binary blobs, will make companies agree to a license or other legal terms which prohibits reverse engineering. Often the direct recipient of the binary blobs (the OEM of the device) cannot legally let their employees nor contractors perform the reverse engineering. Generally, unless a similar license or legal terms are required to be agreed to by the end user, nothing stops the end user from reversing said binary blobs. But before you attempt this, be sure you fully understand every legal document which was presented to you by the device vendor. Click-through EULAs included.
- hoherd 5mo ago
  What would Jon Lech Johansen do?
mschuster91 5mo ago
Some of this stuff you need to probe with very, very expensive equipment and fine-tune parameters, and to make it worse, RF performance also depends on the composition of the PCB layers, the amount of vias, hell even just rotating an unrelated component on a PCB can affect the RF performance of nearby components or traces. If you're into that, Hans Rosenberg has some darn good yt videos [3]. RF is a world of black magic, especially at the frequencies, symbol rates and encodings used for stuff like RAM. And the higher in frequency you go... the less "conventional wisdoms" apply. There's in fact a very old article from 2007 [1] describing the issue from the other end. Some researcher tried to have a primitive form of what we'd call "machine learning" a few years later write FPGA bitstream to get a tone discriminator. Turns out the algorithm and test harness got a working bitstream... but it made no sense at all, it was very finely tuned to individual physical characteristics of that chip. Link training blobs on modern chips do something very, very similar, at each link initialization they evaluate a lot of different parameters per pin to account for the current state the device is in to get the best (i.e. highest stable bit rate) possible link. And the parameters, value ranges and timings all vary between different chips, so if you write a blob for one combination of SoC and memory, it might very well be possible that you need an entirely different blob for another combination. And that is what the BSP vendor does and what is inside the blob... a tuned version of parameters that this specific board's firmware can use to achieve a working good link in the shortest possible time. It's one hell of a ride that Flipper is on, and I seriously wish them all the best. There's a darn good reason this stuff has been proprietary, at best you'll get a high-level summary like [2]. [1] https://www.damninteresting.com/on-the-origin-of-circuits/ https://www.damninteresting.com/on-the-origin-of-circuits/ [2] https://www.ti.com/lit/an/snla415/snla415.pdf?ts=1779305695379 https://www.ti.com/lit/an/snla415/snla415.pdf?ts=17793056953... [3] https://www.youtube.com/watch?v=xhuHAhIKWoM https://www.youtube.com/watch?v=xhuHAhIKWoM
- himata4113 5mo ago
  I understand that much, but we have the firmware - we don't have to extract it or rebuild it in a white-room? Or is some of that information baked into the firmware that we cannot flash? I've reversed some camera sensor binary blobs recently and I didn't have too many issues, only took a really long time (5 days) of running in a loop until source matched the original (sematically). I parsed the blobs into an intermediate language, exported to llvm and optimized it to recover the functional structure I also used this sematics structure to rebuild the source in rust(kinda, it's a variant of rust with snowflake features cause I hate the language). I think I got lucky because the sensors used the same mips opcodes (sv class) rather than some proprietary state machine.

Does anyone know why the binary blobs cannot be reverse engineered in the age of AI and recompiled to closely match the original source? Is it for legal reasons? Is it firmware signatures?

x-complexity 5mo ago

The capability isn't there yet. Some of it is there, but not to the level of reliable reverse engineering. https://programbench.com/ https://programbench.com/

himata4113 5mo ago
I beg to differ I've done this already. This is a harness issue not a model issue. It won't be identical, but as long as the A->B test loop can be closed I've had 100% success rate.

sschueller 5mo ago

They probably can many things but I think things like memory timing is something you can't just easily reverse engineer from a blob. You need to test every state that the device can be in and see how the blob responds which is quite difficult.

himata4113 5mo ago
Why not? Those timings and general initialization are inside the blobs?

bradfa 5mo ago

Many silicon vendors, when providing said binary blobs to a device OEM or even just documentation or source code for the binary blobs, will make companies agree to a license or other legal terms which prohibits reverse engineering. Often the direct recipient of the binary blobs (the OEM of the device) cannot legally let their employees nor contractors perform the reverse engineering. Generally, unless a similar license or legal terms are required to be agreed to by the end user, nothing stops the end user from reversing said binary blobs. But before you attempt this, be sure you fully understand every legal document which was presented to you by the device vendor. Click-through EULAs included.

hoherd 5mo ago
What would Jon Lech Johansen do?

mschuster91 5mo ago

Some of this stuff you need to probe with very, very expensive equipment and fine-tune parameters, and to make it worse, RF performance also depends on the composition of the PCB layers, the amount of vias, hell even just rotating an unrelated component on a PCB can affect the RF performance of nearby components or traces. If you're into that, Hans Rosenberg has some darn good yt videos [3]. RF is a world of black magic, especially at the frequencies, symbol rates and encodings used for stuff like RAM. And the higher in frequency you go... the less "conventional wisdoms" apply. There's in fact a very old article from 2007 [1] describing the issue from the other end. Some researcher tried to have a primitive form of what we'd call "machine learning" a few years later write FPGA bitstream to get a tone discriminator. Turns out the algorithm and test harness got a working bitstream... but it made no sense at all, it was very finely tuned to individual physical characteristics of that chip. Link training blobs on modern chips do something very, very similar, at each link initialization they evaluate a lot of different parameters per pin to account for the current state the device is in to get the best (i.e. highest stable bit rate) possible link. And the parameters, value ranges and timings all vary between different chips, so if you write a blob for one combination of SoC and memory, it might very well be possible that you need an entirely different blob for another combination. And that is what the BSP vendor does and what is inside the blob... a tuned version of parameters that this specific board's firmware can use to achieve a working good link in the shortest possible time. It's one hell of a ride that Flipper is on, and I seriously wish them all the best. There's a darn good reason this stuff has been proprietary, at best you'll get a high-level summary like [2]. [1] https://www.damninteresting.com/on-the-origin-of-circuits/ https://www.damninteresting.com/on-the-origin-of-circuits/ [2] https://www.ti.com/lit/an/snla415/snla415.pdf?ts=1779305695379 https://www.ti.com/lit/an/snla415/snla415.pdf?ts=17793056953... [3] https://www.youtube.com/watch?v=xhuHAhIKWoM https://www.youtube.com/watch?v=xhuHAhIKWoM

himata4113 5mo ago
I understand that much, but we have the firmware - we don't have to extract it or rebuild it in a white-room? Or is some of that information baked into the firmware that we cannot flash? I've reversed some camera sensor binary blobs recently and I didn't have too many issues, only took a really long time (5 days) of running in a loop until source matched the original (sematically). I parsed the blobs into an intermediate language, exported to llvm and optimized it to recover the functional structure I also used this sematics structure to rebuild the source in rust(kinda, it's a variant of rust with snowflake features cause I hate the language). I think I got lucky because the sensors used the same mips opcodes (sv class) rather than some proprietary state machine.