7 ms·

Chaff Bugs: Deterring Attackers by Making Software Buggier

100 points by fniephaus 8y ago

hyper_reality 8y ago
The idea is quite fascinating and 'out there'. However, the authors themselves cover several clear drawbacks on the second page: > We assume that the attacker has access to a compiled (binary) version of the program but not its source code. Although this is a severe limitation (it means that our technique cannot be used to protect open source software), it is a necessary one. Developers are unlikely to be willing to work with source code that has had extra bugs added to it, and more importantly future changes to the code may cause previously non-exploitable bugs to become exploitable. Hence we see our system as useful primarily as an extra stage in the build process, adding non-exploitable bugs. I wonder if running software through this additional build step would also cause headaches for developers debugging and triaging real bugs in the software.
satokema 8y ago
"Half the electronic engineers in the galaxy are constantly trying to find fresh ways of jamming the signals generated by the Thumb, while the other half are constantly trying to find fresh ways of jamming the jamming signals."
sitkack 8y ago
One could think of this as a whitelist for allowed flow control paths. This "optimization" is analogous to the same technique. https://fgiesen.wordpress.com/2012/04/08/metaprogramming-for-madmen/ https://fgiesen.wordpress.com/2012/04/08/metaprogramming-for... https://news.ycombinator.com/item?id=7739599 https://news.ycombinator.com/item?id=7739599 Control Flow Integrity should be built into modern processors.
56chan4 8y ago
Chaff bugs like something from WW2 is a valid technique of obfuscation, provided it doesnt eat into the time and money of the key revenue making activity. Whilst equates, libraries and OOP can reduce the incidence of bugs, perhaps the whole method of coding is flawed from the off? If you can abstract the code enough, you can reduce the incidence of bugs by many factors, but there is only one IDE/language I know of which seriously reduces the incidence of bugs and thats called Clarion by Softvelocity. Its old school (dos) DB orientated but the templates reduce the incidence of bugs but in my experience coders cant get their heads around the abstract nature of the templates. The templates not only give you standardised coding (in any language if someone wants to invest in developing templates for other languages), this means its easy to write programs which can rehash some of the exported app & dct code. In turn every customer/end user can have an identical in function but unique in code and thus hashed EXE to run on their own systems making the exploitation of bugs even harder. I think templates put most programmers off, but its the crown jewels for taking most business systems to the next level.
DSingularity 8y ago
This could be a viable direction but I have some concerns. I feel it will simply push attackers to be selective and it will be difficult for these techniques to avoid standing out. For example, consider their technique of making things non exploitable by overwriting unused variables. What’s to stop adversaries from prioritizing which bug to investigate based on profile data counting uses of variables? Now these chaff bugs need to somehow match the dynamic use counts of real variable. What about the over constraint? Well the authors point out that attacker’s can use SMT solvers to determine that a bug is safely constrained. They propose spreading out the constraining to force the attacker to reason about all paths. Again, Nothing stops the adversary from profiling the memory regions to learn what constraints are being applied followed by prioritizing the least constrained regions. While clever in that it tries to use fundamental limits of reasoning against the adversary, I need more experimental analysis before I am convinced that these chaff bugs are actually indistinguishable to an attacker unless they truly collect profiling data that is exponential in size of programming paths. I think that a few runs would suffice to distinguish the chaff bugs and real bugs.
- heavenlyblue 8y ago
  I think you're missing the point here: the exploitable bugs in their nature will exist forever. As you are either giving up the Turing-completeness, or are allowing them to possibly exist. Just like it were in 2000s: the fight between the people who were capable of unpacking the executables from their respective defensive mechanisms and the people who implement various mechanisms of obfuscating the executable. For example: online en/decryption of the code, or the later stages of this concept implemented first in Assassin's Creed, where the servers would store various connecting components between stages of the game. So you would need to reach point A under constraints C1 and C2, before the server would tell you what follows A. My point is, you can't take defensive mechanisms implemented in software at their extreme. Simply because of Turing-completeness. The moment we've got the tools that could, under a certain amount of constraints, prove that you can never execute any code that have not been compiled into the executable - is the moment when we don't actually need to develop any new software to defend ourselves.
  - 8y ago
snek 8y ago
i feel so bad for security researchers trying to actually make software better when meanwhile stuff like this is being worked on.
eatitraw 8y ago
Imaging having a bug in your software that introduces bugs. Instead of introducing new non-exploitable bugs, it introduces exploitable ones. This would probably be a nightmare to debug.
jajag 8y ago
Vaguely reminiscent of Ron Rivest's chaffing & winnowing scheme for secure communication from twenty years ago: http://people.csail.mit.edu/rivest/chaffing-980701.txt http://people.csail.mit.edu/rivest/chaffing-980701.txt
hw 8y ago
Is this a just a honeypot or a variant of one where you lead the attacker to believe they've succeeded, instead of actually making them succeed with an actual bug that leads nowhere?
- adrianmonk 8y ago
  It sounds like you just overwhelm them with dead-ends so they don't have the resources to exhaustively search for the things that aren't dead ends.
kardos 8y ago
Alternate subtitle: "Complicated method of shutting down your bug bounty program"
WalterBright 8y ago
Back in the olden daze, people would take branded software, overwrite the copyright notice, and sell the result as their own "compatible" version. On my game, it would check to see that the copyright notice was intact. If it wasn't, it would, at random times, overwrite a random byte in memory with a random value. The idea was for the "rebranded" game to appear to be uselessy buggy. I have no idea if it was effective or not. Probably not.
- your-nanny 8y ago
  funny at least
  - WalterBright 8y ago
    The trick was to corrupt memory at a slow enough rate that the crackers wouldn't notice it (and so wouldn't defeat it), but their users would.
- Boulth 8y ago
  It reminds me of the old "protection" in Settlers: when using an illegal game everything would work as normal except the iron smith would produce pigs instead of iron bars. That'd be surely missed by early crackers but definitely not by users.
  - MrEldritch 8y ago
    Earthbound had a particularly punitive bit of hard-to-notice copy protection - it would be exactly identical up until right before the final boss, and then immediately crash and delete your save. At which point it isn't so much copy protection, but copy ... karmic punishment?
- brokenmachine 8y ago
  Probably the users just thought the game was buggy normally and were relieved they didn't buy it.
paulie_a 8y ago
I still like what 3d studio Max did in the 90s. The gradually degraded the polygon of a model over the course of months when a pirated copy was detected. The other funny one was with Adobe and technical support..."sorry but there was no Photoshop 4.5.1, and now we know you are pirating our software"
- michaelmrose 8y ago
  I don't think vandalizing users work is the same as making a game buggy to take it to the extreme what if windows if it decided your key was illegitimate decided to corrupt your files. You could say they had no obligation to you since you didn't pay for the software certainly they have no obligation to run for example or not to crash or lock the user out but it seems like once you start destroying user data you are violating the users rights to their own property.
  - paulie_a 8y ago
    I disagree, you had no actual right to create the property in the first place. It's like trespassing and the actual owner grabbing a dumpster and cleaning out your stuff. Literally the user has no rights in that case.
    - michaelmrose 8y ago
      Property rights including intellectual property rights don't attach to the tools used to create them. This is why your drawings aren't copyright bic and bram moolenaar isn't a billionaire. What property rights do attach do is clearly peoples physical property with precedent going back thousands of years before we dreamed of copyright. By extension the files on said property ought to be secure against vandalism courtesy of more recent rules. Believing that a running application on another persons computer transforms their physical property into your property requires magical thinking. The IP right is an orthogonal issue that has no bearing on their right to their own machine and their own files. Its as if you stole my pen and went on to write a manuscript and I broke into your house and burned it. Both acts are orthogonal and the ethics don't intersect in the wholly magical way you imagine. I would owe you one pen or the money to buy one and I would owe you for your manuscript plus owe the country for the crime of breaking and entry or in this case inappropriate access to a computer.
matte_black 8y ago
The game Earthbound for SNES had code that detected if it was pirated or not. If the game knew it was pirated, it would still quietly run... But it would make the enemies so difficult and powerful that you could barely get out of the first town or even enjoy the game. Oh and as a bonus it would delete your whole game right before the final boss.
- deleted 8y ago
  [deleted]
chris_wot 8y ago
If you can provable show that the bug is non-exploitable, then can’t the attacker?
- moyix 8y ago
  This is why the bugs are made non-exploitable by construction rather than by doing a bunch of work to show they're non-exploitable. There's an information asymmetry at work: the injector can craft all bugs in such a way as to ensure their non-exploitability, but the attacker has to laboriously prove the non-exploitability of each one.
joemag 8y ago
Interesting idea, but the article greatly downplays the dos angle. Most applications consume fair amount of resources to start up - there are config files to parse, caches to warm, and so on. So there is an imbalance of cost between an attacker, who has to submit another input, and target,who has to restart the process. Hence a great dos angle. Pooling only helps if you can replenish the pool faster than the attacker can drain it. So if you running a multi-tenant service, where one of the tenants could be a potential adversary, having a crash-bug is enough for a serious denial of service attack.
- moyix 8y ago
  Yep, there are plenty of applications where crashing is a big deal! But, we think, also plenty where that's not an issue. Note also that the "unused" strategy creates bugs that don't cause crashes (but they could be detected with something like ASAN).

100 points by fniephaus 8y ago

The idea is quite fascinating and 'out there'. However, the authors themselves cover several clear drawbacks on the second page: > We assume that the attacker has access to a compiled (binary) version of the program but not its source code. Although this is a severe limitation (it means that our technique cannot be used to protect open source software), it is a necessary one. Developers are unlikely to be willing to work with source code that has had extra bugs added to it, and more importantly future changes to the code may cause previously non-exploitable bugs to become exploitable. Hence we see our system as useful primarily as an extra stage in the build process, adding non-exploitable bugs. I wonder if running software through this additional build step would also cause headaches for developers debugging and triaging real bugs in the software.

satokema 8y ago

"Half the electronic engineers in the galaxy are constantly trying to find fresh ways of jamming the signals generated by the Thumb, while the other half are constantly trying to find fresh ways of jamming the jamming signals."

sitkack 8y ago

One could think of this as a whitelist for allowed flow control paths. This "optimization" is analogous to the same technique. https://fgiesen.wordpress.com/2012/04/08/metaprogramming-for-madmen/ https://fgiesen.wordpress.com/2012/04/08/metaprogramming-for... https://news.ycombinator.com/item?id=7739599 https://news.ycombinator.com/item?id=7739599 Control Flow Integrity should be built into modern processors.

56chan4 8y ago

Chaff bugs like something from WW2 is a valid technique of obfuscation, provided it doesnt eat into the time and money of the key revenue making activity. Whilst equates, libraries and OOP can reduce the incidence of bugs, perhaps the whole method of coding is flawed from the off? If you can abstract the code enough, you can reduce the incidence of bugs by many factors, but there is only one IDE/language I know of which seriously reduces the incidence of bugs and thats called Clarion by Softvelocity. Its old school (dos) DB orientated but the templates reduce the incidence of bugs but in my experience coders cant get their heads around the abstract nature of the templates. The templates not only give you standardised coding (in any language if someone wants to invest in developing templates for other languages), this means its easy to write programs which can rehash some of the exported app & dct code. In turn every customer/end user can have an identical in function but unique in code and thus hashed EXE to run on their own systems making the exploitation of bugs even harder. I think templates put most programmers off, but its the crown jewels for taking most business systems to the next level.

DSingularity 8y ago

This could be a viable direction but I have some concerns. I feel it will simply push attackers to be selective and it will be difficult for these techniques to avoid standing out. For example, consider their technique of making things non exploitable by overwriting unused variables. What’s to stop adversaries from prioritizing which bug to investigate based on profile data counting uses of variables? Now these chaff bugs need to somehow match the dynamic use counts of real variable. What about the over constraint? Well the authors point out that attacker’s can use SMT solvers to determine that a bug is safely constrained. They propose spreading out the constraining to force the attacker to reason about all paths. Again, Nothing stops the adversary from profiling the memory regions to learn what constraints are being applied followed by prioritizing the least constrained regions. While clever in that it tries to use fundamental limits of reasoning against the adversary, I need more experimental analysis before I am convinced that these chaff bugs are actually indistinguishable to an attacker unless they truly collect profiling data that is exponential in size of programming paths. I think that a few runs would suffice to distinguish the chaff bugs and real bugs.

heavenlyblue 8y ago
I think you're missing the point here: the exploitable bugs in their nature will exist forever. As you are either giving up the Turing-completeness, or are allowing them to possibly exist. Just like it were in 2000s: the fight between the people who were capable of unpacking the executables from their respective defensive mechanisms and the people who implement various mechanisms of obfuscating the executable. For example: online en/decryption of the code, or the later stages of this concept implemented first in Assassin's Creed, where the servers would store various connecting components between stages of the game. So you would need to reach point A under constraints C1 and C2, before the server would tell you what follows A. My point is, you can't take defensive mechanisms implemented in software at their extreme. Simply because of Turing-completeness. The moment we've got the tools that could, under a certain amount of constraints, prove that you can never execute any code that have not been compiled into the executable - is the moment when we don't actually need to develop any new software to defend ourselves.
- 8y ago

snek 8y ago

i feel so bad for security researchers trying to actually make software better when meanwhile stuff like this is being worked on.

eatitraw 8y ago

Imaging having a bug in your software that introduces bugs. Instead of introducing new non-exploitable bugs, it introduces exploitable ones. This would probably be a nightmare to debug.

jajag 8y ago

Vaguely reminiscent of Ron Rivest's chaffing & winnowing scheme for secure communication from twenty years ago: http://people.csail.mit.edu/rivest/chaffing-980701.txt http://people.csail.mit.edu/rivest/chaffing-980701.txt

hw 8y ago

Is this a just a honeypot or a variant of one where you lead the attacker to believe they've succeeded, instead of actually making them succeed with an actual bug that leads nowhere?

adrianmonk 8y ago
It sounds like you just overwhelm them with dead-ends so they don't have the resources to exhaustively search for the things that aren't dead ends.

kardos 8y ago

Alternate subtitle: "Complicated method of shutting down your bug bounty program"

WalterBright 8y ago

Back in the olden daze, people would take branded software, overwrite the copyright notice, and sell the result as their own "compatible" version. On my game, it would check to see that the copyright notice was intact. If it wasn't, it would, at random times, overwrite a random byte in memory with a random value. The idea was for the "rebranded" game to appear to be uselessy buggy. I have no idea if it was effective or not. Probably not.

your-nanny 8y ago
funny at least
- WalterBright 8y ago
  The trick was to corrupt memory at a slow enough rate that the crackers wouldn't notice it (and so wouldn't defeat it), but their users would.
Boulth 8y ago
It reminds me of the old "protection" in Settlers: when using an illegal game everything would work as normal except the iron smith would produce pigs instead of iron bars. That'd be surely missed by early crackers but definitely not by users.
- MrEldritch 8y ago
  Earthbound had a particularly punitive bit of hard-to-notice copy protection - it would be exactly identical up until right before the final boss, and then immediately crash and delete your save. At which point it isn't so much copy protection, but copy ... karmic punishment?
brokenmachine 8y ago
Probably the users just thought the game was buggy normally and were relieved they didn't buy it.

paulie_a 8y ago

I still like what 3d studio Max did in the 90s. The gradually degraded the polygon of a model over the course of months when a pirated copy was detected. The other funny one was with Adobe and technical support..."sorry but there was no Photoshop 4.5.1, and now we know you are pirating our software"

michaelmrose 8y ago
I don't think vandalizing users work is the same as making a game buggy to take it to the extreme what if windows if it decided your key was illegitimate decided to corrupt your files. You could say they had no obligation to you since you didn't pay for the software certainly they have no obligation to run for example or not to crash or lock the user out but it seems like once you start destroying user data you are violating the users rights to their own property.
- paulie_a 8y ago
  I disagree, you had no actual right to create the property in the first place. It's like trespassing and the actual owner grabbing a dumpster and cleaning out your stuff. Literally the user has no rights in that case.
  - michaelmrose 8y ago
    Property rights including intellectual property rights don't attach to the tools used to create them. This is why your drawings aren't copyright bic and bram moolenaar isn't a billionaire. What property rights do attach do is clearly peoples physical property with precedent going back thousands of years before we dreamed of copyright. By extension the files on said property ought to be secure against vandalism courtesy of more recent rules. Believing that a running application on another persons computer transforms their physical property into your property requires magical thinking. The IP right is an orthogonal issue that has no bearing on their right to their own machine and their own files. Its as if you stole my pen and went on to write a manuscript and I broke into your house and burned it. Both acts are orthogonal and the ethics don't intersect in the wholly magical way you imagine. I would owe you one pen or the money to buy one and I would owe you for your manuscript plus owe the country for the crime of breaking and entry or in this case inappropriate access to a computer.

matte_black 8y ago

The game Earthbound for SNES had code that detected if it was pirated or not. If the game knew it was pirated, it would still quietly run... But it would make the enemies so difficult and powerful that you could barely get out of the first town or even enjoy the game. Oh and as a bonus it would delete your whole game right before the final boss.

deleted 8y ago
[deleted]

chris_wot 8y ago

If you can provable show that the bug is non-exploitable, then can’t the attacker?

moyix 8y ago
This is why the bugs are made non-exploitable by construction rather than by doing a bunch of work to show they're non-exploitable. There's an information asymmetry at work: the injector can craft all bugs in such a way as to ensure their non-exploitability, but the attacker has to laboriously prove the non-exploitability of each one.

joemag 8y ago

Interesting idea, but the article greatly downplays the dos angle. Most applications consume fair amount of resources to start up - there are config files to parse, caches to warm, and so on. So there is an imbalance of cost between an attacker, who has to submit another input, and target,who has to restart the process. Hence a great dos angle. Pooling only helps if you can replenish the pool faster than the attacker can drain it. So if you running a multi-tenant service, where one of the tenants could be a potential adversary, having a crash-bug is enough for a serious denial of service attack.

moyix 8y ago
Yep, there are plenty of applications where crashing is a big deal! But, we think, also plenty where that's not an issue. Note also that the "unused" strategy creates bugs that don't cause crashes (but they could be detected with something like ASAN).