4 ms·

Results of the 2015 Underhanded C Contest

71 points by uoaei 3y ago

lifthrasiir 3y ago
I like the 2008 winner most [1], especially because it is so simple and can happen to anyone but also very easy to overlook. And most other entries could have been prevented in other languages, this one isn't. [1] http://www.underhanded-c.org/_page_id_17.html http://www.underhanded-c.org/_page_id_17.html
- pests 3y ago
  I am bad at C, how is he doing multiples of 0 based on the intensity?
  - phreack 3y ago
    Took me a bit but they're replacing the string characters which are numbers with '0' and keeping the rest. So "3 102 45" becomes "0 000 00". Once I got it I thought it was brilliant.
    - jancsika 3y ago
      Such a great cover for the bug: the simplicity of the approach doesn't require reallocation. Also, the author could claim they inspected/analyzed the visual output and the redaction appeared flawless. At least, I assume most image viewers won't display a dialog: Warning: suspiciously redundant image data detected. If any part of this image is supposed to be redacted, please contact your system administrator immediately.
    - pests 3y ago
      Ohhh! That's also how it perserves some intensity information as well!
dwheeler 3y ago
Underhanded code is interesting and worrying. In April 2020 I wrote a paper on some ideas on how to counter underhanded code. Some countermeasures seem promising, especially when combined. More info here: "Initial Analysis of Underhanded Source Code", April, 2020, IDA document: D-13166, https://www.ida.org/research-and-publications/publications/all/i/in/initial-analysis-of-underhanded-source-code https://www.ida.org/research-and-publications/publications/a... I hope that some find it interesting, or even build on it.
- jancsika 3y ago
  I just took a look at Meacham's 2008 winner. After skimming your paper I believe it makes it through your countermeasures unscathed.
  - praptak 3y ago
    This is an insightful example because it's kind of "wrong at the higher level". It follows the spec. Every formal verification method will confirm that. It's the spec that is wrong. Obviously I mean the "replaces digits with zeros" spec, which does not follow the "doesn't leak data" spec. The root problem is that "doesn't leak data" is hard to define in a way that can be formally verified. This is a fundamental problem, not an incidental one.
    - tgv 3y ago
      Indeed. It looks as if it could have been written in Rust and Spark. > doesn't leak data This requirement was translated into "zero a rectangle" without considering every format. Careful analysis is clearly possible, it was done by the author of that code, but understanding the full implications of code is hard.
  - dwheeler 3y ago
    I doubt any set of countermeasures will guarantee catching everything. I think the goal should be to make it harder and riskier for attackers to successfully perform the attack. I can't be certain we can get there... but nothing gets better if we don't try to make it better.
1vuio0pswjnm7 3y ago
Is there an Underhanded Javascript Contest.
- neilv 3y ago
  Thousands of different Web browser surveillance companies.
- speps 3y ago
  It's called NPM.
  - jacquesm 3y ago
    That would be funny if it weren't so true. Or maybe that makes it more funny.
- praptak 3y ago
  Every JS program is enrolled by default (they used to make this joke about Perl and C++).
sn 3y ago
This was a lot of fun! I believe I copied and pasted a buggy method for detecting the number of processors from stack overflow.
dang 3y ago
Related: The Underhanded C Contest - https://news.ycombinator.com/item?id=29347859 https://news.ycombinator.com/item?id=29347859 - Nov 2021 (11 comments) Results of the 2015 Underhanded C Contest - https://news.ycombinator.com/item?id=11032542 https://news.ycombinator.com/item?id=11032542 - Feb 2016 (47 comments) Winners of the 7th Underhanded C Contest - https://news.ycombinator.com/item?id=9670785 https://news.ycombinator.com/item?id=9670785 - June 2015 (44 comments) The 7th Underhanded C Contest is now Open - https://news.ycombinator.com/item?id=8550227 https://news.ycombinator.com/item?id=8550227 - Nov 2014 (3 comments)
raverbashing 3y ago
But then there hasn't been any newer contests after 2015? (It seems not)
saulrh 3y ago
I've always found it... somewhat unfitting, I think... that this is called the "underhanded" C contest, because so many of the winners work by getting trivial type and preprocessor stuff wrong in ways that are concerningly easy for the average programmer. Like, the 2015 winner interpreting an array of n doubles as an array of 2n floats because it screwed up a typedef of float_t, or the 2008 winner reading 3 bits out of something because it left the parentheses off of a macro definition. These are core operations for using the language as intended! There's nothing fancy going on! This is code that you, yes, you, could have written following every best practice on earth. The contest is almost tailor-made to produce evidence that C should never be used for anything that requires reliability or correctness.
- lovich 3y ago
  Just to make sure this was something you were considering when I internet your comment. You know that the two examples you referenced are from 9 and then 16 years ago right? Is this an increasing problem or one from a previous age of software? This is a legit question. I work with managed languages for web development. I’ve only looked at c/c++/assembly in high school classes
  - londons_explore 3y ago
    Low level languages might not be used in many applications these days, but it is still very much the core of OS's, drivers, embedded, etc. Those things are still being made today, and the impact of a security flaw sneaks in is probably bigger than a flaw in application code.
    - agsnu 3y ago
      C++ is still widely used for applications in some industries (games, finance), although this is less true than it was a decade ago
- rcxdude 3y ago
  I think that is kind of the point. It's putting the 'accidents happen' aspect of C into an adversarial environment, and it turns out in a language it's easy to write bugs in it's really easy to hide deliberate bugs.
ThinkBeat 3y ago
I am sad this is not still an active project. Perhaps one could be made for underhanded Rust as well.
- pornel 3y ago
  There was one, but it was underwhelming. Rust's macros and scopes are barely confusing enough, nothing like a text-based preprocessor. It lacks ifs with non-bracketed bodies, implicit integer/pointer conversions, goto, surprising operator precedence, silent ignoring of returned errors, and other quirks that make Underhanded contest work.
mrparrot 3y ago
The worrying part is that the winning entry would bust immediately if someone wrote a single test to check if the computed numbers are correct. I'm not sure what I find more disconcerting: That the judges considered this a feasible attack or that they are probably right.
38 3y ago
> Stephen Dolan I wonder if its JQ man: https://github.com/stedolan https://github.com/stedolan

71 points by uoaei 3y ago

I like the 2008 winner most [1], especially because it is so simple and can happen to anyone but also very easy to overlook. And most other entries could have been prevented in other languages, this one isn't. [1] http://www.underhanded-c.org/_page_id_17.html http://www.underhanded-c.org/_page_id_17.html

pests 3y ago
I am bad at C, how is he doing multiples of 0 based on the intensity?
- phreack 3y ago
  Took me a bit but they're replacing the string characters which are numbers with '0' and keeping the rest. So "3 102 45" becomes "0 000 00". Once I got it I thought it was brilliant.
  - jancsika 3y ago
    Such a great cover for the bug: the simplicity of the approach doesn't require reallocation. Also, the author could claim they inspected/analyzed the visual output and the redaction appeared flawless. At least, I assume most image viewers won't display a dialog: Warning: suspiciously redundant image data detected. If any part of this image is supposed to be redacted, please contact your system administrator immediately.
  - pests 3y ago
    Ohhh! That's also how it perserves some intensity information as well!

dwheeler 3y ago

Underhanded code is interesting and worrying. In April 2020 I wrote a paper on some ideas on how to counter underhanded code. Some countermeasures seem promising, especially when combined. More info here: "Initial Analysis of Underhanded Source Code", April, 2020, IDA document: D-13166, https://www.ida.org/research-and-publications/publications/all/i/in/initial-analysis-of-underhanded-source-code https://www.ida.org/research-and-publications/publications/a... I hope that some find it interesting, or even build on it.

jancsika 3y ago
I just took a look at Meacham's 2008 winner. After skimming your paper I believe it makes it through your countermeasures unscathed.
- praptak 3y ago
  This is an insightful example because it's kind of "wrong at the higher level". It follows the spec. Every formal verification method will confirm that. It's the spec that is wrong. Obviously I mean the "replaces digits with zeros" spec, which does not follow the "doesn't leak data" spec. The root problem is that "doesn't leak data" is hard to define in a way that can be formally verified. This is a fundamental problem, not an incidental one.
  - tgv 3y ago
    Indeed. It looks as if it could have been written in Rust and Spark. > doesn't leak data This requirement was translated into "zero a rectangle" without considering every format. Careful analysis is clearly possible, it was done by the author of that code, but understanding the full implications of code is hard.
- dwheeler 3y ago
  I doubt any set of countermeasures will guarantee catching everything. I think the goal should be to make it harder and riskier for attackers to successfully perform the attack. I can't be certain we can get there... but nothing gets better if we don't try to make it better.

1vuio0pswjnm7 3y ago

Is there an Underhanded Javascript Contest.

neilv 3y ago
Thousands of different Web browser surveillance companies.
speps 3y ago
It's called NPM.
- jacquesm 3y ago
  That would be funny if it weren't so true. Or maybe that makes it more funny.
praptak 3y ago
Every JS program is enrolled by default (they used to make this joke about Perl and C++).

sn 3y ago

This was a lot of fun! I believe I copied and pasted a buggy method for detecting the number of processors from stack overflow.

dang 3y ago

Related: The Underhanded C Contest - https://news.ycombinator.com/item?id=29347859 https://news.ycombinator.com/item?id=29347859 - Nov 2021 (11 comments) Results of the 2015 Underhanded C Contest - https://news.ycombinator.com/item?id=11032542 https://news.ycombinator.com/item?id=11032542 - Feb 2016 (47 comments) Winners of the 7th Underhanded C Contest - https://news.ycombinator.com/item?id=9670785 https://news.ycombinator.com/item?id=9670785 - June 2015 (44 comments) The 7th Underhanded C Contest is now Open - https://news.ycombinator.com/item?id=8550227 https://news.ycombinator.com/item?id=8550227 - Nov 2014 (3 comments)

raverbashing 3y ago

But then there hasn't been any newer contests after 2015? (It seems not)

saulrh 3y ago

I've always found it... somewhat unfitting, I think... that this is called the "underhanded" C contest, because so many of the winners work by getting trivial type and preprocessor stuff wrong in ways that are concerningly easy for the average programmer. Like, the 2015 winner interpreting an array of n doubles as an array of 2n floats because it screwed up a typedef of float_t, or the 2008 winner reading 3 bits out of something because it left the parentheses off of a macro definition. These are core operations for using the language as intended! There's nothing fancy going on! This is code that you, yes, you, could have written following every best practice on earth. The contest is almost tailor-made to produce evidence that C should never be used for anything that requires reliability or correctness.

lovich 3y ago
Just to make sure this was something you were considering when I internet your comment. You know that the two examples you referenced are from 9 and then 16 years ago right? Is this an increasing problem or one from a previous age of software? This is a legit question. I work with managed languages for web development. I’ve only looked at c/c++/assembly in high school classes
- londons_explore 3y ago
  Low level languages might not be used in many applications these days, but it is still very much the core of OS's, drivers, embedded, etc. Those things are still being made today, and the impact of a security flaw sneaks in is probably bigger than a flaw in application code.
  - agsnu 3y ago
    C++ is still widely used for applications in some industries (games, finance), although this is less true than it was a decade ago
rcxdude 3y ago
I think that is kind of the point. It's putting the 'accidents happen' aspect of C into an adversarial environment, and it turns out in a language it's easy to write bugs in it's really easy to hide deliberate bugs.

ThinkBeat 3y ago

I am sad this is not still an active project. Perhaps one could be made for underhanded Rust as well.

pornel 3y ago
There was one, but it was underwhelming. Rust's macros and scopes are barely confusing enough, nothing like a text-based preprocessor. It lacks ifs with non-bracketed bodies, implicit integer/pointer conversions, goto, surprising operator precedence, silent ignoring of returned errors, and other quirks that make Underhanded contest work.

mrparrot 3y ago

The worrying part is that the winning entry would bust immediately if someone wrote a single test to check if the computed numbers are correct. I'm not sure what I find more disconcerting: That the judges considered this a feasible attack or that they are probably right.

38 3y ago

> Stephen Dolan I wonder if its JQ man: https://github.com/stedolan https://github.com/stedolan