6 ms·

Considering file is supposed to be invoked on unknown files, I fail to see how tests other than fuzzing would give any assurance security wise. A very anal code

by brohee 12y ago

Considering file is supposed to be invoked on unknown files, I fail to see how tests other than fuzzing would give any assurance security wise. A very anal code review on the other hand... And the OpenBSD people are very aware of fuzzing, this new implementation of file is a direct reaction to Michal Zalewski findings...

randomfool 12y ago
Fuzzing is like rolling dice- there's only a chance it'll find the lurking issues. Unit tests enforce contracts and assist future maintenance. I sincerely hope the lack of unit tests is because it's covered by existing tests.
- e12e 12y ago
  Well, with tools like AFL (American Fuzzy Loop) it is more like rolling dice to find a path through a maze of numbered doors - adjusting the dice rolled (6-sided, 2-sided ...) based on how far one has come, and how many doors there is to choose from... ok, it's not really like rolling dice any more at all. Did you read: http://lcamtuf.blogspot.no/2014/11/pulling-jpegs-out-of-thin-air.html http://lcamtuf.blogspot.no/2014/11/pulling-jpegs-out-of-thin...
  - Animats 12y ago
    "American Fuzzy Lop" - it's named after a breed of rabbit where the ears sag down.
    - e12e 12y ago
      And here I thought it was named after some obscure cereal brand ;-)
cnst 12y ago
Interestingly, the old file(1) was also written by an OpenBSD developer -- Ian Darwin of Toronto. I met him at a BSDCan, nice chap. http://mdoc.su/f/file.1 http://mdoc.su/f/file.1 http://darwinsys.com/file/ http://darwinsys.com/file/ Apparently, Ian no longer considers himself a maintainer, devoting such title to Christos Zoulas of NetBSD and Two Sigma.
thrownaway2424 12y ago
A good and systematic approach to incorporate fuzzing findings would be as follows: 1) Identify a harmful input through fuzzing 2) Reduce the input to minimal testcase 3) Contribute testcase and fix to existing code Note the absence of "rewrite the whole thing without tests" in this process. From-scratch rewrites that may or may not have fewer bugs than their predecessors are known as CADT. http://www.jwz.org/doc/cadt.html http://www.jwz.org/doc/cadt.html
- userbinator 12y ago
  CADT refers mostly to rewrites in which lots of new features and complexity are being added - the most likely sources of bugs - not to ones like this that go in the opposite direction.
  - acqq 12y ago
    Let me quote from the source of the CADT term: http://www.jwz.org/doc/cadt.html http://www.jwz.org/doc/cadt.html "This is, I think, the most common way for my bug reports to open source software projects to ever become closed. I report bugs; they go unread for a year, sometimes two; and then (surprise!) that module is rewritten from scratch -- and the new maintainer can't be bothered to check whether his new version has actually solved any of the known problems that existed in the previous version." The OP rightly points that there are no known tests that demonstrate the known problems in the older implementations and also no proofs that the new implementation passes them.
    - raverbashing 12y ago
      Don't compare the Gnome development process with the OpenBSD development process. They are very different. Especially one being a core utility another a Desktop Environment. I agree with that critique of the Gnome development process, but the OpenBSD development process prides itself on security and that's a first commit, new changes are being added (as it can be seen by the log of that file - http://cvsweb.openbsd.org/cgi-bin/cvsweb/src/usr.bin/file/file.c?sortby=date http://cvsweb.openbsd.org/cgi-bin/cvsweb/src/usr.bin/file/fi... )
perlgeek 12y ago
Security isn't the only concern. What about correctness? Tests could help there. I'm sure I'll manage to write a secure file(1) implementation, if it's not required to produce correct output.

by brohee 12y ago

randomfool 12y ago

Fuzzing is like rolling dice- there's only a chance it'll find the lurking issues. Unit tests enforce contracts and assist future maintenance. I sincerely hope the lack of unit tests is because it's covered by existing tests.

e12e 12y ago
Well, with tools like AFL (American Fuzzy Loop) it is more like rolling dice to find a path through a maze of numbered doors - adjusting the dice rolled (6-sided, 2-sided ...) based on how far one has come, and how many doors there is to choose from... ok, it's not really like rolling dice any more at all. Did you read: http://lcamtuf.blogspot.no/2014/11/pulling-jpegs-out-of-thin-air.html http://lcamtuf.blogspot.no/2014/11/pulling-jpegs-out-of-thin...
- Animats 12y ago
  "American Fuzzy Lop" - it's named after a breed of rabbit where the ears sag down.
  - e12e 12y ago
    And here I thought it was named after some obscure cereal brand ;-)

cnst 12y ago

Interestingly, the old file(1) was also written by an OpenBSD developer -- Ian Darwin of Toronto. I met him at a BSDCan, nice chap. http://mdoc.su/f/file.1 http://mdoc.su/f/file.1 http://darwinsys.com/file/ http://darwinsys.com/file/ Apparently, Ian no longer considers himself a maintainer, devoting such title to Christos Zoulas of NetBSD and Two Sigma.

thrownaway2424 12y ago

A good and systematic approach to incorporate fuzzing findings would be as follows: 1) Identify a harmful input through fuzzing 2) Reduce the input to minimal testcase 3) Contribute testcase and fix to existing code Note the absence of "rewrite the whole thing without tests" in this process. From-scratch rewrites that may or may not have fewer bugs than their predecessors are known as CADT. http://www.jwz.org/doc/cadt.html http://www.jwz.org/doc/cadt.html

userbinator 12y ago
CADT refers mostly to rewrites in which lots of new features and complexity are being added - the most likely sources of bugs - not to ones like this that go in the opposite direction.
- acqq 12y ago
  Let me quote from the source of the CADT term: http://www.jwz.org/doc/cadt.html http://www.jwz.org/doc/cadt.html "This is, I think, the most common way for my bug reports to open source software projects to ever become closed. I report bugs; they go unread for a year, sometimes two; and then (surprise!) that module is rewritten from scratch -- and the new maintainer can't be bothered to check whether his new version has actually solved any of the known problems that existed in the previous version." The OP rightly points that there are no known tests that demonstrate the known problems in the older implementations and also no proofs that the new implementation passes them.
  - raverbashing 12y ago
    Don't compare the Gnome development process with the OpenBSD development process. They are very different. Especially one being a core utility another a Desktop Environment. I agree with that critique of the Gnome development process, but the OpenBSD development process prides itself on security and that's a first commit, new changes are being added (as it can be seen by the log of that file - http://cvsweb.openbsd.org/cgi-bin/cvsweb/src/usr.bin/file/file.c?sortby=date http://cvsweb.openbsd.org/cgi-bin/cvsweb/src/usr.bin/file/fi... )

perlgeek 12y ago

Security isn't the only concern. What about correctness? Tests could help there. I'm sure I'll manage to write a secure file(1) implementation, if it's not required to produce correct output.