5 ms·

I've never done any fuzzing personally, but wouldn't this be discoverable internally during testing? I would think that they would attempt sending all possible

I've never done any fuzzing personally, but wouldn't this be discoverable internally during testing? I would think that they would attempt sending all possible characters, especially because they've had issues with this in the past.

benmmurphy 9y ago
it looks like the single character was made up of 4 utf8 code points. so might have been trickier to detect from fuzzing than it looks.
- q3k 9y ago
  Don't knock instrumentation-based fuzzing when it comes to discovering complex edgecases and code paths. I've used AFL [1] to generate semantically valid, crashing multi-statement SQL queries when given ~50 CPU days and a simplistic SQL-like database. All from thin air, with an empty text file as the initial testcase. It can also come up with valid JPEGs [2] given enough time. [1] - http://lcamtuf.coredump.cx/afl/ http://lcamtuf.coredump.cx/afl/ [2] - https://lcamtuf.blogspot.ie/2014/11/pulling-jpegs-out-of-thin-air.html https://lcamtuf.blogspot.ie/2014/11/pulling-jpegs-out-of-thi...
mattnewton 9y ago
I would hope that after all the character-crashing-everything bugs that have hit iOS there is a team dedicated to replacing the text rendering code with something more fault tolerant.
- yoz-y 9y ago
  In chrome I can see the character for a while before it crashes. That leads me to believe that the actual crash is not in the text rendering code but in some component above that receives some garbage at some point.
- gmueckl 9y ago
  I wonder if its the renderer or the font itself that is at fault here. Fonts are almost more code than data these days.
valarauca1 9y ago
the issue is in the compositor, what determines where the windows are that contains all your GUI widgets. You can reproduce the bug on some of the newer versions of OSX. Likely the next two programs are disagreeing on character, and therefore widget size.
xoa 9y ago
>I've never done any fuzzing personally, but wouldn't this be discoverable internally during testing? Yes. This seems like a somewhat embarrassing fail in terms of whatever automated testing Apple does. It's a pretty minimal and restricted case of fuzzing, in fact I'm not sure it'd qualify as "fuzzing" at all because here the crasher is just an actual Unicode character, part of a fixed set that can be completely run through in deterministic time. Unicode is certainly large in human terms but in terms of automated test data sets it's not. Given the notorious difficulties and edge cases Unicode parsing has long created, testing every single individual character in Unicode as part of standard unit/regression testing seems like something that should just be done as a bare minimum for an operating system or parsing framework/library release. That's not to say there wouldn't be more complex inputs that would cause problems that sufficient fuzzing could discover, but this kind of a error from a single real language character shouldn't have slipped through, it's not random. Unicode and image/document handling frameworks are both areas that should have extensive fuzzing as well as deterministic input sets for testing. They shouldn't need to be messed with much either once they're "done", so in the case of an org like Apple this is the sort of place where it might even make sense to really devote resources to getting at least some parts of it formally verified.
- rspeer 9y ago
  I wouldn't be so quick to call Unicode characters a "fixed set", because of the way that they can be arbitrarily composed from multiple codepoints. If you've encountered zalgo-text, for example, you've encountered characters made from a base letter and like 20 combining codepoints. Enumerating all of the possible characters is infeasible. To be specific, here are the codepoints in the character from Pastebin: U+0C1C [Lo] TELUGU LETTER JA U+0C4D [Mn] TELUGU SIGN VIRAMA U+0C1E [Lo] TELUGU LETTER NYA U+200C [Cf] ZERO WIDTH NON-JOINER U+0C3E [Mn] TELUGU VOWEL SIGN AA I don't know if the ZWNJ is in there to make the bug happen or to neutralize it. But the entire group of five codepoints renders and selects as one character for me (in Chrome on Ubuntu).
jmull 9y ago
Well of course they could have done that. To be honest, though, if someone on my team suggested we implement an automated test that tries sending every Unicode character (and it would be applied to every interface of every app and API, right?) I would have objected that this was an over-complicated, over-engineered solution that will probably be too slow. I'd argue that a set of test data selected to cover a range of patterns, especially ones that are considered risky (either is know to have caused problems in the past or appear complicated or tricky) would have almost as good coverage and be an order of magnitude more useful. The problem with "run-every-case" tests is that they start off slow and get exponentially slower if you try to go deep you very quickly end up with tests that take too long to run to be useful. (e.g., {every Unicode character} is one thing... {Every Unicode character} X {every interface and app} is a LOT more. {Every Unicode character} X {every interface and app} X {every build} X {every device model} X etc. is impossible). So you end up with very broad but very shallow tests. And that means less coverage ultimately, not more. Generally, I think you'll get more efficient, effective, useful tests if you tailor the test data sets to the problems you see rather than going for blanket coverage.
- mitchty 9y ago
  > Generally, I think you'll get more efficient, effective, useful tests if you tailor the test data sets to the problems you see rather than going for blanket coverage. Sure, but fuzzing and "throw everything and the kitchen sink at it" type testing will generally expose logic bugs you weren't aware existed. Aka the unknown unknowns. Its not like this is an either/or proposition, you could run the fuzzing type gauntlet tests every week or so.
- wvenable 9y ago
  Since this affects a huge number of applications and even desktop applications, it's a low-level library that is causing the problem. I'm not sure it's unreasonable to test every unicode codepoint against shared library if that library is responsible for the rendering of unicode text. But I agree it would be unreasonable to do that for every app that uses the library.
  - 9y ago

benmmurphy 9y ago

it looks like the single character was made up of 4 utf8 code points. so might have been trickier to detect from fuzzing than it looks.

q3k 9y ago
Don't knock instrumentation-based fuzzing when it comes to discovering complex edgecases and code paths. I've used AFL [1] to generate semantically valid, crashing multi-statement SQL queries when given ~50 CPU days and a simplistic SQL-like database. All from thin air, with an empty text file as the initial testcase. It can also come up with valid JPEGs [2] given enough time. [1] - http://lcamtuf.coredump.cx/afl/ http://lcamtuf.coredump.cx/afl/ [2] - https://lcamtuf.blogspot.ie/2014/11/pulling-jpegs-out-of-thin-air.html https://lcamtuf.blogspot.ie/2014/11/pulling-jpegs-out-of-thi...

mattnewton 9y ago

I would hope that after all the character-crashing-everything bugs that have hit iOS there is a team dedicated to replacing the text rendering code with something more fault tolerant.

yoz-y 9y ago
In chrome I can see the character for a while before it crashes. That leads me to believe that the actual crash is not in the text rendering code but in some component above that receives some garbage at some point.
gmueckl 9y ago
I wonder if its the renderer or the font itself that is at fault here. Fonts are almost more code than data these days.

valarauca1 9y ago

the issue is in the compositor, what determines where the windows are that contains all your GUI widgets. You can reproduce the bug on some of the newer versions of OSX. Likely the next two programs are disagreeing on character, and therefore widget size.

xoa 9y ago

>I've never done any fuzzing personally, but wouldn't this be discoverable internally during testing? Yes. This seems like a somewhat embarrassing fail in terms of whatever automated testing Apple does. It's a pretty minimal and restricted case of fuzzing, in fact I'm not sure it'd qualify as "fuzzing" at all because here the crasher is just an actual Unicode character, part of a fixed set that can be completely run through in deterministic time. Unicode is certainly large in human terms but in terms of automated test data sets it's not. Given the notorious difficulties and edge cases Unicode parsing has long created, testing every single individual character in Unicode as part of standard unit/regression testing seems like something that should just be done as a bare minimum for an operating system or parsing framework/library release. That's not to say there wouldn't be more complex inputs that would cause problems that sufficient fuzzing could discover, but this kind of a error from a single real language character shouldn't have slipped through, it's not random. Unicode and image/document handling frameworks are both areas that should have extensive fuzzing as well as deterministic input sets for testing. They shouldn't need to be messed with much either once they're "done", so in the case of an org like Apple this is the sort of place where it might even make sense to really devote resources to getting at least some parts of it formally verified.

rspeer 9y ago
I wouldn't be so quick to call Unicode characters a "fixed set", because of the way that they can be arbitrarily composed from multiple codepoints. If you've encountered zalgo-text, for example, you've encountered characters made from a base letter and like 20 combining codepoints. Enumerating all of the possible characters is infeasible. To be specific, here are the codepoints in the character from Pastebin: U+0C1C [Lo] TELUGU LETTER JA U+0C4D [Mn] TELUGU SIGN VIRAMA U+0C1E [Lo] TELUGU LETTER NYA U+200C [Cf] ZERO WIDTH NON-JOINER U+0C3E [Mn] TELUGU VOWEL SIGN AA I don't know if the ZWNJ is in there to make the bug happen or to neutralize it. But the entire group of five codepoints renders and selects as one character for me (in Chrome on Ubuntu).

jmull 9y ago

Well of course they could have done that. To be honest, though, if someone on my team suggested we implement an automated test that tries sending every Unicode character (and it would be applied to every interface of every app and API, right?) I would have objected that this was an over-complicated, over-engineered solution that will probably be too slow. I'd argue that a set of test data selected to cover a range of patterns, especially ones that are considered risky (either is know to have caused problems in the past or appear complicated or tricky) would have almost as good coverage and be an order of magnitude more useful. The problem with "run-every-case" tests is that they start off slow and get exponentially slower if you try to go deep you very quickly end up with tests that take too long to run to be useful. (e.g., {every Unicode character} is one thing... {Every Unicode character} X {every interface and app} is a LOT more. {Every Unicode character} X {every interface and app} X {every build} X {every device model} X etc. is impossible). So you end up with very broad but very shallow tests. And that means less coverage ultimately, not more. Generally, I think you'll get more efficient, effective, useful tests if you tailor the test data sets to the problems you see rather than going for blanket coverage.

mitchty 9y ago
> Generally, I think you'll get more efficient, effective, useful tests if you tailor the test data sets to the problems you see rather than going for blanket coverage. Sure, but fuzzing and "throw everything and the kitchen sink at it" type testing will generally expose logic bugs you weren't aware existed. Aka the unknown unknowns. Its not like this is an either/or proposition, you could run the fuzzing type gauntlet tests every week or so.
wvenable 9y ago
Since this affects a huge number of applications and even desktop applications, it's a low-level library that is causing the problem. I'm not sure it's unreasonable to test every unicode codepoint against shared library if that library is responsible for the rendering of unicode text. But I agree it would be unreasonable to do that for every app that uses the library.
- 9y ago