9 ms·

The problem with bugs deep in the stack is that it is really time consuming to establish that they are in fact as deep as they are. I wrote a Swift iOS app onc

The problem with bugs deep in the stack is that it is really time consuming to establish that they are in fact as deep as they are. I wrote a Swift iOS app once, and came across an issue with one of the collection classes. Of course, nobody thinks that the Swift libs will be wrong as a first guess. So I worked through a number of hypotheses about my own code, slowly stripping out pieces that I thought might contain an error. And then combinations. I also tried reducing the number of entries just to simplify the logs. This worked, but of course you are not going to think that there's a library bug affecting collections with size > 16, and it wasn't actually a theory until I randomly decided to reduce the n. I also discovered that it worked just fine on release but not debug, so I thought maybe I have some race condition. More and more stripping down occurred, until I eventually gave up using my own project and just started a new one just to see about the collection class. I did it for the sake of being thorough, rather than actually thinking the lib had a bug in its debug implementation. But lo and behold, when I managed to make it reproducible and put it on SO, someone from Apple acknowledged that they could also see it, and they fixed it. Naturally if I'd gone direct to testing the lib I'd have saved a huge amount of time, but I guess that's the tradeoff from the most sensible heuristic: test your own code first, the bug is there.

yjftsjthsd-h 5y ago
Reminds me of: "It Is Never a Compiler Bug Until It Is" (https://r6.ca/blog/20200929T023701Z.html https://r6.ca/blog/20200929T023701Z.html , https://news.ycombinator.com/item?id=24636326 https://news.ycombinator.com/item?id=24636326). The bottom of the modern stack is really reliable, until it isn't;)
- Smoosh 5y ago
  Not just "the modern stack". I work mainframes and always felt the IBM-supplied environment (compilers, transaction processing systems, databases) was rock solid. Then one day I discovered APARs were a thing. https://www.ibm.com/support/pages/open-apars-ibm-products-available-web https://www.ibm.com/support/pages/open-apars-ibm-products-av...
cesarb 5y ago
> but I guess that's the tradeoff from the most sensible heuristic: test your own code first, the bug is there. Also known as "select is not broken" (see for instance https://blog.codinghorror.com/the-first-rule-of-programming-its-always-your-fault/ https://blog.codinghorror.com/the-first-rule-of-programming-...).
gh123man 5y ago
> nobody thinks that the Swift libs will be wrong as a first guess This is highly dependent on which version of Swift you started with! When Swift introduced the new substring API I hit a bug where certain UTF-8 character sequences caused an index out of bounds error internally. Unfortunately we learned this in production when an entire organization couldn't launch our app due to a string they were feeding through it. That is how your trust in the standard libs is forever broken. Library and compiler bugs were quite common in the Swift 1-3 days.
- jcelerier 5y ago
  Yeah, over the course of my allegedly short career (I'm 29) I've reported dozens of bugs against GCC, Clang, MSVC, binutils, Qt, SDL, glibc, PortAudio, macOS and other foundational stuff... I'm not saying I automatically assume "toolchain bug", but my cutoff for seriously pondering "is it a bug in $underlying_stuff" is around 30 minutes of "I really can't see where in my code things were done wrong" and so far this heuristic has consistently held...
  - deleted 5y ago
    [deleted]
  - resonious 5y ago
    Yup this kind of thing piles on and you start to understand why a lot of experienced programmers eventually adopt a "no libraries" policy. I've also found that depending on my employer, the open source codebases we use often don't actually hold up to the standard of quality that we hold our own code to. But we don't check it because all libraries are assumed to work.
twic 5y ago
Similar story with a bug in the IBM JDK's implementation of BigDecimal. Surely if anyone is going to get decimals right it's IBM! Took us a long time to stop looking at our code. (turns out that IBM do get decimals right if you're running on z/Architecture, where the code diverts to some hardware-accelerated fast path; just not on x86-64 machines used by paupers like my project)
dnautics 5y ago
I found a bug deep in the Erlang internals that apparently came about due to a missed case in an optimization that the team put in. It was not terribly hard to find though, on account that it was tripped by a unit test that was working just fine on an earlier version of BEAM, and thus was also easy to reduce to a minimal case by A/Bing against both versions.
alasdair_ 5y ago
My personal favorite ios bug that I found was in the middle of a finance exam. The ios calculator app gave completely different answers depending on whether or not you tilted the phone slightly during the calculation. It wasn’t just a visual thing - the calculation itself was changed by the toggling briefly to scientific mode and back again. I no longer fully trust calculators.

yjftsjthsd-h 5y ago

Reminds me of: "It Is Never a Compiler Bug Until It Is" (https://r6.ca/blog/20200929T023701Z.html https://r6.ca/blog/20200929T023701Z.html , https://news.ycombinator.com/item?id=24636326 https://news.ycombinator.com/item?id=24636326). The bottom of the modern stack is really reliable, until it isn't;)

Smoosh 5y ago
Not just "the modern stack". I work mainframes and always felt the IBM-supplied environment (compilers, transaction processing systems, databases) was rock solid. Then one day I discovered APARs were a thing. https://www.ibm.com/support/pages/open-apars-ibm-products-available-web https://www.ibm.com/support/pages/open-apars-ibm-products-av...

cesarb 5y ago

> but I guess that's the tradeoff from the most sensible heuristic: test your own code first, the bug is there. Also known as "select is not broken" (see for instance https://blog.codinghorror.com/the-first-rule-of-programming-its-always-your-fault/ https://blog.codinghorror.com/the-first-rule-of-programming-...).

gh123man 5y ago

> nobody thinks that the Swift libs will be wrong as a first guess This is highly dependent on which version of Swift you started with! When Swift introduced the new substring API I hit a bug where certain UTF-8 character sequences caused an index out of bounds error internally. Unfortunately we learned this in production when an entire organization couldn't launch our app due to a string they were feeding through it. That is how your trust in the standard libs is forever broken. Library and compiler bugs were quite common in the Swift 1-3 days.

jcelerier 5y ago
Yeah, over the course of my allegedly short career (I'm 29) I've reported dozens of bugs against GCC, Clang, MSVC, binutils, Qt, SDL, glibc, PortAudio, macOS and other foundational stuff... I'm not saying I automatically assume "toolchain bug", but my cutoff for seriously pondering "is it a bug in $underlying_stuff" is around 30 minutes of "I really can't see where in my code things were done wrong" and so far this heuristic has consistently held...
- deleted 5y ago
  [deleted]
- resonious 5y ago
  Yup this kind of thing piles on and you start to understand why a lot of experienced programmers eventually adopt a "no libraries" policy. I've also found that depending on my employer, the open source codebases we use often don't actually hold up to the standard of quality that we hold our own code to. But we don't check it because all libraries are assumed to work.

twic 5y ago

Similar story with a bug in the IBM JDK's implementation of BigDecimal. Surely if anyone is going to get decimals right it's IBM! Took us a long time to stop looking at our code. (turns out that IBM do get decimals right if you're running on z/Architecture, where the code diverts to some hardware-accelerated fast path; just not on x86-64 machines used by paupers like my project)

dnautics 5y ago

I found a bug deep in the Erlang internals that apparently came about due to a missed case in an optimization that the team put in. It was not terribly hard to find though, on account that it was tripped by a unit test that was working just fine on an earlier version of BEAM, and thus was also easy to reduce to a minimal case by A/Bing against both versions.

alasdair_ 5y ago

My personal favorite ios bug that I found was in the middle of a finance exam. The ios calculator app gave completely different answers depending on whether or not you tilted the phone slightly during the calculation. It wasn’t just a visual thing - the calculation itself was changed by the toggling briefly to scientific mode and back again. I no longer fully trust calculators.