2 ms·
> but being wrong under load sounds like a horrible internal architecture problem. Tons of software can be "wrong under load" - things like race conditions and
by hn_throwaway_99 4y ago
> but being wrong under load sounds like a horrible internal architecture problem.
Tons of software can be "wrong under load" - things like race conditions and memory leaks are common problems, and they don't necessarily point to a huge architectural defect. E.g. Cloudflare had a bug years ago that caused highly sensitive data to leak, but only for a teeny (relatively) portion of their site. Similar issue happened to GitHub: https://github.blog/2021-03-18-how-we-found-and-fixed-a-rare-race-condition-in-our-session-handling/ https://github.blog/2021-03-18-how-we-found-and-fixed-a-rare...
- lupire 4y ago"huge" is not the same as "horrible". One racy boi can take out an airplane.
- hn_throwaway_99 4y agoBut calling something an "internal architectural problem" implies things are fundamentally designed wrong, and the code needs a rewrite. My point was the architecture can definitely be fine, but you can still have bugs (even catastrophic ones) because someone left out a `synchronized` keyword somewhere.
- thrashh 4y agoIt could still be For example, one way to use SQL is to escape your parameters and then concat them straight into your SQL string. However, if someone leaves out an ‘escape’ somewhere, you have a major security incident The other way is to pass parameters separately as data to your SQL driver. It completely negates the problem If you’ve chosen the first way in your project already, you’ve committed to a major internal architectural problem. In the same vein, maybe if your code requires sprinkling ‘synchronized’ everywhere, you did it wrong But that said, we don’t know what actually is the problem since it’s just a marketing statement saying that high load was the cause and that doesn’t tell us anything
- Ozzie_osman 4y agoTrue, but software for domains where people can die is very different than Cloudflare. The scale is much higher in the latter, and it needs to be less "bulletproof". If you're writing software for giant metal flying cylinder carrying hundreds of people, we can't just brush mistakes off the same way we might with web-based software (and yes, privacy is important, but it's not life or death).
- hn_throwaway_99 4y agoI'm not "brushing mistakes off", at all. The process that led to the bugs needs a full review and root-cause corrective action, especially enhanced testing scenarios. I'm just pointing out that software "being wrong under load" is not some sort of crazy, unusual problem that points to poor software architecture. These bugs can easily be caused by a single line of code that wasn't synchronized correctly.
- pas 4y ago... ... that's the problem. why isn't it running isolated?
- AdrianB1 4y agoIn most industries that are not IT, testing the IT systems very well is needed, but rarely achieved. In my area (FDA regulated) performance testing is almost never done correctly and (for sure) never completely, but as long as it works and the auditors never look at these details nobody from management is willing to know about it, because fixing it costs money. For many years the people signing off for all the software were intentionally not business people with zero IT knowledge, so they could not be accused of knowingly signing off low quality code. That saved tens or hundreds of millions of dollars in the past 20 years alone.
- oarsinsync 4y ago> In most industries, testing the IT systems very well is needed, but rarely achieved. FTFY. IT shops on the whole aren’t any better here.
- jki275 4y agoThat's still an architecture defect. Software's accuracy being affected by load is completely unacceptable.
- hn_throwaway_99 4y agoPlease let us know where these software magicians are that write perfect, bug-free code.
- jki275 4y agoWho said anything about that? You're arguing against a straw man nobody has set up but yourself.. There's no such thing as perfect software, but there is definitely software that doesn't make up a false value if it can't work.
- forgottenlogin9 4y agoAnd if there is a bug in the logic that determines if it can or can’t work?
- puffoflogic 4y agoFor what it's worth, I had the same reaction upon reading this paragraph. It seems there are HN commenters who understand software design and those who do not. To be fair to the other commenters, it's conceivable that the practical difference between the current unsound architecture and a sound replacement might come down to to some small defect. But an attempt to fix the problem by fixing just the defect, even some kind of RCA process on it, will fail if the architecture itself is not fixed, and that requires a change in attitudes and understanding by the devs responsible for the system, not just a software change. That is where the real flaw lies. But some people don't believe that it's possible for the flaw to be in these places, outside the code. All the RCAs in the world can't help them.