3 ms·

Just amazed that ‘better testing’ isn’t one of the takeaways in the summary. Just amazed. Yea ‘write code carefully’ as if suggesting that’ll fix it is a rooki

by jarym 1y ago

Just amazed that ‘better testing’ isn’t one of the takeaways in the summary. Just amazed. Yea ‘write code carefully’ as if suggesting that’ll fix it is a rookie mistake. So so frustrating when developers treat user machines like their test bed!

fifilura 1y ago
Contrarian approach: $8000 is not a lot in this context. What did the CEO think of this? Most of the time it is just a very small speed bump in the overall finances of the company. Avoidable, unfortunate, but the cost of slowing down development progress e.g. 10% is much higher. But agree that senior gatekeepers should know by heart some places where review needs to be extra careful. Like security pitfalls, exponential fallback of error handling, and yeah, probably this.
- stevage 1y ago
  I'm sure it cost a lot more than $8000. That was only the direct visible cost to them. There were likely users hit with costs for the additional downloads, who never even knew what was the issue. Users working on a mobile hotspot who had to pay for extra data etc etc.
- rvz 1y ago
  Imagine if that was Meta that had over 1B users with their messenger desktop app update functionality that did just that. The loss would be in the hundreds of millions. > But agree that senior gatekeepers should know by heart some places where review needs to be extra careful. Like security pitfalls, exponential fallback of error handling, and yeah, probably this. The lesson here is much better use of automated tests (The app likely has no tests at all) and proper use of basic testing principles like TDD would prevent such junior-level embarrassing bugs creeping up in production paid software. That is the difference between a $100 problem vs a $200M problem. See the case of Knight Capital [0] who lost $460M, due to a horrific deploy. [0] https://www.henricodolfing.com/2019/06/project-failure-case-study-knight-capital.html https://www.henricodolfing.com/2019/06/project-failure-case-...
- latexr 1y ago
  > What did the CEO think of this? I doubt there’s a CEO. Despite the use of “we”, pretty sure this is one guy building the app. All the copyright notices and social media go back to one person.
  - deleted 1y ago
    [deleted]
Klaster_1 1y ago
How do you adjust your testing approach to catch cases like this? In my experience, timing related issues are hard to catch and can linger for years unnoticed.
- doix 1y ago
  I would mock/hook/monkey patch/whatever the functions to get the current time/elapsed time, simulate a period of time (a day/week/month/year/whatever), make sure the function to download the file is called the correct amount of times. That would probably catch this bug. Although, after such a fuck up, I would be tempted to make a pre-release check that tests the compiled binary, not any unit test or whatever. Use LD_PRELOAD to hook the system timing functions(a quick google shows that libfaketime[0] exists, but I've never used it), launch the real program and speed up time to make sure it doesn't try to download more than once. [0] https://github.com/wolfcw/libfaketime https://github.com/wolfcw/libfaketime
- 01HNNWZ0MV43FF 1y ago
  Similar to doix said, consider reading the time as IO and then rewrite the code in sans-IO style so you can inject the time. Then it's a unit test that looks too obvious to exist until you read the ticket mentioned in the comment above it No need for monkey patching or hooking or preload But before that you add a couple checkmarks to the manual pre-release test list: "1 hour soak test" and "check network transfer meters before and after, expect under 50 MB used in 1 hour (see bug #6969)" In Linux they're under /sys/class/net I think
- ikiris 1y ago
  I imagine they’d have to start testing first…
- coip 1y ago
  not to ~dupe my top-level comment, but ETag headers can be used to manage this from a ~caching perspective
hbsbsbsndk 1y ago
I worked on a product where there was basically no automated testing, just a huge product surface to click around with a bunch of options. Because of technical debt some of the options would trigger different code paths, but it was up to the developer to memorize all the code paths and test accordingly. After I shipped a bug the Director of Engineering told me I should "test better" (by clicking around the app). This was about 1 step away from "just don't write bugs" IMO.
- stevage 1y ago
  Yep, my first job was at a company like that. Huge Windows desktop app built in Delphi. No automated testing of any kind. No testing scripts either. Just a lot of clicking around.
  - cryptonym 1y ago
    My first job was exactly that, selling windows app in Delphi. I joined the new team working on .net windows apps and we had an army of people clicking on UI all day long. They maintained their "test plan" on a custom software where they could report failures. TBH, that was well done for what it was but really called for automation and lacked unit-testing.
    - 01HNNWZ0MV43FF 1y ago
      I remember a test plan in a spreadsheet where no test had an ID. I wish I could teach everything I learned the hard way at that job
    - HdS84 1y ago
      I am forced to use a custom kv store for my current t project. That pos has a custom dsl, which can only be imported through a swing ui, by clicking five buttons. Also, the ui is for 1024 screens, they are tiny in my 4k monitor
  - hbsbsbsndk 1y ago
    To be clear this is a pretty modern company, not 10 years ago. CI/CD absolutely was a common best practice.
stevage 1y ago
>Just amazed that ‘better testing’ isn’t one of the takeaways in the summary. I don't get the impression they did any testing at all.
jlarocco 1y ago
They were using a typed language, so testing was unnnecessary ;-)
- rvz 1y ago
  Quite shameful if it was a language based on Javascript with zero tests in 2025.
256_ 1y ago
I don't think the author is wrong for saying that certain kinds of code should be written carefully. I object to the implication that other code shouldn't. From TFA: "Write your auto-updater code very carefully. Actually, write any code that has the potential to generate costs carefully." So the focus is on code that "generate[s] costs". I think this is a common delusion programmers have; that some code is inherently unrelated to security (or cost), so they can get lazy with it. I see it like gun safety. You have to always treat a gun like it's loaded, not because it always is (although sometimes it may be loaded when you don't expect it), but because it teaches you to always be careful, so you don't absent-mindedly fall back into bad habits when you handle a loaded one. Telling people to write code carefully sounds simplistic but I believe for some people it's genuinely the right advice.

by jarym 1y ago

fifilura 1y ago

Contrarian approach: $8000 is not a lot in this context. What did the CEO think of this? Most of the time it is just a very small speed bump in the overall finances of the company. Avoidable, unfortunate, but the cost of slowing down development progress e.g. 10% is much higher. But agree that senior gatekeepers should know by heart some places where review needs to be extra careful. Like security pitfalls, exponential fallback of error handling, and yeah, probably this.

stevage 1y ago
I'm sure it cost a lot more than $8000. That was only the direct visible cost to them. There were likely users hit with costs for the additional downloads, who never even knew what was the issue. Users working on a mobile hotspot who had to pay for extra data etc etc.
rvz 1y ago
Imagine if that was Meta that had over 1B users with their messenger desktop app update functionality that did just that. The loss would be in the hundreds of millions. > But agree that senior gatekeepers should know by heart some places where review needs to be extra careful. Like security pitfalls, exponential fallback of error handling, and yeah, probably this. The lesson here is much better use of automated tests (The app likely has no tests at all) and proper use of basic testing principles like TDD would prevent such junior-level embarrassing bugs creeping up in production paid software. That is the difference between a $100 problem vs a $200M problem. See the case of Knight Capital [0] who lost $460M, due to a horrific deploy. [0] https://www.henricodolfing.com/2019/06/project-failure-case-study-knight-capital.html https://www.henricodolfing.com/2019/06/project-failure-case-...
latexr 1y ago
> What did the CEO think of this? I doubt there’s a CEO. Despite the use of “we”, pretty sure this is one guy building the app. All the copyright notices and social media go back to one person.
- deleted 1y ago
  [deleted]

Klaster_1 1y ago

How do you adjust your testing approach to catch cases like this? In my experience, timing related issues are hard to catch and can linger for years unnoticed.

doix 1y ago
I would mock/hook/monkey patch/whatever the functions to get the current time/elapsed time, simulate a period of time (a day/week/month/year/whatever), make sure the function to download the file is called the correct amount of times. That would probably catch this bug. Although, after such a fuck up, I would be tempted to make a pre-release check that tests the compiled binary, not any unit test or whatever. Use LD_PRELOAD to hook the system timing functions(a quick google shows that libfaketime[0] exists, but I've never used it), launch the real program and speed up time to make sure it doesn't try to download more than once. [0] https://github.com/wolfcw/libfaketime https://github.com/wolfcw/libfaketime
01HNNWZ0MV43FF 1y ago
Similar to doix said, consider reading the time as IO and then rewrite the code in sans-IO style so you can inject the time. Then it's a unit test that looks too obvious to exist until you read the ticket mentioned in the comment above it No need for monkey patching or hooking or preload But before that you add a couple checkmarks to the manual pre-release test list: "1 hour soak test" and "check network transfer meters before and after, expect under 50 MB used in 1 hour (see bug #6969)" In Linux they're under /sys/class/net I think
ikiris 1y ago
I imagine they’d have to start testing first…
coip 1y ago
not to ~dupe my top-level comment, but ETag headers can be used to manage this from a ~caching perspective

hbsbsbsndk 1y ago

I worked on a product where there was basically no automated testing, just a huge product surface to click around with a bunch of options. Because of technical debt some of the options would trigger different code paths, but it was up to the developer to memorize all the code paths and test accordingly. After I shipped a bug the Director of Engineering told me I should "test better" (by clicking around the app). This was about 1 step away from "just don't write bugs" IMO.

stevage 1y ago
Yep, my first job was at a company like that. Huge Windows desktop app built in Delphi. No automated testing of any kind. No testing scripts either. Just a lot of clicking around.
- cryptonym 1y ago
  My first job was exactly that, selling windows app in Delphi. I joined the new team working on .net windows apps and we had an army of people clicking on UI all day long. They maintained their "test plan" on a custom software where they could report failures. TBH, that was well done for what it was but really called for automation and lacked unit-testing.
  - 01HNNWZ0MV43FF 1y ago
    I remember a test plan in a spreadsheet where no test had an ID. I wish I could teach everything I learned the hard way at that job
  - HdS84 1y ago
    I am forced to use a custom kv store for my current t project. That pos has a custom dsl, which can only be imported through a swing ui, by clicking five buttons. Also, the ui is for 1024 screens, they are tiny in my 4k monitor
- hbsbsbsndk 1y ago
  To be clear this is a pretty modern company, not 10 years ago. CI/CD absolutely was a common best practice.

stevage 1y ago

>Just amazed that ‘better testing’ isn’t one of the takeaways in the summary. I don't get the impression they did any testing at all.

jlarocco 1y ago

They were using a typed language, so testing was unnnecessary ;-)

rvz 1y ago
Quite shameful if it was a language based on Javascript with zero tests in 2025.

256_ 1y ago

I don't think the author is wrong for saying that certain kinds of code should be written carefully. I object to the implication that other code shouldn't. From TFA: "Write your auto-updater code very carefully. Actually, write any code that has the potential to generate costs carefully." So the focus is on code that "generate[s] costs". I think this is a common delusion programmers have; that some code is inherently unrelated to security (or cost), so they can get lazy with it. I see it like gun safety. You have to always treat a gun like it's loaded, not because it always is (although sometimes it may be loaded when you don't expect it), but because it teaches you to always be careful, so you don't absent-mindedly fall back into bad habits when you handle a loaded one. Telling people to write code carefully sounds simplistic but I believe for some people it's genuinely the right advice.