5 ms·

Exactly. Users are subsidizing the software provider with CPU cycles and employee time. Assume it costs $800 for an engineer-day. Assume your software has 10,0

Exactly. Users are subsidizing the software provider with CPU cycles and employee time. Assume it costs $800 for an engineer-day. Assume your software has 10,000 daily users and that the wasted time cost is 20 seconds (assume this is actual wasted time when an employee is actively waiting and not completing some other task). Assume the employees using the software earn on average 1/8 of what the engineer makes. It would take less than 4 days to make up for the employee's time. That $800 would save about $80,000 per year. Obviously, this is a contrived example, but I think it's a conservative one. I'm overpaying the engineer (on average) and probably under-estimating time wasted and user cost.

knorker 4y ago
If humans wait, yes. If you can just buy another server: no. I 100% agree on saving human time. Human time is expensive. CPU time is absolutely not.
- pjvsvsrtrxc 4y ago
  Servers are expensive, too. Humans waiting on servers to process something is even more expensive. No software runs in a vacuum; someone is waiting on it somewhere. Adding more servers doesn't generally make things faster (latency). It only raises capacity (bandwidth). It does, however, generally cost quite a bit on development. Just about the only thing worse than designing a complex system is designing a complex distributed system.
  - knorker 4y ago
    I'm of course aware of all this. If you don't want to take the advise of running the numbers that's up to you. E.g. if end user latency is 10ms (and it's not voip or VR or something) then that's fast enough. Doesn't matter if it's optimizable to 10 us. If this is code running on your million CPU farm 24/7, then yeah. But always run the numbers first. Like I said, the vast majority of code optimization opportunities are not worth taking. Some are, but only after running the numbers. On the flip side optimizing for human time is almost always worth it, be it end users or other developers. But run the numbers for your company. How much does a CPU core cost per hour of it's lifetime? Your developers cost maybe $100, but maybe $1000 in opportunity cost. Depending on what you do a server may cost you as much as one day of developer opportunity time. And then you have the server for years. (Subject to electricity) Latency and throughput may be better solved by adding machines.
    - deleted 4y ago
      [deleted]
    - inkblotuniverse 4y ago
      You should, however, not pessimize. People make cargo-cult architecture choices that bloat their codebase, make itnless readable, and make it 100x slower.
      - knorker 4y ago
        Maybe. Using actual numbers vetted by actual expenses in an actual company, if you can save 100 CPU cores by spending 3h a year keeping it optimized, then it is NOT worth it. It is cheaper to burn CPU, even if you could spend one day a year making it max out one CPU core instead of 100. It can be better for the business to cargo cult. Not always. But you should remember that the point of the code is to solve a problem, at a low cost. Reducing complexity reduces engineer cost in the future and may also make things faster. Put it this way: Would you hire someone at $300k doing nothing but optimizing your pipeline so that it takes one machine instead of one rack, or would you spend half that money (TCO over its lifetime) just buying a rack of machines? If you wouldn't hire them to do it, then you shouldn't spend current engineers time doing it.
        inkblotuniverse 4y ago
        I wasn't talking about optimization! I was talking about non-pessimization, which includes not prematurely abstracting/generalizing your code. I've seen people making poor decisions at the outset, and having code philosophies that actively make new code 100x slower without any clear gain. Over-generalization, 100 classes and subclasses, everything is an overriden virtual method, dogmatic TDD (luckily, nobody followed that.) The dogma was to make things more complicated and illegible, 'because SOLID'.
    - loup-vaillant 4y ago
      > Like I said, the vast majority of code optimization opportunities are not worth taking. Some are, but only after running the numbers. Casey Muratori said it best: there are 3 philosophies of optimisation. You're talking about the first: actual optimisation where you measure and decide what to tackle. It's rarely used, and with good reason. The second philosophy however is very different: it's non-pessimisation. That is, avoid having the CPU do useless work all the time. That one should be applied in a fairly systematic basis, and it's not. To apply it in practice you need to have an idea of how much time your algorithm requires. Count how many bytes are processed, how many operations are made… this should give a nice upper bound on performance. If you're within an order of magnitude of this theoretical maximum, you're probably good. Otherwise you probably missed something. The third philosophy is fake optimisation: heuristics misapplied out of context. This one should never be used, but is more frequent than we care to admit.

knorker 4y ago

If humans wait, yes. If you can just buy another server: no. I 100% agree on saving human time. Human time is expensive. CPU time is absolutely not.

pjvsvsrtrxc 4y ago
Servers are expensive, too. Humans waiting on servers to process something is even more expensive. No software runs in a vacuum; someone is waiting on it somewhere. Adding more servers doesn't generally make things faster (latency). It only raises capacity (bandwidth). It does, however, generally cost quite a bit on development. Just about the only thing worse than designing a complex system is designing a complex distributed system.
- knorker 4y ago
  I'm of course aware of all this. If you don't want to take the advise of running the numbers that's up to you. E.g. if end user latency is 10ms (and it's not voip or VR or something) then that's fast enough. Doesn't matter if it's optimizable to 10 us. If this is code running on your million CPU farm 24/7, then yeah. But always run the numbers first. Like I said, the vast majority of code optimization opportunities are not worth taking. Some are, but only after running the numbers. On the flip side optimizing for human time is almost always worth it, be it end users or other developers. But run the numbers for your company. How much does a CPU core cost per hour of it's lifetime? Your developers cost maybe $100, but maybe $1000 in opportunity cost. Depending on what you do a server may cost you as much as one day of developer opportunity time. And then you have the server for years. (Subject to electricity) Latency and throughput may be better solved by adding machines.
  - deleted 4y ago
    [deleted]
  - inkblotuniverse 4y ago
    You should, however, not pessimize. People make cargo-cult architecture choices that bloat their codebase, make itnless readable, and make it 100x slower.
    - knorker 4y ago
      Maybe. Using actual numbers vetted by actual expenses in an actual company, if you can save 100 CPU cores by spending 3h a year keeping it optimized, then it is NOT worth it. It is cheaper to burn CPU, even if you could spend one day a year making it max out one CPU core instead of 100. It can be better for the business to cargo cult. Not always. But you should remember that the point of the code is to solve a problem, at a low cost. Reducing complexity reduces engineer cost in the future and may also make things faster. Put it this way: Would you hire someone at $300k doing nothing but optimizing your pipeline so that it takes one machine instead of one rack, or would you spend half that money (TCO over its lifetime) just buying a rack of machines? If you wouldn't hire them to do it, then you shouldn't spend current engineers time doing it.
      - inkblotuniverse 4y ago
        I wasn't talking about optimization! I was talking about non-pessimization, which includes not prematurely abstracting/generalizing your code. I've seen people making poor decisions at the outset, and having code philosophies that actively make new code 100x slower without any clear gain. Over-generalization, 100 classes and subclasses, everything is an overriden virtual method, dogmatic TDD (luckily, nobody followed that.) The dogma was to make things more complicated and illegible, 'because SOLID'.
  - loup-vaillant 4y ago
    > Like I said, the vast majority of code optimization opportunities are not worth taking. Some are, but only after running the numbers. Casey Muratori said it best: there are 3 philosophies of optimisation. You're talking about the first: actual optimisation where you measure and decide what to tackle. It's rarely used, and with good reason. The second philosophy however is very different: it's non-pessimisation. That is, avoid having the CPU do useless work all the time. That one should be applied in a fairly systematic basis, and it's not. To apply it in practice you need to have an idea of how much time your algorithm requires. Count how many bytes are processed, how many operations are made… this should give a nice upper bound on performance. If you're within an order of magnitude of this theoretical maximum, you're probably good. Otherwise you probably missed something. The third philosophy is fake optimisation: heuristics misapplied out of context. This one should never be used, but is more frequent than we care to admit.