4 ms·

It doesn’t need to be “As fast as possible” it just needs to be faster than the I/O. For sure, congrats that it is X times faster but the writer doesn’t really

It doesn’t need to be “As fast as possible” it just needs to be faster than the I/O. For sure, congrats that it is X times faster but the writer doesn’t really describe performance in terms of real world constraints.

jrockway 4y ago
I/O is fast, and efficiency matters for every workload. The sooner your CPU is done running code, the sooner it can go into an energy saving state. Energy saving means a longer battery life on portable computers, and less cost in the server case. Remember that people are running algorithms like this "in the cloud" against millions of concurrent requests. The 40Gbps network card can keep the CPU busy, and users are waiting for you to be done. Writing efficient code to make 1 computer do the work of 2 is just operational cost savings 101. When I was at Google there was a "rules of thumb" page that described how much of your time X performance was worth. I always looked at that before micro-optimizing, but also always came out ahead. I remember a coworker and I redesigning one of our subsystems late on some Friday evening; the rules of thumb said that the performance improvement we predicted, at our scale, was worth a month of SWE time. We did it in 2 hours. So we came out ahead, and our system worked better. (I joked that my colleague and I would be taking 2 extra weeks of vacation.) TL;DR performance matters everywhere. The one user using an interactive tool on their workstation will appreciate their day not being wasted by random pauses. The person out and about on their phone will appreciate the additional battery life. And your company's bean counters will be quite happy to hear that your cloud bill or data center expense forecast for next year is down. Finally, it's fun! Truly a win/win. Make it fast!
- evancox100 4y ago
  What was the rule of thumb?
  - jrockway 4y ago
    This was based on yearly RAM cost. I worked on a log analysis system. In the beginning, it ran on one computer. That one program would read logs, generate fleet-wide analytics, and serve those out of RAM. Eventually we wanted to run on more than one computer, for both scaling and reliability reasons. The design of the system allowed us to basically do the same serving with many replicas, so for a long time we just ran a fleet of replicas that still had all the capabilities of the monolith. The change we made was to move the aggregation stage to a new dedicated program, that we only ran 3 copies of worldwide. (A man with 2 replicas never knows which one is broken, they say.) This meant that the mappers became significantly lighter RAM-wise, they just existed to use as much CPU time as they could, and then we had 3 beefy reducer replicas to aggregate and serve data. This was a very easy change; we just made a new main.go and an RPC to send the data to aggregate. The system was designed internally to be logically isolated across that boundary, so we just stuck in the RPC and then the other side of the boundary could be another data center. In the end, I think we saved a few terabytes of RAM-years. Not a big deal, but it was something.
adgjlsfhk1 4y ago
Not really. If you're 4x faster than IO, the CPU can do something else while waiting for IO (e.g. run another thread).

jrockway 4y ago

I/O is fast, and efficiency matters for every workload. The sooner your CPU is done running code, the sooner it can go into an energy saving state. Energy saving means a longer battery life on portable computers, and less cost in the server case. Remember that people are running algorithms like this "in the cloud" against millions of concurrent requests. The 40Gbps network card can keep the CPU busy, and users are waiting for you to be done. Writing efficient code to make 1 computer do the work of 2 is just operational cost savings 101. When I was at Google there was a "rules of thumb" page that described how much of your time X performance was worth. I always looked at that before micro-optimizing, but also always came out ahead. I remember a coworker and I redesigning one of our subsystems late on some Friday evening; the rules of thumb said that the performance improvement we predicted, at our scale, was worth a month of SWE time. We did it in 2 hours. So we came out ahead, and our system worked better. (I joked that my colleague and I would be taking 2 extra weeks of vacation.) TL;DR performance matters everywhere. The one user using an interactive tool on their workstation will appreciate their day not being wasted by random pauses. The person out and about on their phone will appreciate the additional battery life. And your company's bean counters will be quite happy to hear that your cloud bill or data center expense forecast for next year is down. Finally, it's fun! Truly a win/win. Make it fast!

evancox100 4y ago
What was the rule of thumb?
- jrockway 4y ago
  This was based on yearly RAM cost. I worked on a log analysis system. In the beginning, it ran on one computer. That one program would read logs, generate fleet-wide analytics, and serve those out of RAM. Eventually we wanted to run on more than one computer, for both scaling and reliability reasons. The design of the system allowed us to basically do the same serving with many replicas, so for a long time we just ran a fleet of replicas that still had all the capabilities of the monolith. The change we made was to move the aggregation stage to a new dedicated program, that we only ran 3 copies of worldwide. (A man with 2 replicas never knows which one is broken, they say.) This meant that the mappers became significantly lighter RAM-wise, they just existed to use as much CPU time as they could, and then we had 3 beefy reducer replicas to aggregate and serve data. This was a very easy change; we just made a new main.go and an RPC to send the data to aggregate. The system was designed internally to be logically isolated across that boundary, so we just stuck in the RPC and then the other side of the boundary could be another data center. In the end, I think we saved a few terabytes of RAM-years. Not a big deal, but it was something.

adgjlsfhk1 4y ago

Not really. If you're 4x faster than IO, the CPU can do something else while waiting for IO (e.g. run another thread).