9 ms·

I must have heard this story and forgot it, because I used this argument on my team when I ran the group at Blizzard that did installing and downloading and pat

I must have heard this story and forgot it, because I used this argument on my team when I ran the group at Blizzard that did installing and downloading and patching. “We have 10 million people downloading and installing this patch, so every minute extra we take is another fraction of a human life we’re spending”. Sure, overly dramatic, and corny, but helped drive improvements. The other more important metric I pushed was “speed of light”. When installing from a DVD (yeah, olden times), the “speed of light” there was the rotational speed of the disc and so we should install as close to that speed as possible. Keep improving speed of operations until you butt up against whatever physical limits exist. Time is precious, you don’t get more of it.

hobs 3y ago
That last part is important. I have worked with many engineers who I would even classify as hard working, but spent little to no time understanding the hardware they were running on and the possibilities that it provided them. I have heard "that's slow" or "that's good" too many times in performance talks that have completely ignored the underlying machine and what was possible.
- TillE 3y ago
  Learning about how the CPU cache works is probably the most useful thing you can do if you write anything that's not I/O limited. There are definitely a ton of experienced programmers who don't quite understand how often the CPU is just waiting around for data from RAM.
  - arrowsmith 3y ago
    Okay, you've made me want to learn about it. Where do I start? What concepts do I need to understand? Any reading recommendations?
    - Mockapapella 3y ago
      Haven't read through it, but I suspect this would be a good place to start: https://cpu.land/ https://cpu.land/ HN Discussion: https://news.ycombinator.com/item?id=36823605 https://news.ycombinator.com/item?id=36823605
    - hayley-patton 3y ago
      Drepper's "What every programmer should know about memory", though you mightn't find it all interesting. https://gwern.net/doc/cs/hardware/2007-drepper.pdf https://gwern.net/doc/cs/hardware/2007-drepper.pdf
  - mcculley 3y ago
    It is a shame that there are not better monitoring tools that surface this. When I use Activity Monitor on macOS, it would be useful to see how much of “% CPU” is just waiting on memory. I know I can drill down with various profilers, but having it more accessible is way overdue.
    - saagarjha 3y ago
      Instruments?
      - sakras 3y ago
        Instruments is not nearly good enough for any serious performance work. Instruments only tells me what percent of time is spent in which part of the code. This is fine for a first pass, but it doesn’t tell me _why_ something is slow. I really need a V-Tune-like profiler on macOS.
        saagarjha 3y ago
        I’ve used it professionally and generally been happy with it. What are you missing from it?
        sakras 3y ago
        I’ve tried to use it professionally, but always end up switching to my x86 desktop to profile my code, just so I can use V-Tune. It’s missing any kind of deeper statistics such as memory bandwidth, cache misses, branch mispredictions, etc. I think fundamentally Apple is geared towards application development, whereas I’m working on more HPC-like things.
        saagarjha 3y ago
        Have you tried using the performance counters? They've been useful in my experience, although I don't touch them often. Instruments is definitely not geared towards this since most application developers rarely need to do profiling at this level, but it has some level of this built in when you need it.
  - saagarjha 3y ago
    It’s only useful once you understand how algorithmic complexity works, and how to profile your code, and how you language runtime does things. Before that your CPU cache is largely opaque and trying to peer into it is probably counterproductive.
tazjin 3y ago
Back in the day I was hacking on WoW-related stuff like server emulators, and it was always very noticeable how much care Blizzard put into this kind of stuff. The (iirc) torrent-based patch distribution for WoW etc. was really well done. Kudos, especially in such a high-pressure industry!
breakingrules 3y ago
[dead]
opportune 3y ago
I wish more engineers thought this way. As someone who works in infrastructure it’s the story I tell myself to justify/rationalize my place in the world. When I ship big infrastructure performance improvements it’s not about the speed or money saved per se, it’s less CO2 in the atmosphere and more human life (amortized over millions of people) spent on something other than waiting for a computer to respond. We aren’t doctors saving individuals’ lives but what we can do is give people fractions of their lives back. Some software is used by hundreds of millions or billions of people, so small changes there can save many “lives” worth of time.
llimos 3y ago
> Time is precious, you don’t get more of it. In this particular example, the time saved on the download will go towards the noble cause of ... playing video games? Is that so much better use of time than the wait for it to download?
- niels_bom 3y ago
  That’s assuming people play more when the download is faster. And to answer your question: for everybody involved it’s better yes.
whartung 3y ago
Having lived through the early horrors of WoW patches and updates, I have nothing but praise for how WoW works today in terms of updates and distribution. My favorite feature is how it supports incremental loading. WoW is a huge game, but you can start playing with a fraction of the assets. It will play in a downgraded way with place holders and lower quality assets as well as skipping entire areas completely. You can reinstall from scratch and be up and playing in minutes. It’s one of the hidden joys of the platform that players mostly take for granted, but I appreciate the no doubt legion effort involved to pull this off, to change the wheels on a moving train, and to deliver just uncountable amount of data with little drama and great performance. So kudos to you for whatever your contribution was to making such a core facility to the system so painless for the end user.

hobs 3y ago

That last part is important. I have worked with many engineers who I would even classify as hard working, but spent little to no time understanding the hardware they were running on and the possibilities that it provided them. I have heard "that's slow" or "that's good" too many times in performance talks that have completely ignored the underlying machine and what was possible.

TillE 3y ago
Learning about how the CPU cache works is probably the most useful thing you can do if you write anything that's not I/O limited. There are definitely a ton of experienced programmers who don't quite understand how often the CPU is just waiting around for data from RAM.
- arrowsmith 3y ago
  Okay, you've made me want to learn about it. Where do I start? What concepts do I need to understand? Any reading recommendations?
  - Mockapapella 3y ago
    Haven't read through it, but I suspect this would be a good place to start: https://cpu.land/ https://cpu.land/ HN Discussion: https://news.ycombinator.com/item?id=36823605 https://news.ycombinator.com/item?id=36823605
  - hayley-patton 3y ago
    Drepper's "What every programmer should know about memory", though you mightn't find it all interesting. https://gwern.net/doc/cs/hardware/2007-drepper.pdf https://gwern.net/doc/cs/hardware/2007-drepper.pdf
- mcculley 3y ago
  It is a shame that there are not better monitoring tools that surface this. When I use Activity Monitor on macOS, it would be useful to see how much of “% CPU” is just waiting on memory. I know I can drill down with various profilers, but having it more accessible is way overdue.
  - saagarjha 3y ago
    Instruments?
    - sakras 3y ago
      Instruments is not nearly good enough for any serious performance work. Instruments only tells me what percent of time is spent in which part of the code. This is fine for a first pass, but it doesn’t tell me _why_ something is slow. I really need a V-Tune-like profiler on macOS.
      - saagarjha 3y ago
        I’ve used it professionally and generally been happy with it. What are you missing from it?
        sakras 3y ago
        I’ve tried to use it professionally, but always end up switching to my x86 desktop to profile my code, just so I can use V-Tune. It’s missing any kind of deeper statistics such as memory bandwidth, cache misses, branch mispredictions, etc. I think fundamentally Apple is geared towards application development, whereas I’m working on more HPC-like things.
        saagarjha 3y ago
        Have you tried using the performance counters? They've been useful in my experience, although I don't touch them often. Instruments is definitely not geared towards this since most application developers rarely need to do profiling at this level, but it has some level of this built in when you need it.
- saagarjha 3y ago
  It’s only useful once you understand how algorithmic complexity works, and how to profile your code, and how you language runtime does things. Before that your CPU cache is largely opaque and trying to peer into it is probably counterproductive.

tazjin 3y ago

Back in the day I was hacking on WoW-related stuff like server emulators, and it was always very noticeable how much care Blizzard put into this kind of stuff. The (iirc) torrent-based patch distribution for WoW etc. was really well done. Kudos, especially in such a high-pressure industry!

breakingrules 3y ago

[dead]

opportune 3y ago

I wish more engineers thought this way. As someone who works in infrastructure it’s the story I tell myself to justify/rationalize my place in the world. When I ship big infrastructure performance improvements it’s not about the speed or money saved per se, it’s less CO2 in the atmosphere and more human life (amortized over millions of people) spent on something other than waiting for a computer to respond. We aren’t doctors saving individuals’ lives but what we can do is give people fractions of their lives back. Some software is used by hundreds of millions or billions of people, so small changes there can save many “lives” worth of time.

llimos 3y ago

> Time is precious, you don’t get more of it. In this particular example, the time saved on the download will go towards the noble cause of ... playing video games? Is that so much better use of time than the wait for it to download?

niels_bom 3y ago
That’s assuming people play more when the download is faster. And to answer your question: for everybody involved it’s better yes.

whartung 3y ago

Having lived through the early horrors of WoW patches and updates, I have nothing but praise for how WoW works today in terms of updates and distribution. My favorite feature is how it supports incremental loading. WoW is a huge game, but you can start playing with a fraction of the assets. It will play in a downgraded way with place holders and lower quality assets as well as skipping entire areas completely. You can reinstall from scratch and be up and playing in minutes. It’s one of the hidden joys of the platform that players mostly take for granted, but I appreciate the no doubt legion effort involved to pull this off, to change the wheels on a moving train, and to deliver just uncountable amount of data with little drama and great performance. So kudos to you for whatever your contribution was to making such a core facility to the system so painless for the end user.