3 ms·

In your example, it sounds like you mean throughput-optimized. According to the original post, brewing hot water for 10 cups would introduce additional latency.

by fiter 7y ago

dragontamer 7y ago
> In your example, it sounds like you mean throughput-optimized. According to the original post, brewing hot water for 10 cups would introduce additional latency. Nope. CPUs are latency-optimized. The "1st cup of coffee" always takes the same amount of time in CPU-land. The 2nd-cup of coffee was speculatively made, but never "slowed down the first cup of coffee". ---------- A throughput optimized machine, like GPUs (and strangely enough: hard drives), are willing to slow down the 1st-cup of coffee for better overall throughput. Hard drives are interesting: if you have the following "reads": #1: Read location 1 #2: Read location 100 #3: Read location 50 The hard drive will re-arrange the reads into: Read 1, Read 50, Read 100, because the hard-drive head will reach location50 before location100. Remember, hard drives are physically moving their arms to each physical location. This means that Read100 is "slowed down", its latency got significantly worse. But the three reads all together all completed at the same time.
- fiter 7y ago
  > The "1st cup of coffee" always takes the same amount of time in CPU-land. The 2nd-cup of coffee was speculatively made, but never "slowed down the first cup of coffee". Just to be clear, then: the analogy from the original post doesn't apply.
  - dragontamer 7y ago
    > Just to be clear, then: the analogy from the original post doesn't apply. The analogy from the original post applies to the cases the original post discusses. The original "coffee latency" blogpost innately applies to a 1980s style computer: a simple in-order machine. Its truly correct for that model of simple computing. I've added in complications: pipelining, superscalar, and speculative execution, which were inventions deployed in the early 90s and 00s to CPUs. So things work differently on modern machines, because modern machines have many, many more features than the "original" computer designs. The original "cups of coffee" are a good way to start thinking about latency vs bandwidth problem. I really like the analogy. But it would take a LOT more writing before I really cover everything going on in modern CPUs.
    - fiter 7y ago
      Your original post was missing explanation because you referenced the original analogy without addressing how the original analogy no longer applied to the scenario you were discussing. For what it's worth, in all my replies I have not been confused about the behavior of a CPU, but only about how you are trying to use the analogy to fit your exposition.
majormajor 7y ago
Nah, this would be brewing 1 cup at a time 10 times in parallel, an example not considered in the original post. To extend the original post, this would be trading capital - to buy more equipment - to improve both latency and throughput.

In your example, it sounds like you mean throughput-optimized. According to the original post, brewing hot water for 10 cups would introduce additional latency.

by fiter 7y ago

In your example, it sounds like you mean throughput-optimized. According to the original post, brewing hot water for 10 cups would introduce additional latency.

dragontamer 7y ago

> In your example, it sounds like you mean throughput-optimized. According to the original post, brewing hot water for 10 cups would introduce additional latency. Nope. CPUs are latency-optimized. The "1st cup of coffee" always takes the same amount of time in CPU-land. The 2nd-cup of coffee was speculatively made, but never "slowed down the first cup of coffee". ---------- A throughput optimized machine, like GPUs (and strangely enough: hard drives), are willing to slow down the 1st-cup of coffee for better overall throughput. Hard drives are interesting: if you have the following "reads": #1: Read location 1 #2: Read location 100 #3: Read location 50 The hard drive will re-arrange the reads into: Read 1, Read 50, Read 100, because the hard-drive head will reach location50 before location100. Remember, hard drives are physically moving their arms to each physical location. This means that Read100 is "slowed down", its latency got significantly worse. But the three reads all together all completed at the same time.

fiter 7y ago
> The "1st cup of coffee" always takes the same amount of time in CPU-land. The 2nd-cup of coffee was speculatively made, but never "slowed down the first cup of coffee". Just to be clear, then: the analogy from the original post doesn't apply.
- dragontamer 7y ago
  > Just to be clear, then: the analogy from the original post doesn't apply. The analogy from the original post applies to the cases the original post discusses. The original "coffee latency" blogpost innately applies to a 1980s style computer: a simple in-order machine. Its truly correct for that model of simple computing. I've added in complications: pipelining, superscalar, and speculative execution, which were inventions deployed in the early 90s and 00s to CPUs. So things work differently on modern machines, because modern machines have many, many more features than the "original" computer designs. The original "cups of coffee" are a good way to start thinking about latency vs bandwidth problem. I really like the analogy. But it would take a LOT more writing before I really cover everything going on in modern CPUs.
  - fiter 7y ago
    Your original post was missing explanation because you referenced the original analogy without addressing how the original analogy no longer applied to the scenario you were discussing. For what it's worth, in all my replies I have not been confused about the behavior of a CPU, but only about how you are trying to use the analogy to fit your exposition.

majormajor 7y ago

Nah, this would be brewing 1 cup at a time 10 times in parallel, an example not considered in the original post. To extend the original post, this would be trading capital - to buy more equipment - to improve both latency and throughput.