4 ms·

I'm not a low level developer (whatever that means today). I've written a very minimal amount of production C. Everything else has been higher level languages.

I'm not a low level developer (whatever that means today). I've written a very minimal amount of production C. Everything else has been higher level languages. So I apologize in advance for my ignorance. Isn't the problem really memory bandwidth? Aren't most CPU cycles "wasted" on speculative execution while waiting on fetches from memory slower than CPU caches? Wouldn't a hardware model where all of the RAM (SRAM?) was on the CPU die (fabrication and DRAM clock speed aside) solve this problem and make everything an order of magnitude faster? What am I missing? Is this really the programmers fault? The CPU literally spends most of its time waiting on my instructions, not choking on them, so why is this my fault for being lazy?

baybal2 8y ago
>What am I missing? You are missing money... SRAM and wider data paths are expensive. You are totally right about that taking more logic into synchronous domain domain is an almost universal solution to getting more performance. The faster the chips become, the more IO gap becomes. Experienced MCU programmers know quite a number of algorithms that may run faster on "dumb" 100mhz MCUs than on a 20-cored Xeon, and are proud to show them off. Even when some of MCU cores may take like 6 cycles to do an arithmetic op, their strong side is that it will be taking them these 6 cycles to do that all and every time consistently, and that their memory runs synchronously. Say, in a 1000000 cycle loop, where every operation results in cache miss, Xeon will spend more time on a loop than a wimpy 100Mhz MCU that simply don't take any penalty on unpredictable, heavy on synchronous ops cope. Apple A11 is so fast for an ARM just because they choose the simplest, the surest, most direct, but also the most expensive way to scale performance: tons of SRAM and prefetch logic everywhere to reduce IO gap as much as possible.
marcosdumay 8y ago
Memory bandwidth is a problem. A large one. But saying that "most" CPU cycles are wasted to it is a huge overrate. I have read estimates that go as high as 1/3 of the CPU lost to it (sorry, no reference here), but even those were not usual.
paulmd 8y ago
20 MB of RAM should be enough for anybody! /s I was reading a thing a while back on here or Reddit where someone asked about an OS that ran entirely in cache, after all in principle a modern processor has more L3 than an original Unix minicomputer. Since many of the memory subsystems operate on the assumption that changes should be propagated to memory that you would still need to have RAM there even if you weren't really "using" it. There was a suggestion that since the bringup code begins entirely on-processor (before initializing RAM) that you could hack the Management Engine and keep running in that mode forever.

baybal2 8y ago

>What am I missing? You are missing money... SRAM and wider data paths are expensive. You are totally right about that taking more logic into synchronous domain domain is an almost universal solution to getting more performance. The faster the chips become, the more IO gap becomes. Experienced MCU programmers know quite a number of algorithms that may run faster on "dumb" 100mhz MCUs than on a 20-cored Xeon, and are proud to show them off. Even when some of MCU cores may take like 6 cycles to do an arithmetic op, their strong side is that it will be taking them these 6 cycles to do that all and every time consistently, and that their memory runs synchronously. Say, in a 1000000 cycle loop, where every operation results in cache miss, Xeon will spend more time on a loop than a wimpy 100Mhz MCU that simply don't take any penalty on unpredictable, heavy on synchronous ops cope. Apple A11 is so fast for an ARM just because they choose the simplest, the surest, most direct, but also the most expensive way to scale performance: tons of SRAM and prefetch logic everywhere to reduce IO gap as much as possible.

marcosdumay 8y ago

Memory bandwidth is a problem. A large one. But saying that "most" CPU cycles are wasted to it is a huge overrate. I have read estimates that go as high as 1/3 of the CPU lost to it (sorry, no reference here), but even those were not usual.

paulmd 8y ago

20 MB of RAM should be enough for anybody! /s I was reading a thing a while back on here or Reddit where someone asked about an OS that ran entirely in cache, after all in principle a modern processor has more L3 than an original Unix minicomputer. Since many of the memory subsystems operate on the assumption that changes should be propagated to memory that you would still need to have RAM there even if you weren't really "using" it. There was a suggestion that since the bringup code begins entirely on-processor (before initializing RAM) that you could hack the Management Engine and keep running in that mode forever.