3 ms·
I think the better way to look at it is that the norm is a very heterogenous architecture with all sorts of purpose built accelerators. However, while Moore’s l
by vlovich123 4y ago
I think the better way to look at it is that the norm is a very heterogenous architecture with all sorts of purpose built accelerators. However, while Moore’s law was in play the general compute ate all the accelerators (hence integrated fpus, simd, etc). Now that it’s dead, we’re going to go back and this is likely the new normal unless we somehow unlock a new era of exponential growth (eg optical compute). However, at this time that seems less likely so we’re likely to see more of these accelerators. The challenge of course is scale - specialized accelerators by definition have smaller market niches which in turn limits how cheap they can be and the creativity of what people think to do with it (imagine what computing would look like if only supercomputers had floating point units to support numerical simulations).
- ethbr0 4y agoAmdahl's law always resonated for me for this reason: you only gain speedup equal to the portion of the total work that can be accelerated. From that perspective, speedup is less about hyperefficient design of a brand new thing, and more about recognizing large portions of existing workload that are amenable to acceleration. More successful accelerators have been borne from the latter approach (superscalar/hyperthreading, GPUs once OpenGL/DirectX dominated, fixed function video decode hardware) than the former (Itanium/VLIW)*. Or in simpler form, never start a value proposition with "First, rewrite all your software..." * ML is an odd duck, as it's somewhat co-evolving with its own accelerators?