3 ms·
CPUs __ARE__ an ASIC. You can get a commodity CPU today at 4.7+ GHz clock and 64 MB of L3 cache (AMD Ryzen 9 5950X). There's no FPGA / configurable logic in th
by dragontamer 5y ago
CPUs __ARE__ an ASIC.
You can get a commodity CPU today at 4.7+ GHz clock and 64 MB of L3 cache (AMD Ryzen 9 5950X). There's no FPGA / configurable logic in the world that comes even close. AMD's even shown off a test chip with 2x96 MB of L3 cache.
GPUs __ARE__ an ASIC, but with a different configuration than CPUs. An AMD MI100 comes with 120 compute units, each CU has 4x4x16 SIMD-lanes and each lane has 256 x 32-bit __registers__. (AMD runs each lane once-per-4 clock ticks)
That's 32MB of __registers__, accessible in once every 4th clock tick. Sure, a clock of 1.5GHz is much slower but you ain't getting this number of registers or SRAM on any FPGA.
> I am not an expert in this space, but it seems like the only algorithms I've seen lately people proposing custom hardware for at scale are neural network evaluation
The FPGA / ASIC stuff is almost always "pipelined systolic array". The systolic array is even more parallel than a GPU, and requires data to move "exactly" as planned. The idea is that instead of moving data to / from central registers, you move data across the systolic array to each area that needs it.
----------
CPUs: Today's CPUs __discover__ parallelism in standard assembly code (aka: Instruction level parallelism). It spends a lot of energy "decoding", which is really "scheduling" which instruction should run in which of some 8 to 16 pipelines (depending on AMD Zen vs Skylake vs Apple M1). Each pipeline has different characteristics (this one can add / multiply, but another one can add / multiply / divide). Sometimes some instructions take up the whole pipeline (divide), other times, multiply takes 5 clock ticks but can "take an instruction" every clock tick.
CPUs spend a huge amount of effort discovering the optimal, parallel, execution for any chunk of assembly code, and today is scanning ~400 to 700 instructions to do so (the rough size of a typical CPU's reorder buffer). That's why branch prediction is so important.
In effect: CPUs are today's ultimate MIMD (multiple instructions / multiple data) computers... with a big decoder in front ensuring the various pipelines remain full.
-----------
GPUs: SIMD execution. If parallelism is known ahead of time, this is a better architecture. Instead of spending so much space and energy on discovering parallelism, you have the programmer explicitly write a parallel program from the start.
GPUs are the today's ultimate SIMD computers (single instruction / multiple data) computers.
-------------
Systolic Array: Will never be a general processor, must be designed for each specific task at hand. Can only work if all data-movements are set in stone ahead of time (such as matrix multiplication). Under these conditions, is even more parallel and efficient than a GPU.
- moonbug 5y agoRemind what the ASIC acronym means
- dragontamer 5y agoApplication specific integrated circuit. The Intel / AMD CPUs are an ASIC that execute x86 as quickly as possible. Turns out to be a very important application :-) My point being: you aren't going to beat a CPU at a CPU's job. A CPU is an ASIC that executes assembly language as quickly as possible. Everything in that chip is designed to accelerate the processing of assembly language. -------- You win vs CPUs by grossly changing the architecture. Traditionally, by using a systolic array. (Which is very difficult to make a general purpose processor for).
- Brian_K_White 5y agoI think it's at least a fair question to ask what makes "execute x86" such an important application? Sure the x86 cpu is the best way to execute x86 instructions, but so what? I do not actually care about x86. I don't write it or read it. I write bash and c and kicad and ooenscad and markdown etc..., and really even those are just todays convenient means of expression. The same "what's so untouchable about x86" is true for c. I actually care about manipulating data. I don't mean databases, I mean everything, like the input from a sensor and the output to an actuator or display is all just manipulating data at the lowest level. Maybe this new architecture idea can not perform my freecad modelling task faster or more efficiently than my i7, but I see nothing about the macroscopic job that dictates it's already being mapped to hardware in the most elegant way possible translating to x86 ISA and executing on an x86 asic.
- dragontamer 5y ago> I think it's at least a fair question to ask what makes "execute x86" such an important application? x86, ARM, POWER9, and RISC-V are all the same class of assembly languages. There's really not much difference today in their architectures. All of them are heavily pipelined, heavily branch predicted, superscalar out-of-order speculative processors with cache coherence / snooping to provide some kind of memory model that's standardizing upon Acquire/Release semantics. (Though x86 remains in Total-store ordering model instead). It has been demonstrated that this architecture is the fastest for executing high level code from Bash, C, Java, Python, etc. etc. Any language that compiles down into a set of registers / jumps / calls (including indirect calls) that supports threads of execution are inevitably going to look a hell of a lot like x86 / ARM / POWER9. ---------- If you're willing to change to OpenCL / CUDA, then you can execute on SIMD-computers such as NVidia Ampere or AMD CDNA. Its a completely different execution model than x86 / ARM / POWER9 / RISC-V, with a different language to support the differences in performance (ex: x86 / POWER9 have very fast spinlocks. CUDA / OpenCL has very fast thread-barriers). There's a CUDA-like compiler for x86 AVX + ARM-NEON called "ispc" for people who want to have CUDA-like programming on a CPU. But it executes slower, because CPUs have much smaller SIMD-arrays than a GPU. (but there's virtually no latency, because x86 AVX / ARM-NEON SIMD registers are in the same core as the rest of their register space. Like... 1 clock or 2 clocks latency but nothing like the 10,000+ clock ticks to communicate to a remote GPU) ---------- Look, if webservers and databases and Java JITs / Bash interpreters / Python interpreters could be executed by something different (ex: a systolic array), I'm sure someone would have tried by now. But look at the companies who made Java: IBM and Sun. What kind of computers did they make? POWER9 / SPARC. That's the fruit of their research: the computer architecture that best suites Java programming according to at least two different sets of researchers. And what is POWER9? Its a heavily pipelined, heavily branch predicted, superscalar out-of-order speculative core with cache-coherent acquire/release semantics for multicore communication. Basically the same model as a x86 processor. POWER9 even has the same AES-acceleration and 128-bit vector SIMD units similar to x86's AVX or SSE instructions. You get a few differences (SMT4 on POWER9 and bigger L3 cache), but the overall gameplan is extremely similar to x86.