5 ms·

I have a computer engineering degree and what you are saying about using FPGAs for general purpose computing was correct when I graduated 20 years ago. It was 1

I have a computer engineering degree and what you are saying about using FPGAs for general purpose computing was correct when I graduated 20 years ago. It was 10-100 times more correct a decade after that, and is another 10-100 times more correct today. So somewhere between 100 and 10,000 times more correct now than in the 90s. And between 1000 and 1,000,000 times more correct in 2029! But we're still using the same 3 GHz (effectively single threaded) chips today as then. Sure, RAM frequency is higher, but so is latency. Computers today are closer to 10 times faster than computers of the 90s, not 10,000 times faster like they should be. Except for video cards, which really are 100 or 1000 times faster, because they break with the single-threaded model. FPGAs would work great for general purpose computing, but we're still missing a proper language for programming them. VHDL/Verilog is more like assembly language than the functional language we would need to do it effectively. I haven't really kept up on this because it's been too depressing, but here are some promising approaches: https://github.com/dgrnbrg/piplin https://github.com/dgrnbrg/piplin https://catherineh.github.io/programming/2016/12/26/haskell-on-a-xilinx-fpga https://catherineh.github.io/programming/2016/12/26/haskell-... https://www.eetimes.com/document.asp?doc_id=%201329857&page_number=6 https://www.eetimes.com/document.asp?doc_id=%201329857&page_... https://news.ycombinator.com/item?id=14546535 https://news.ycombinator.com/item?id=14546535 So we'd need: * An HDL wrapper that guarantees that any hardware description downloaded to an FPGA won't short it out. [might already exist, but needs proven examples/unit tests of pathological edge cases] * A Lisp to HDL compiler, preferably with optimization. [time-consuming, but not difficult] * A high-level functional or vector or stream language (Clojure/Elixir/MATLAB/Erlang/Go) to Lisp transpiler. [somewhere between trivial and straightforward, might already exist] * An example implementation of a CPU written in a functional/vector/stream language, probably MIPS [not difficult, just time consuming to convert an existing spec to code] It would be good to keep the transistor count under 1 million (say 100,000 gates), and possibly start with an 8, 16 or 32 bit implementation and emulate 64 bit and higher operations in microcode. Most CPU gates are wasted on out-of-order execution and cache, which aren't as important in parallel and stream-based/DSP computing: https://en.wikipedia.org/wiki/Transistor_count https://en.wikipedia.org/wiki/Transistor_count When I graduated, I saw a brand new chip where 75% of the die was for cache, although I can't remember which model number it was. Chips today are probably worse, since their speed to transistor count ratio has mostly gone down. It looks like FPGAs might have stopped reporting gate count since they've doubled down on the proprietary route. A Xilinx Virtex-7 2000T FPGA from 2011 had 6.8 billion transistors which could implement roughly 20 million ASIC gates: https://www.eetimes.com/document.asp?doc_id=1316816 https://www.eetimes.com/document.asp?doc_id=1316816 So 8 years later with Moore's law that should be 5 doublings (32 times), or roughly 200 billion transistors and 640 million ASIC gates on a 2019 FPGA. The article says it takes about 340 transistors to form a gate on an FPGA, which seems high to me. But conservatively, 640 million gates would allow us to put 6,400 of our 100,000 gate cores on one FPGA. Maybe someone ambitious can figure out how much RAM could be allocated per core in a 2D grid, based on the total core count. I'd like at least 1 MB per core, but that might conservatively require something like 10 million gates, which might limit us to 64 cores in RAM alone: https://www.quora.com/How-many-transistors-flip-flops-and-gates-are-required-to-store-one-byte-of-data-in-a-memory-chip-online https://www.quora.com/How-many-transistors-flip-flops-and-ga... It’s unfortunate that there is so much hand waving around transistor, gate and logic cell count. But it looks like we could put something like 64, 640 or 6,400 cores on a single FPGA today depending on how much RAM we allocate per core, and if we had the right software. And this would be a true parallel computer, probably running between 100 MHz and 1 GHz. So we could play around with things like copy-on-write (COW), content-addressable memory and embarrassingly parallel computation in all the areas that today's computers are weak at (things like ray tracing, voxels, genetic programming, neural nets and so on) and at least see how classic (UNIX-style) programming compares with the bare-hands/burdensome languages like CUDA and OpenCL. If an FPGA can do all that, then surely it could devote some of its gates to the kind of reprogrammable logic you mentioned (perhaps for video transcoding, bitcoin mining, emulating video game consoles, things of that nature).

erichocean 7y ago
> Computers today are closer to 10 times faster than computers of the 90s, not 10,000 times faster like they should be. This isn't even close to true for many classes of problems. For 3D rendering software running on CPUs, this is observably not true: todays machines are computing 10,000x faster than what was possible in 1994, when Toy Story was rendered. Not to mention massively larger data sets are being rendered now thanks to storage, memory, and general I/O improvements. In fact, path tracing (now the dominant 3D rendering method, including at Pixar) was considered elegant but impractical in the 90s because it required way too many CPU cycles and way too much memory (the whole scene, roughly, needs to be in memory at all times). It took a 1000x (now 10,000x) increase in computer performance across all dimension to make path tracing viable. Does any of that sound like just a roughly 10x improvement in 25 years?
- zackmorris 7y ago
  I was excluding video cards, and comparing a 1999 computer (say a 300 MHz 1 core Intel Pentium II with 100 MHz RAM) to a 2019 computer (say a 3 GHz 8 core Intel I9 with 1 GHz RAM). There is some wiggle room there on clock and bus speeds and width, but all within a similar order of magnitude: https://www.cpubenchmark.net/singleThread.html https://www.cpubenchmark.net/singleThread.html This chart doesn’t go as low as we need, but we can extrapolate the Pentium II to a score of about 50 and the I9 to 2500 ( although if we include the I9’s 8 cores, we get a score more like 20,000 https://www.cpubenchmark.net/cpu.php?cpu=Intel+Core+i9-9900K+%40+3.60GHz&id=3334 https://www.cpubenchmark.net/cpu.php?cpu=Intel+Core+i9-9900K... ). So we’re looking at about 50x faster single-threaded performance between 1999 and 2019. That means that since 2019 clocks are 10x faster than 1999 clocks, there has been only a 5x performance increase per cycle (most of this probably due to longer pipelines, larger cache and wider busses which are evolutionary rather than revolutionary technologies). Yes, there have been major price decreases. But only minor increases in performance. Computers have exponentially more transistors, but only linearly better processing power and bus speed. Had performance kept up with the 100x increase in transistor count per decade (10,000x every 20 years), The I9 with 8 cores should have scored about 5010,0008 = 4,000,000. But it only scored 20,000, so it’s only reaching about half a percent of its potential performance. We don’t hear any of this talked about much because it’s an inconvenient truth. But from my perspective, Moore’s law ended around 2000. If it was up to me, I would scrap the rat race towards incrementally faster single-threaded performance. One divided by half a percent indicates that chips today should have roughly 8*50 = 400 cores to adequately make use of their potential processing power. For a hardwired (non-FPGA) chipset, I’d reach higher than that and target 1024 cores running at 3 GHz, with as much as 16 GB of on-chip RAM running at 1 GHz (16 MB per core so they can be programmed with local copies of their own operating system), for a chip with roughly the same transistor count as today, and costing under $1000. My training was for designing a MIPS-style chip running at roughly 100 MHz. I don’t see any mystery in designing the scaled-up chip I’m suggesting. In fact, it would be far simpler and cheaper to design because we’d only have to do it once for perhaps a 4-6 stage pipeline with little or no branch prediction or cache logic and then repeat that in a 2D mesh. A single core would run up to 5x slower than an I9’s, but we’d have over 1000 more of them to do whatever we wanted with.

erichocean 7y ago

> Computers today are closer to 10 times faster than computers of the 90s, not 10,000 times faster like they should be. This isn't even close to true for many classes of problems. For 3D rendering software running on CPUs, this is observably not true: todays machines are computing 10,000x faster than what was possible in 1994, when Toy Story was rendered. Not to mention massively larger data sets are being rendered now thanks to storage, memory, and general I/O improvements. In fact, path tracing (now the dominant 3D rendering method, including at Pixar) was considered elegant but impractical in the 90s because it required way too many CPU cycles and way too much memory (the whole scene, roughly, needs to be in memory at all times). It took a 1000x (now 10,000x) increase in computer performance across all dimension to make path tracing viable. Does any of that sound like just a roughly 10x improvement in 25 years?

zackmorris 7y ago
I was excluding video cards, and comparing a 1999 computer (say a 300 MHz 1 core Intel Pentium II with 100 MHz RAM) to a 2019 computer (say a 3 GHz 8 core Intel I9 with 1 GHz RAM). There is some wiggle room there on clock and bus speeds and width, but all within a similar order of magnitude: https://www.cpubenchmark.net/singleThread.html https://www.cpubenchmark.net/singleThread.html This chart doesn’t go as low as we need, but we can extrapolate the Pentium II to a score of about 50 and the I9 to 2500 ( although if we include the I9’s 8 cores, we get a score more like 20,000 https://www.cpubenchmark.net/cpu.php?cpu=Intel+Core+i9-9900K+%40+3.60GHz&id=3334 https://www.cpubenchmark.net/cpu.php?cpu=Intel+Core+i9-9900K... ). So we’re looking at about 50x faster single-threaded performance between 1999 and 2019. That means that since 2019 clocks are 10x faster than 1999 clocks, there has been only a 5x performance increase per cycle (most of this probably due to longer pipelines, larger cache and wider busses which are evolutionary rather than revolutionary technologies). Yes, there have been major price decreases. But only minor increases in performance. Computers have exponentially more transistors, but only linearly better processing power and bus speed. Had performance kept up with the 100x increase in transistor count per decade (10,000x every 20 years), The I9 with 8 cores should have scored about 5010,0008 = 4,000,000. But it only scored 20,000, so it’s only reaching about half a percent of its potential performance. We don’t hear any of this talked about much because it’s an inconvenient truth. But from my perspective, Moore’s law ended around 2000. If it was up to me, I would scrap the rat race towards incrementally faster single-threaded performance. One divided by half a percent indicates that chips today should have roughly 8*50 = 400 cores to adequately make use of their potential processing power. For a hardwired (non-FPGA) chipset, I’d reach higher than that and target 1024 cores running at 3 GHz, with as much as 16 GB of on-chip RAM running at 1 GHz (16 MB per core so they can be programmed with local copies of their own operating system), for a chip with roughly the same transistor count as today, and costing under $1000. My training was for designing a MIPS-style chip running at roughly 100 MHz. I don’t see any mystery in designing the scaled-up chip I’m suggesting. In fact, it would be far simpler and cheaper to design because we’d only have to do it once for perhaps a 4-6 stage pipeline with little or no branch prediction or cache logic and then repeat that in a 2D mesh. A single core would run up to 5x slower than an I9’s, but we’d have over 1000 more of them to do whatever we wanted with.