6 ms·

Damn i hoped we would one day get a customizable fpga into our CPUs. I hoped that it would make sense to install certain instructions on your fpga depending on

• tverbeure 3y ago
  > I hoped that it would make sense to install certain instructions on your fpga depending on your workloads. It's one of those things that seem like a good idea, but they just don't work out in practice. FPGA LUTs are just way too slow. You'd have to find a case where doing something on a 3GHz CPU clock running multiple instruction parallel gets outperformed by LUTs that runs at 700MHz (at best). And when you cascade the LUTs, they become slower too. And that's without solving the problem of closely coupling a CPU pipeline with FPGA logic. > What do not see what they mean by ratio here. Do they mean die ratio between cpu and fpga? What they mean is: in something like the Zynq FPGA family, I want a die with 2 CPU cores and 5000K LUTs. The other guy wants 8 CPU cores and 2000K LUTs. It works for narrow applications like signal processing where power efficiency and cost isn't a top concern, but for a hyperscaler, power consumption is a very important metric. As is the cost of paying for a significant part of the silicon die that's sitting there unused.
  • almatabata 3y ago
    Thanks for clarifying.
  • mikewarot 3y ago
    The right kind of sea of LUTs can outperform anything even if it's clocked at 100 Mhz... the trick is to get a pipeline filled, instead of trying to outrun light. Imagine an LLM with a new token every 10 nS
    • coryrc 3y ago
      But you can't fit that "sea" into a processor core as a separate unit powered by unique instructions (like an FPU or ALU) because the speed demands the cache and ALU be there (or else it can't run at 3GHz any more). Once it's not that close it doesn't make sense to be on the same chip; maybe a neighboring chiplet. But anybody could do that.
    • jeffreygoesto 3y ago
      It is a totally different way of programming and thinking though and you need to spend a lot of thought on design to make it really fast. We could outpace a 3GHz CPU with a Zynq for image processing but it needs some really good FPGA people and the right split between CPU and FPGA. Nothing the average programmer wants to dive into, more something for cost and power sensitive markets with a relative fixed task, not needing ASICs yet. For networks it could pay off if you have the right amountof multipliers in the right places, but only then. As you wrote, don't leave the chip during the processing and you are good to go.
  • duskwuff 3y ago
    > It's one of those things that seem like a good idea, but they just don't work out in practice. GPGPU sucks a lot of air out of the room as well. There aren't many purely computational problems which FPGAs can solve better than a compute-optimized GPU; even though GPUs aren't quite as flexible, they clock a lot faster, they're cheaper, and they're easier to develop for.
    • rowanG077 3y ago
      I actually think there are a ton of problems which could benefit. But there is just very little mindshare. You basically have 1000x the people working on building software to run efficiently on the GPU. No way FPGA touches that. In addition it's a specialized set of skills to build high performance circuits. Whereas running on a GPU is "just" software. So even though I think FPGA could win out from an efficiency perspective for some problems there is just not enough people/companies working on it to win out. It also doesn't help the tools are raging proprietary garbage fires.
      • ethbr1 3y ago
        That's been a historically really good bet for processor architectures -- developer volume beats hypothetical performance. Essentially the only time it's gone the other way was when there was a toehold in a critical market, where the benefits were so obvious and profitable that they made up for the difficulty (e.g. graphics).
  • j_not_j 3y ago
    > FPGA LUTs are just way too slow If, and of course that is a big if, you can repackage a (parallelizable) calculation into FPGA look-up tables and implement multiples of this (e.g. 8 to 80 times) then you can think maybe it's quicker than CPU at 3GHz. However, you have to include DMA of the data to and fro. It's unlikely to be worth the very extensive effort of integrating two wildly different technologies. On the other hand, it may not be a complicated calculation but FPGA can do much lower latency and smaller variance in latency (hello high-frequency traders). That is a very narrow niche. A simple board with CPU and FPGA is the Arduino MKR Vidor 4000: ARM Cortex 32-bit CPU and Intel Cyclone 10 FPGA). Hardware cost: $85. Full suite of development software $1000 or more (although lesser tools are available for free.)
    • imtringued 3y ago
      >However, you have to include DMA of the data to and fro. It's unlikely to be worth the very extensive effort of integrating two wildly different technologies. That is exactly the part where having the FPGA next to the CPU helps... You can transparently access the CPU cache via an AXI slave port on the CPU on AMD's MPSoCs at a rate of up to 16 bytes per cycle and you get multiple of those.
      • andrepd 3y ago
        Hmm, very interesting! Didn't know that wa possible.
  • KirillPanov 3y ago
    > where doing something on a 3GHz CPU clock running multiple instruction parallel gets outperformed by LUTs that runs at 700MHz Easy: go wide. Make the FPGA-CPU interface four times wider on the FPGA side than the CPU side. Each tick of the CPU clock reads (or writes) one quarter of the bits.
• bfrog 3y ago
  "a customizable fpga into our CPUs" that already happened, it just didn't happen in x86 land. There have been a good number of products from various vendors that connect up hard cores and fpga fabric. power pc cores, riscv cores, and by large arm cores
  • tverbeure 3y ago
    That's not what OP meant though. They were talking about custom CPU instructions implemented with FPGA logic.
    • bfrog 3y ago
      That doesn't sound that beneficial honestly
• eschneider 3y ago
  This doesn't make a lot of sense. I mean, there are SOCs out there with asymmetric cores (say, an ARM A53 and an ARM M4 on the same die) for folks who's workloads warrant that sorta thing. I'd expect there'd be s similar market for CPUs, with built in FPGAs of various sizes.
  • tverbeure 3y ago
    It only makes sense for a few applications. See the popular Xilinx Zynq UltraScale MPSoC product line. They are popular for digital signal processing, for example. But they are not power efficient, and they are very expensive. Good enough for a low volume custom solution for which custom silicon is too expensive. Not for a hyperscaler.
• throwaway4590 3y ago
  Whenever I see talk about Intel's FPGA unit, I link back to an invention I submitted to Intel while I was an intern there [0]. I went through the patent pipeline, but to my knowledge they never did anything with it. This was during the excitement of Intel's original acquisition of Altera. In fairness, I never mocked up a true enough implementation in Verilog to get an idea of real world speedup, and even now, I'm not sure exactly what operations you could see real gain with from small reconfigurable fabrics near the CPU. Still, I liked the elegance of having L1-L3+ FPGA's for speeding up operations of increasing levels of complexity, and I figured programmers smarter than me would find creative ways of using the FPGA's with the added instructions. [0] https://patents.google.com/patent/US10310868B2/ https://patents.google.com/patent/US10310868B2/
  • almatabata 3y ago
    Thanks for sharing. Small question about Image 20, does that represent a use case for an instruction translator? For example you have an arm chip and you want to run x86 code so you offload the x86 instructions to the fpga?
    • throwaway4590 3y ago
      I believe my contributions start at Image 25 on Google. Images 1-24 are generic CPU boilerplate images that the lawyers add to most patents in the field.
  • rowanG077 3y ago
    I mean. If you, the original author, didn't even bother implementing and testing the idea why would you expect someone else would? I personally wouldn't touch such a project with a thousand foot pole.
    • throwaway4590 3y ago
      Having worked in both software and hardware, I understand and appreciate this sentiment in software. But most software engineers don't understand the amount of time and compute that would be required to mock up a novel, yet sufficiently complex as to be realistic, CPU. I did indeed have a basic HDL implementation, as is required for most patents in the US (reduction to practice). But to implement it fully enough to understand what kind of performance changes you could get in a modern CPU... it's safe to estimate it as an order of magnitude harder than the most complex pure software project you've ever built alone, and well beyond the resources of an intern who's just doing this in his spare time. Software is a great gig, because it's easy and you can do it all on your laptop. And I'm sure computer engineers like myself don't appreciate the difficulty of manufacturing true mechanical hardware at companies like Tesla, where a team and a budget would be even more essential to build anything useful. Anyway, the technical committee thought it was worth patenting, and the idea itself is pretty digestible without it working on production silicon, but I have no idea who comes up with product roadmaps at Intel. It was probably buried in the patent graveyard, or maybe someone qualified actually looked at it and realized it wouldn't work.
• amluto 3y ago
  Integrating an FPGA with the actual front-end and register files seems so you can invoke it synchronously, with fast instructions at low latency, seems neat but rather complicated. As for an FPGA asynchronously accessing application memory, I tentatively expect CXL with some shared virtual memory trickery to succeed in this space, at least in a couple years when the dust hopefully settles, and then you can do whatever you want.
• dralley 3y ago
  > Damn i hoped we would one day get a customizable fpga into our CPUs. I hoped that it would make sense to install certain instructions on your fpga depending on your workloads. I guess this either kills that possibility or pushes it into a very far future. Depends on what AMD does with Xilinx.
  • gsmecher 3y ago
    > Depends on what AMD does with Xilinx. Currently the AMD/Xilinx dynamic seems to reverse this: "Depends on what Xilinx does with AMD". AMD's software roadmap for AI/datacentre leans heavily on Vitis (for software) and AI Engines (as an execution platform). CPUs that integrate AI engines are already shipping (Ryzen AI). It's Xilinx technology, but you should expect it to look more like a GPU accelerator than a traditional LUTs-and-routing FPGA. And, as duskwuff have pointed out, this sucks a lot of the oxygen out of the CPU-with-FPGA design space.
    • mathisfun123 3y ago
      > AMD's software roadmap for AI/datacentre leans heavily on Vitis (for software) and AI Engines (as an execution platform). This is incorrect along all 3 dimensions: 1. AMD has its own data-center class GPUs - I don't know how good they are because I don't work on them 2. Vitis is just a brand and will be taken out of the equation before the end of the year. 3. I don't know what execution platform means because AI Engine is one core in a grid of such cores on the chiplets that are on the Phoenix platform (shipped with new Ryzens) and the VCK boards. > It's Xilinx technology, but you should expect it to look more like a GPU accelerator than a traditional LUTs-and-routing FPGA. It is correct that there are no LUTs in the fabric but there are "switchboxes" for data traffic (between cores) and you do have do the routing yourself (or rely on the compiler).
  • imtringued 3y ago
    I am actually surprised how AMD managed to successfully leverage it's FPGAs for machine learning inference. It is competing with Nvidia's Jetson.