14 ms·
A Cloud-Scale Acceleration Architecture: FPGAs in Microsoft’s Datacenters
- ddorian43 10y agoSo what algorithms/data-structures/dbs do they use the fpgas for ?
- algorithmsRcool 10y agoThey are using the FPGAs for specializing network hardware on the fly. This lets them decrease latency across their network as well as allow centralized management. Edit: It looks like they are also allowing applications to use the FGPA-FPGA communication layer to reduce latency for distributed systems. > "All network traffic is routed the FPGA, allowing it to accelerate high-bandwidth flows. An independent PCIe connection to the host is also provided, allowing the FPGA to be used as a local accelerator..." > "By enabling the FPGAs to generate and their own networking packets independent of the each and every FPGA in the datacenter can reach every other one (at a scale of hundreds of thousands) in small number of microseconds, without any intervening. This capability allows hosts to use remote FPGAs for with low latency, improving the economics of the deployment, as hosts running services that do not their local FPGAs can donate them to a global pool and value which would otherwise be stranded."
- pliu 10y agoIf you read the PDF, it seems the FPGAs can be used in a number of ways: 1) As a local co-processor, they have been deployed to accelerate selected Bing search ranking operations. This is described a little bit in the white paper, but I think the most interesting parts are going to be proprietary. 2) As a network accelerator, because they are inline with the server network interface. In the white paper using the FPGA for crypto acceleration is described. This is cool. The CPU overhead of encrypting 40Gbps is significant and managing keys for this many hosts is probably non trivial. Moving encryption to another layer would simplify things significantly I think, let app developers not think about it and let datacenter peeps manage keys independently of whatever is going on in the host. 3) As a large scale distributed processing grid. Because the FPGAs have their own network interfaces, they can be used independent of the host. Regular host network traffic gets passed through unaffected while the FPGA is simultaneously running distributed compute tasks from the grid. The white paper described training a DNN, but also describes their Hardware As A Service delivery model, meaning developers I guess have access to the grid and can deploy whatever they want. I would guess there is a lot of machine learning and map reduce type tasks going here, but who knows. The whitepaper also tantalizingly contains the phrase "cross-datacenter" which implies a globally distributed network of these things. Rad. So that's a long way to say that I think there are very many use cases for this kind of thing. Coupling the FPGAs with regular compute nodes is operationally beneficial I think - they can be co-processors or they can be part of the grid. Microsoft is doing some pretty cool shit. Edit: This whole thing is called Project Catapult. There are some other good papers here https://www.microsoft.com/en-us/research/project/project-catapult/ https://www.microsoft.com/en-us/research/project/project-cat...
- CalChris 10y agoTL;DR Microsoft uses FPGAs. Google uses ASICs. MapD uses GPUs. FPGAs are more flexible. ASICs are faster. I think the FPGA approach is generally more useful and will trickle out into the marketplace. Indeed they are already available. So this article is more about Microsoft's fabric. https://www.mrcy.com/products/ensemble-fcn8213-server-class-fpga-processing-blade-advancedtca/ It's not impossible to think this would make its way into an Azure cloud product. That won't happen with ASICs. But Google will clearly have an advantage on the problems they focus on.
- DannyBee 10y agoIf that's the TL;DR, it don't make a lot of sense. Microsoft/Google/whoever is going to use what makes the most sense for a given set of applications/etc they have, whether it's CPU's, GPU's, FPGA's, ASIC's, or massive armies of intelligent pigeons. I'm not sure why one would think all of them don't do all of them :) (in fact, i'd pretty much guarantee it)
- mycall 10y ago> I think the FPGA approach is generally more usefu Intel agrees with its purchase of Altera.
- Cyph0n 10y agoFPGAs are already powering key parts of both Azure and Bing. I asked Doug Burger after his presentation two weeks back at my university, and he said that currently the fabric is not reconfigurable. In other words, hardware acceleration is not available to customers yet, but it looks like they're working towards that. A nice little tidbit from the talk. They were able to translate all of Wikipedia from English to Russian using 90% of the deployed FPGAs in 0.1 seconds.
- trhway 10y agolooks like a new class of hardware component emerging - NIC with an additional FPGA chip. Or may be a NIC which has only FPGA, an FPGA larger than the NIC's code alone would require. Or just FPGA card with NIC ports on it.
- deadgrey19 10y agoThe company I work for builds exactly that. NICs, which are actually FPGA's with area larger than what is necessary to build the NIC alone. Our customers then either just use the NIC or add custom firmware for their own applications. http://exablaze.com/exanic-x10 http://exablaze.com/exanic-x10 http://exablaze.com/exanic-x40 http://exablaze.com/exanic-x40 We also build switches this way too: http://exablaze.com/exalink-fusion http://exablaze.com/exalink-fusion
- dom0 10y agoMarketing hint: Don't use renders of hardware as hero images if you already have the real hardware (as the video below shows). Using a render always makes the impression that you didn't build it yet.
- deadgrey19 10y agoHa. Thanks yes. Too busy selling stuff to get around to fixing the website. It's on the list of things to do...
- deleted 10y ago[deleted]
- kijiki 10y agoNetFPGA. Been around in various revisions since 2007. https://en.wikipedia.org/wiki/NetFPGA https://en.wikipedia.org/wiki/NetFPGA
- xellisx 10y agoI see they haven't done 25 GB / 100 GB stuff yet.
- webaholic 10y agoI wish they would work out how to run the chisel toolchain on those fpgas. From what I understand, you have to program in verilog currently which is a huge pain. Please enable chisel and put it in your Azure cloud. It will be a great platform for custom software.
- sargun 10y agoAzure and Bing are two different departments.
- LyalinDotCom 10y agoThings will never be perfect as it is a huge company, but thanks to Satya teams are working a lot more together
- webaholic 10y agoThey are moving towards homogeneous systems in all their data centers. Each and every system will have these fpgas.
- wyager 10y agoI've tried both Chisel and Clash. They are both huge improvements over Verilog, but I was more impressed with the latter. Clash converts run-of-the-mill non-recursive Haskell code into hardware, and does so fairly efficiently. It turns out there's a really really close mapping between lazy functional semantics and hardware semantics.
- DigitalJack 10y agoCan you recommend some resources on chisel? I've been an ASIC/FPGA designer for 20 years. We mostly do VHDL and SystemVerilog for design, and SystemVerilog for verification. I've only heard about chisel briefly, used in reference to a risc design I think. I've considered making a lisp like language for design on a few occasions. But maybe chisel solves the pain points?
- aseipp 10y ago
- jackyinger 10y agoThe reuse of their network infrastructure with this approach is elegant, BUT it takes about half of the FPGA to support the network pass through and lightweight transport layer. FPGAs are about as expensive as high end CPUs, can you afford to buy a high end server and burn half the cores on your OS? On the flip side, if Microsoft can really eat that cost and offer FPGA space as a public cloud service, as an FPGA dev, I'd rather pay to play than bust 5K on a high end dev board.
- astrodust 10y agoFPGAs are really expensive today, but I think we're starting to see a real shift in production volumes, especially with Intel jumping in, which might rapidly cut costs on them. Exciting opportunities ahead for those that can take advantage of hardware like this.
- jackyinger 10y agoIt certainly is exciting. Tech that began by replacing discrete logic ICs (TI 7400 series) is getting powerful enough to pack a computing punch. Look up systolic array architectures for matrix multiplication if you're not already familiar. They play nice with FPGAs (which are design space limited in comparison to ASICs). And can efficiently implement algorithms that are prone to combinatoric explosion by amortizing expenisve off chip memory accesses over multiple operations as the data passes trough the sys array. If the FPGA manufactures want to open the advantages to all comers they'd make their design software free (as in beer, not freedom) an pAy for it with silicon sales.
- dom0 10y ago> It certainly is exciting. Tech that began by replacing discrete logic ICs (TI 7400 series) is getting powerful enough to pack a computing punch. <nitpick> Those were PALs, basically fuse-programmed logic-in-tables. Even a single of the later CPLDs could already replace a board or two filled with SSI logic. FPGAs took that to replacing an entire cabinet.
- adrianratnapala 10y agoIs this entirely about how Microsoft is using FPGAs for some of its own internal datacenter needs. In that case I can understand why Google might see ASICs as a better way to do the same thing. Where I see FPGAs shining on the cloud is if vendors started putting them online so that clients could program them for their own applications. But I don't know enough about such things to know if there is yet a market for that.
- youdontknowtho 10y agoI think that it's only a matter of time until you see that from Amazon and Azure. Financial companies have been offloading hot paths in code to FPGA for about five years.
- user5994461 10y ago> [From the article] In this paper we propose a new cloud architecture that uses reconfigurable logic to accelerate both network plane functions and applications. This Configurable Cloud architecture places a layer of reconfigurable logic (FPGAs) between the network switches and the servers, enabling network flows to be programmably transformed at line rate... Network planes are done with FPGA and ASIC since a long time. Microsoft is late to the game.
- LargoLasskhyfv 10y agoMaybe late rolling it out in production. Experimenting with it not so much. 2007 https://www.microsoft.com/en-us/research/project/emips/ https://www.microsoft.com/en-us/research/project/emips/ Even open sourcing it in 2011 http://blog.netbsd.org/tnf/entry/support_for_microsoft_emips_extensible http://blog.netbsd.org/tnf/entry/support_for_microsoft_emips...
- gumby 10y agoI'm not convinced that FPGAs are effective in general purpose computation. FPGAs are quite expensive in terms of power, density and overall cost per function. It's not like this hasn't been tried many times before (Google even bought a company that used FPGAs for a reconfigurable network stack processor, but as far as I can see the product never made it).