6 ms·

Microsoft Bets Its Future on a Reprogrammable Computer Chip

88 points by benaadams 10y ago

spydum 10y ago
Curious to hear what exactly the devices do? Article hints at compression and such, and the fact that they've moved devices to the edge of the machine to handle network connectivity makes me think it's a shift for better interaction with SDN more than anything else. I don't get how it has much to do with AI though?
- Neeek 10y ago
  Anything they want. I think it's like having a blank breadboard circuit, Microsoft can mock up a new purpose designed chip and program it in to these FPGAs instead of having to fab one up from scratch. It's hard to tell from the article though, I'm just guessing.
  - j1vms 10y ago
    There are resource limitations to contend with, roughly like an limit on total number of logic gates (e.g. AND, NOR,..) that can be "rewired", and also the bus speed to memory can become prohibitive for some applications. This is why GPUs have been so successful at the subset of problems they are targeted at, just like GPUs as compared to CPUs. However, FPGAs are becoming ever more malleable and heterogeneous: the embedded toolkit (like on-chip hardwired DSPs) is expanding, and the potential for live reconfiguration of parts of an FPGA while other parts are in-use (dynamic/active partial reconfiguration) [0] is pretty exciting. Edit: Though I think it was not mentioned in the article, Microsoft and Intel/Altera have indeed gone this route in no small part due to the empirical death of Moore's Law (which has gone much discussed on HN over the past few years). [0] https://en.wikipedia.org/wiki/Reconfigurable_computing#Partial_re-configuration https://en.wikipedia.org/wiki/Reconfigurable_computing#Parti...
    - Neeek 10y ago
      Yeah, not thinking they are all powerful, just sounds like they can be fine tuned easier than designing a whole new for-purpose chips. Thank you for the info though!
- gradys 10y ago
  FTA > ... in the coming weeks, they will drive new search algorithms based on deep neural networks—artificial intelligence modeled on the structure of the human brain—executing this AI several orders of magnitude faster than ordinary chips could.
  - aab0 10y ago
    That raises as many questions as it answers. Why FPGAs and not GPUs which can run just about any deep neural network but usually faster and more efficiently?
    - Const-me 10y ago
      One reason is FPGAs are more flexible. Sure, GPUs deliver impressive raw performance. To be useful, the task must benefit from massively parallel hardware. GPU hardware works fantastic for shading polygons, training neural networks, or raytracing. For compression and encryption algorithms however, GPUs aren’t terribly good. Another reason is while a GPU delivers impressive bandwidth on parallel-friendly workloads, it’s usually possible to achieve lower latencies with FPGA. An FPGA doesn’t decode any instructions, and its computing modules exchange data directly.
    - Eridrus 10y ago
      GPUs give you great throughput, but they're expensive, eat a lot of power, only work well in batch and aren't tuned to prediction (eg poor INT8 performance and too much VRAM).
    - emcq 10y ago
      GPUs have worse performance per watt than a tuned FPGA. Some newer FPGAs can have 400 megabits of on chip RAM - that's huge, significantly larger than the 128-256k cache typically available on chip for a GPU that turns into big energy savings.
      - p1esk 10y ago
        GPUs have worse performance per watt than a tuned FPGA Citation needed. Maxwell Jetson TX1 is claimed to achieve 1TFlops FP16 at <10W, and soon to be released Pascal based replacement will probably be even more efficient.
- Scaevolus 10y ago
  In Bing, it's used to score results for queries: "One of these FPGA nodes did the Bing search scoring for a 48-node server pod, and was linked to the nodes by four 10 Gb/sec Ethernet ports." http://www.enterprisetech.com/2014/09/03/microsoft-using-fpgas-speed-bing-search/ http://www.enterprisetech.com/2014/09/03/microsoft-using-fpg... Here's a paper about it: https://www.microsoft.com/en-us/research/wp-content/uploads/2016/02/Catapult_ISCA_2014.pdf https://www.microsoft.com/en-us/research/wp-content/uploads/... 4.4, Feature Extraction: "The first stage of the scoring acceleration pipeline, Feature Extraction (FE), calculates numeric scores for a variety of “features” based on the query and document combination. There are potentially thousands of unique features calculated for each document, as each feature calculation produces a result for every stream in the request—furthermore, some features produce a result per query term as well. Our FPGA accelerator offers a significant advantage over software because each of the feature extraction engines can run in parallel, working on the same input stream. This is effectively a form of Multiple Instruction Single Data (MISD) computation."
xt00 10y ago
It's a little odd saying that Microsoft is betting its future on reprogramable chips.. From reading the article they are just simply now using FPGAs where previously they (along with many other companies) were not. It would be like saying Amazon is betting its future on UPS trucks.. What you put in the FPGA matters.. The reprogramability is really just to allow them to implement algorithms that they use heavily that currently are not executed efficiently using current Intel chips. Basically what this is showing us is that the pace of development in software and algorithms that power the cloud is pretty fast so they need the ability to improve performance on data and computation flows that currently (and maybe in the future) will not be well supported on current silicon. FPGAs are not very cost efficient, so unlikely they would not buy that many compared to the number of Intel processors and nvidia gpus. They basically fill in gaps that currently are not being well supported. Intel now owns Alterra so this internal thrust at MSFT may be why they bought them..
- RandomOpinion 10y ago
  >Intel now owns Alterra so this internal thrust at MSFT may be why they bought them. Intel openly acknowledges it. From the article: "Microsoft’s services are so large, and they use so many FPGAs, that they’re shifting the worldwide chip market. The FPGAs come from a company called Altera, and Intel vice president Diane Bryant tells me that Microsoft is why Intel acquired Altera last summer—a deal worth $16.7 billion, the largest acquisition in the history of the largest chipmaker on Earth. By 2020, she says, a third of all servers inside all the major cloud computing companies will include FPGAs." Altera strikes me as an odd choice though. I'd have thought Intel would buy out Xilinx, the industry leader, instead.
  - nickpsecurity 10y ago
    They're similar in capabilities, Altera's tooling is easier-to-use per comments I see, and it's much cheaper buy. Funny thing was that, when Intel was buying Altera, people were suggesting Xilinx should buy AMD. That's how funny the financial situation is between top CPU and FPGA vendors. Haha.
    - 10y ago
andreyk 10y ago
Brief summary: Microsoft is now using FPGAs (flexible hardware that can be 'programmed' to implemebt various chips) as part of its Cloud tech stack. The FPGAs can run their algorithms with better speed and energy efficiency than CPUs, but are less flexible (a pain to alter). The article does not explain it very well, I think MS itself lays it out quite clearly: https://www.microsoft.com/en-us/research/project/project-catapult/ https://www.microsoft.com/en-us/research/project/project-cat... Also annoyingly this article does not link to the paper about this, which also explains it better than the article. I recall this one from MS about how they use the FPGAs in Bing; was pretty impressed by it at the time. https://www.microsoft.com/en-us/research/publication/a-reconfigurable-fabric-for-accelerating-large-scale-datacenter-services/ https://www.microsoft.com/en-us/research/publication/a-recon...
nickpsecurity 10y ago
I'm surprised some cloud vendor hasn't acquired eASIC yet. They throw together stuff easily whose price/performance is between FPGA's and standard ASIC's. Have a bunch of machines for rapid, chip prototyping from FPGA's to their S-ASIC's at 90nm, 45nm, and 28nm. Any IP for networking, storage, security, whatever could be turned into a chip easily. Vendor could pull a IBM or SGI with little boards with high-end CPU's + FPGA's or S-ASIC's for various coprocessors. Not sure if it's ultimately best idea but surprised I haven't seen anyone try it. Wait, just looked up their press releases and it seems they're doing something through OpenPOWER: http://www.easic.com/easic-joins-the-openpower-foundation-to-offer-custom-designed-accelerator-chips/ http://www.easic.com/easic-joins-the-openpower-foundation-to...
- ttul 10y ago
  eASIC is a great little niche-filler. You can get a design fabbed in test quantities for as little as $150K or so, which is akin to "free" next to real ASIC design fabrication.
  - ttul 10y ago
    Note that Altera also offers a similar service but it costs more.
foobarcrunch 10y ago
Whaaa? Isn't this what Tilera and Tabula were chasing? Maybe they were too early and/or didn't have the momentum to drive the industry? It does seem like an inevitable technological evolution direction, however compilers, debuggers and so on will need to optimize for an almost entirely new set of constraints. The thing with using FPGA's in systems (they're great for low-volume, high-priced items where ASICs would be too costly) is they end up just emulating logic which could be more cheaply implemented as actual execution units (as many modern FPGA's things like cache, ROM, ALUs, etc.). That is, it's expensive flexibility that isn't really all that useful. Sure you could reconfigure a "computer" from doing database things to suddenly add more GPU cores to play games, but how useful or power/cost efficient would that be? Sure it's nice to cut down on ASICs and upgrading them after the fact, but it seems like more like category development than offering practical advantages to solve a real problem. Maybe a super-fast HPC-on-a-chip would be possible, but I don't see that we're storage or compute constrained, however we maybe bandwidth and latency constrained in terms of shrinking clusters to a single rack of ridiculously power-hungry reprogrammable chips. Instead of infinitely customizable, arbitrary logic, you might have a crap-ton of simplified RISC cores with some memory and lots of interconnected bandwidth or something in-between FPGA and MPPA. https://en.wikipedia.org/wiki/Massively_parallel_processor_array https://en.wikipedia.org/wiki/Massively_parallel_processor_a...
- Eridrus 10y ago
  I keep hearing that FPGAs suck, but they seem like a reasonable middle ground between CPUs and ASICs in terms of cost efficiency as this article mentions.

88 points by benaadams 10y ago

Curious to hear what exactly the devices do? Article hints at compression and such, and the fact that they've moved devices to the edge of the machine to handle network connectivity makes me think it's a shift for better interaction with SDN more than anything else. I don't get how it has much to do with AI though?

Neeek 10y ago
Anything they want. I think it's like having a blank breadboard circuit, Microsoft can mock up a new purpose designed chip and program it in to these FPGAs instead of having to fab one up from scratch. It's hard to tell from the article though, I'm just guessing.
- j1vms 10y ago
  There are resource limitations to contend with, roughly like an limit on total number of logic gates (e.g. AND, NOR,..) that can be "rewired", and also the bus speed to memory can become prohibitive for some applications. This is why GPUs have been so successful at the subset of problems they are targeted at, just like GPUs as compared to CPUs. However, FPGAs are becoming ever more malleable and heterogeneous: the embedded toolkit (like on-chip hardwired DSPs) is expanding, and the potential for live reconfiguration of parts of an FPGA while other parts are in-use (dynamic/active partial reconfiguration) [0] is pretty exciting. Edit: Though I think it was not mentioned in the article, Microsoft and Intel/Altera have indeed gone this route in no small part due to the empirical death of Moore's Law (which has gone much discussed on HN over the past few years). [0] https://en.wikipedia.org/wiki/Reconfigurable_computing#Partial_re-configuration https://en.wikipedia.org/wiki/Reconfigurable_computing#Parti...
  - Neeek 10y ago
    Yeah, not thinking they are all powerful, just sounds like they can be fine tuned easier than designing a whole new for-purpose chips. Thank you for the info though!
gradys 10y ago
FTA > ... in the coming weeks, they will drive new search algorithms based on deep neural networks—artificial intelligence modeled on the structure of the human brain—executing this AI several orders of magnitude faster than ordinary chips could.
- aab0 10y ago
  That raises as many questions as it answers. Why FPGAs and not GPUs which can run just about any deep neural network but usually faster and more efficiently?
  - Const-me 10y ago
    One reason is FPGAs are more flexible. Sure, GPUs deliver impressive raw performance. To be useful, the task must benefit from massively parallel hardware. GPU hardware works fantastic for shading polygons, training neural networks, or raytracing. For compression and encryption algorithms however, GPUs aren’t terribly good. Another reason is while a GPU delivers impressive bandwidth on parallel-friendly workloads, it’s usually possible to achieve lower latencies with FPGA. An FPGA doesn’t decode any instructions, and its computing modules exchange data directly.
  - Eridrus 10y ago
    GPUs give you great throughput, but they're expensive, eat a lot of power, only work well in batch and aren't tuned to prediction (eg poor INT8 performance and too much VRAM).
  - emcq 10y ago
    GPUs have worse performance per watt than a tuned FPGA. Some newer FPGAs can have 400 megabits of on chip RAM - that's huge, significantly larger than the 128-256k cache typically available on chip for a GPU that turns into big energy savings.
    - p1esk 10y ago
      GPUs have worse performance per watt than a tuned FPGA Citation needed. Maxwell Jetson TX1 is claimed to achieve 1TFlops FP16 at <10W, and soon to be released Pascal based replacement will probably be even more efficient.
Scaevolus 10y ago
In Bing, it's used to score results for queries: "One of these FPGA nodes did the Bing search scoring for a 48-node server pod, and was linked to the nodes by four 10 Gb/sec Ethernet ports." http://www.enterprisetech.com/2014/09/03/microsoft-using-fpgas-speed-bing-search/ http://www.enterprisetech.com/2014/09/03/microsoft-using-fpg... Here's a paper about it: https://www.microsoft.com/en-us/research/wp-content/uploads/2016/02/Catapult_ISCA_2014.pdf https://www.microsoft.com/en-us/research/wp-content/uploads/... 4.4, Feature Extraction: "The first stage of the scoring acceleration pipeline, Feature Extraction (FE), calculates numeric scores for a variety of “features” based on the query and document combination. There are potentially thousands of unique features calculated for each document, as each feature calculation produces a result for every stream in the request—furthermore, some features produce a result per query term as well. Our FPGA accelerator offers a significant advantage over software because each of the feature extraction engines can run in parallel, working on the same input stream. This is effectively a form of Multiple Instruction Single Data (MISD) computation."

xt00 10y ago

It's a little odd saying that Microsoft is betting its future on reprogramable chips.. From reading the article they are just simply now using FPGAs where previously they (along with many other companies) were not. It would be like saying Amazon is betting its future on UPS trucks.. What you put in the FPGA matters.. The reprogramability is really just to allow them to implement algorithms that they use heavily that currently are not executed efficiently using current Intel chips. Basically what this is showing us is that the pace of development in software and algorithms that power the cloud is pretty fast so they need the ability to improve performance on data and computation flows that currently (and maybe in the future) will not be well supported on current silicon. FPGAs are not very cost efficient, so unlikely they would not buy that many compared to the number of Intel processors and nvidia gpus. They basically fill in gaps that currently are not being well supported. Intel now owns Alterra so this internal thrust at MSFT may be why they bought them..

RandomOpinion 10y ago
>Intel now owns Alterra so this internal thrust at MSFT may be why they bought them. Intel openly acknowledges it. From the article: "Microsoft’s services are so large, and they use so many FPGAs, that they’re shifting the worldwide chip market. The FPGAs come from a company called Altera, and Intel vice president Diane Bryant tells me that Microsoft is why Intel acquired Altera last summer—a deal worth $16.7 billion, the largest acquisition in the history of the largest chipmaker on Earth. By 2020, she says, a third of all servers inside all the major cloud computing companies will include FPGAs." Altera strikes me as an odd choice though. I'd have thought Intel would buy out Xilinx, the industry leader, instead.
- nickpsecurity 10y ago
  They're similar in capabilities, Altera's tooling is easier-to-use per comments I see, and it's much cheaper buy. Funny thing was that, when Intel was buying Altera, people were suggesting Xilinx should buy AMD. That's how funny the financial situation is between top CPU and FPGA vendors. Haha.
  - 10y ago

andreyk 10y ago

Brief summary: Microsoft is now using FPGAs (flexible hardware that can be 'programmed' to implemebt various chips) as part of its Cloud tech stack. The FPGAs can run their algorithms with better speed and energy efficiency than CPUs, but are less flexible (a pain to alter). The article does not explain it very well, I think MS itself lays it out quite clearly: https://www.microsoft.com/en-us/research/project/project-catapult/ https://www.microsoft.com/en-us/research/project/project-cat... Also annoyingly this article does not link to the paper about this, which also explains it better than the article. I recall this one from MS about how they use the FPGAs in Bing; was pretty impressed by it at the time. https://www.microsoft.com/en-us/research/publication/a-reconfigurable-fabric-for-accelerating-large-scale-datacenter-services/ https://www.microsoft.com/en-us/research/publication/a-recon...

nickpsecurity 10y ago

I'm surprised some cloud vendor hasn't acquired eASIC yet. They throw together stuff easily whose price/performance is between FPGA's and standard ASIC's. Have a bunch of machines for rapid, chip prototyping from FPGA's to their S-ASIC's at 90nm, 45nm, and 28nm. Any IP for networking, storage, security, whatever could be turned into a chip easily. Vendor could pull a IBM or SGI with little boards with high-end CPU's + FPGA's or S-ASIC's for various coprocessors. Not sure if it's ultimately best idea but surprised I haven't seen anyone try it. Wait, just looked up their press releases and it seems they're doing something through OpenPOWER: http://www.easic.com/easic-joins-the-openpower-foundation-to-offer-custom-designed-accelerator-chips/ http://www.easic.com/easic-joins-the-openpower-foundation-to...

ttul 10y ago
eASIC is a great little niche-filler. You can get a design fabbed in test quantities for as little as $150K or so, which is akin to "free" next to real ASIC design fabrication.
- ttul 10y ago
  Note that Altera also offers a similar service but it costs more.

foobarcrunch 10y ago

Whaaa? Isn't this what Tilera and Tabula were chasing? Maybe they were too early and/or didn't have the momentum to drive the industry? It does seem like an inevitable technological evolution direction, however compilers, debuggers and so on will need to optimize for an almost entirely new set of constraints. The thing with using FPGA's in systems (they're great for low-volume, high-priced items where ASICs would be too costly) is they end up just emulating logic which could be more cheaply implemented as actual execution units (as many modern FPGA's things like cache, ROM, ALUs, etc.). That is, it's expensive flexibility that isn't really all that useful. Sure you could reconfigure a "computer" from doing database things to suddenly add more GPU cores to play games, but how useful or power/cost efficient would that be? Sure it's nice to cut down on ASICs and upgrading them after the fact, but it seems like more like category development than offering practical advantages to solve a real problem. Maybe a super-fast HPC-on-a-chip would be possible, but I don't see that we're storage or compute constrained, however we maybe bandwidth and latency constrained in terms of shrinking clusters to a single rack of ridiculously power-hungry reprogrammable chips. Instead of infinitely customizable, arbitrary logic, you might have a crap-ton of simplified RISC cores with some memory and lots of interconnected bandwidth or something in-between FPGA and MPPA. https://en.wikipedia.org/wiki/Massively_parallel_processor_array https://en.wikipedia.org/wiki/Massively_parallel_processor_a...

Eridrus 10y ago
I keep hearing that FPGAs suck, but they seem like a reasonable middle ground between CPUs and ASICs in terms of cost efficiency as this article mentions.