5 ms·
Brainwave FPGAs sit directly on the NIC of the server; every FPGA essentially acts as a 40GB/s passthru NIC to a host machine, and each one can talk to the Micr
by aseipp 8y ago
Brainwave FPGAs sit directly on the NIC of the server; every FPGA essentially acts as a 40GB/s passthru NIC to a host machine, and each one can talk to the Microsoft SDN completely on its own independent of the host CPU, in order to bind multiple devices together to create aggregate compute units. FPGAs in a server are partitioned into multiple RTL blocks (so 1 FPGA can host/execute multiple workloads concurrently, not unlike a cloud server), and the "outer" RTL simply routes messages, shuffling packets between RTL blocks, or RTL blocks on different hosts. Thus the CPU is more-or-less cut out of the fast path in a lot of cases, and acts more like a control plane.
Brainwave can deploy to hybrid FPGA/CPU topologies, however (where part of the design is on an FPGA, and part is on a CPU, and they coordinate over the SDN), so the choice of underlying accelerator is flexible and they can scale independent of available capacity.
Microsoft has effectively created a hardware microservice architecture for their large scale FPGA deployments, because the capacity is essentially elastic, and the hardware exposes miniature services that directly talk on the network and coordinate to execute large-scale jobs.
- daxfohl 8y agoOkay so speaking of FPGA as the next gen of AI is a bit misleading? Because it's really just a networking optimization? (I ask because I am intrigued by FPGAs and a bit disheartened if that is all there is to it, not because I'm bashing anything.)
- aseipp 8y agoWell, any "next-gen, hyperscale AI system" at the level of Microsoft is going to be part hardware, part software, part infrastructure; the scale is simply too large these days to short-change any one of these and remain competitive. Microsoft's network and the decoupling of the FPGA from the host machine are major aspects of Brainwave's success for them, but any successful project like this, even with different specific choices -- is going to largely be sustained off "boring" things like a solid networking backbone. Plus there are other concerns at this scale, like power usage, so just thinking of it in terms networking is misleading... Microsoft has had major success with Brainwave and they seem committed to it, but overall the field is advancing pretty rapidly and there will be lots of competitors (not just FPGAs) in this space. And their needs and use cases are unique, so drawing generalized conclusions is a little tough. (As a disclosure, I work on DNNs-on-FPGAs for $WORK, but am speaking generally.)