3 ms·
> So a transformation called 'lowering' is used to convert those convolutions into matrix-matrix multiplications (GEMMs) which GPUs can execute efficiently. Th
by radarsat1 10y ago
> So a transformation called 'lowering' is used to convert those convolutions into matrix-matrix multiplications (GEMMs) which GPUs can execute efficiently.
This confuses me a bit, I'd love to know the details of this "lowering"? How is convolution not already a very straight-forward linear algebra operation?
- zebrafish 10y agohttps://cs.brown.edu/~sk/Publications/Papers/Published/bck-lowering-opt-trans-frp/paper.pdf https://cs.brown.edu/~sk/Publications/Papers/Published/bck-l... FrTime induces construction of the dataflow graph by redefining operations through an implicit lifting transformation. Lifting takes a function that operates on constant values and produces a new function that performs the same operation on time-varying values. Each time the program applies a lifted function to time-varying arguments, it builds a new node and connects it to the nodes representing the arguments... ...Unfortunately, this implicit graph construction can be very inefficient. Every application of a lifted function may create a new dataflow node, whose construction and maintenance consume significant amounts of time and space.... ...The technique works by collapsing regions of the dataflow graph into individual nodes. This moves computation from the dataflow model back to traditional call-by-value, which the runtime system executes much more efficiently. Because this technique undoes the process of lifting, we call it lowering. edit* or maybe?: http://physics.gmu.edu/~joe/PHYS428/Topic5.pdf http://physics.gmu.edu/~joe/PHYS428/Topic5.pdf slide 16?
- radarsat1 10y agoInteresting. Thanks!
- dnautics 10y agoI'm not 100% sure but I think justin is explaining the lowering transformation here: https://www.youtube.com/watch?v=dUTzeP_HTZg&t=36m30s https://www.youtube.com/watch?v=dUTzeP_HTZg&t=36m30s