2 ms·
> Rather that it was expensive to automate for all the hardware variations that were around Is the “automate” that I’ve italicized supposed to be hand-tune or
by bee_rider 3y ago
> Rather that it was expensive to automate for all the hardware variations that were around
Is the “automate” that I’ve italicized supposed to be hand-tune or something along those lines? If so, I think I agree. I think this was the justification for it, and it made sense at the time, and it might make sense for future architectures or exotic ones now.
But eventually Kazushige Gotō wrote gotoBLAS, which really emphasized the importance of the matrix-multiplication kernel and showed how much performance could come from getting it right. Since it is only a handful of routines, it is possible to hand-tune them. BLIS went on to formalize this in a nice open-source library, so any vendor can drop their kernels in.
What is special about a particular CPU from this point of view is the SIMD extension and the cache hierarchy/memory system. SIMD extensions, somebody will write a matrix-multiply kernel in assembly or intrinsics before optimizing compilers get really good at using that extension (if they ever get to it!). Cache hierarchy is more abstractable I guess, but there are lots of rectangles and other nice shapes that dedicated tuners can think about, i.e, (PDF warning)
https://www.cs.utexas.edu/~flame/pubs/GotoTOMS_revision.pdf https://www.cs.utexas.edu/~flame/pubs/GotoTOMS_revision.pdf
I’m sure one could search across all the possible sizes, but it is easier to just have a human identify the right ones.
Maybe someone can train an ML model to identify good sizes. Machine learning uses lots of linear algebra, so I guess we’ll have a nice feedback loop, maybe that will kick off the singularity, haha.
- ska 3y ago> Is the “automate” that I’ve italicized supposed to be hand-tune or something along those lines? I Gah, yes, typing faster than coffee. Fixed now, thanks. Agree on your points about more recent work.