4 ms·

I thought the point of ATLAS wasn't the idea that automagical tuning was going to necessarily be best. Rather that it was expensive to [edit hand tune] for all

by ska 3y ago

I thought the point of ATLAS wasn't the idea that automagical tuning was going to necessarily be best. Rather that it was expensive to [edit hand tune] for all the hardware variations that were around and this was a practical approach to get to pretty damn good even if nobody had spent the money to produce a hand optimized library for your hardware (or, they had, but you didn't want to pay what they were asking). e.g. I remember vaguely people using it on AMD cpus because they couldn't use icc/mkl effectively...

bee_rider 3y ago
> Rather that it was expensive to automate for all the hardware variations that were around Is the “automate” that I’ve italicized supposed to be hand-tune or something along those lines? If so, I think I agree. I think this was the justification for it, and it made sense at the time, and it might make sense for future architectures or exotic ones now. But eventually Kazushige Gotō wrote gotoBLAS, which really emphasized the importance of the matrix-multiplication kernel and showed how much performance could come from getting it right. Since it is only a handful of routines, it is possible to hand-tune them. BLIS went on to formalize this in a nice open-source library, so any vendor can drop their kernels in. What is special about a particular CPU from this point of view is the SIMD extension and the cache hierarchy/memory system. SIMD extensions, somebody will write a matrix-multiply kernel in assembly or intrinsics before optimizing compilers get really good at using that extension (if they ever get to it!). Cache hierarchy is more abstractable I guess, but there are lots of rectangles and other nice shapes that dedicated tuners can think about, i.e, (PDF warning) https://www.cs.utexas.edu/~flame/pubs/GotoTOMS_revision.pdf https://www.cs.utexas.edu/~flame/pubs/GotoTOMS_revision.pdf I’m sure one could search across all the possible sizes, but it is easier to just have a human identify the right ones. Maybe someone can train an ML model to identify good sizes. Machine learning uses lots of linear algebra, so I guess we’ll have a nice feedback loop, maybe that will kick off the singularity, haha.
- ska 3y ago
  > Is the “automate” that I’ve italicized supposed to be hand-tune or something along those lines? I Gah, yes, typing faster than coffee. Fixed now, thanks. Agree on your points about more recent work.

by ska 3y ago

bee_rider 3y ago

> Rather that it was expensive to automate for all the hardware variations that were around Is the “automate” that I’ve italicized supposed to be hand-tune or something along those lines? If so, I think I agree. I think this was the justification for it, and it made sense at the time, and it might make sense for future architectures or exotic ones now. But eventually Kazushige Gotō wrote gotoBLAS, which really emphasized the importance of the matrix-multiplication kernel and showed how much performance could come from getting it right. Since it is only a handful of routines, it is possible to hand-tune them. BLIS went on to formalize this in a nice open-source library, so any vendor can drop their kernels in. What is special about a particular CPU from this point of view is the SIMD extension and the cache hierarchy/memory system. SIMD extensions, somebody will write a matrix-multiply kernel in assembly or intrinsics before optimizing compilers get really good at using that extension (if they ever get to it!). Cache hierarchy is more abstractable I guess, but there are lots of rectangles and other nice shapes that dedicated tuners can think about, i.e, (PDF warning) https://www.cs.utexas.edu/~flame/pubs/GotoTOMS_revision.pdf https://www.cs.utexas.edu/~flame/pubs/GotoTOMS_revision.pdf I’m sure one could search across all the possible sizes, but it is easier to just have a human identify the right ones. Maybe someone can train an ML model to identify good sizes. Machine learning uses lots of linear algebra, so I guess we’ll have a nice feedback loop, maybe that will kick off the singularity, haha.

ska 3y ago
> Is the “automate” that I’ve italicized supposed to be hand-tune or something along those lines? I Gah, yes, typing faster than coffee. Fixed now, thanks. Agree on your points about more recent work.