3 ms·

Sounds similar to ATLAS, an automatically tuned BLAS library from the 1990s https://softlib.rice.edu/pub/CRPC-TRs/reports/CRPC-TR98751.pdf https://softlib.rice.

by fritzo 3y ago

Sounds similar to ATLAS, an automatically tuned BLAS library from the 1990s https://softlib.rice.edu/pub/CRPC-TRs/reports/CRPC-TR98751.pdf https://softlib.rice.edu/pub/CRPC-TRs/reports/CRPC-TR98751.p...

onetimeuse92304 3y ago
I am not surprised. I like to say that it is hard to have an original thought. Even if you think you are original, it is more likely somebody else has already done it in the past. Most of the computing novelties I am seeing are really old ideas, revisited and potentially improved upon. Thanks for the link, I will take a look when I have a moment:)
- bee_rider 3y ago
  Eventually the pendulum swung back for BLAS libraries, the currently popular strategy is hand-tuned assembly kernel in libraries where the most important kernels have been identified. But BLAS is a very weird case, there are only a couple kernels that you care about (and the most important, GEMM, has been studied like crazy). And BLAS is used under the hood by pretty much the whole HPC community, so there’s incentive for tons of expert human time investment. You can look at ilbflame/BLIS for the modern strategy. ATLAS is still neat, perhaps one can even derive more generally applicable lessons from that library, not sure…
  - ska 3y ago
    I thought the point of ATLAS wasn't the idea that automagical tuning was going to necessarily be best. Rather that it was expensive to [edit hand tune] for all the hardware variations that were around and this was a practical approach to get to pretty damn good even if nobody had spent the money to produce a hand optimized library for your hardware (or, they had, but you didn't want to pay what they were asking). e.g. I remember vaguely people using it on AMD cpus because they couldn't use icc/mkl effectively...
    - bee_rider 3y ago
      > Rather that it was expensive to automate for all the hardware variations that were around Is the “automate” that I’ve italicized supposed to be hand-tune or something along those lines? If so, I think I agree. I think this was the justification for it, and it made sense at the time, and it might make sense for future architectures or exotic ones now. But eventually Kazushige Gotō wrote gotoBLAS, which really emphasized the importance of the matrix-multiplication kernel and showed how much performance could come from getting it right. Since it is only a handful of routines, it is possible to hand-tune them. BLIS went on to formalize this in a nice open-source library, so any vendor can drop their kernels in. What is special about a particular CPU from this point of view is the SIMD extension and the cache hierarchy/memory system. SIMD extensions, somebody will write a matrix-multiply kernel in assembly or intrinsics before optimizing compilers get really good at using that extension (if they ever get to it!). Cache hierarchy is more abstractable I guess, but there are lots of rectangles and other nice shapes that dedicated tuners can think about, i.e, (PDF warning) https://www.cs.utexas.edu/~flame/pubs/GotoTOMS_revision.pdf https://www.cs.utexas.edu/~flame/pubs/GotoTOMS_revision.pdf I’m sure one could search across all the possible sizes, but it is easier to just have a human identify the right ones. Maybe someone can train an ML model to identify good sizes. Machine learning uses lots of linear algebra, so I guess we’ll have a nice feedback loop, maybe that will kick off the singularity, haha.
      - ska 3y ago
        > Is the “automate” that I’ve italicized supposed to be hand-tune or something along those lines? I Gah, yes, typing faster than coffee. Fixed now, thanks. Agree on your points about more recent work.
  - owlbite 3y ago
    Also worth observing BLAS is an even weirder case as odds are the CPU architecture designed to ensure they could hit a high % of peak FMA throughput for that kernel, so there's a lot of SW/HW co-design going on there too.
mattkrause 3y ago
The Fourier transform library FFTW does this too. Its "wisdom" system will plan/measure the best way to do transforms of a particular size/type and cache it for future use. More here: https://www.fftw.org/fftw3_doc/Words-of-Wisdom_002dSaving-Plans.html https://www.fftw.org/fftw3_doc/Words-of-Wisdom_002dSaving-Pl...

by fritzo 3y ago

onetimeuse92304 3y ago

I am not surprised. I like to say that it is hard to have an original thought. Even if you think you are original, it is more likely somebody else has already done it in the past. Most of the computing novelties I am seeing are really old ideas, revisited and potentially improved upon. Thanks for the link, I will take a look when I have a moment:)

bee_rider 3y ago
Eventually the pendulum swung back for BLAS libraries, the currently popular strategy is hand-tuned assembly kernel in libraries where the most important kernels have been identified. But BLAS is a very weird case, there are only a couple kernels that you care about (and the most important, GEMM, has been studied like crazy). And BLAS is used under the hood by pretty much the whole HPC community, so there’s incentive for tons of expert human time investment. You can look at ilbflame/BLIS for the modern strategy. ATLAS is still neat, perhaps one can even derive more generally applicable lessons from that library, not sure…
- ska 3y ago
  I thought the point of ATLAS wasn't the idea that automagical tuning was going to necessarily be best. Rather that it was expensive to [edit hand tune] for all the hardware variations that were around and this was a practical approach to get to pretty damn good even if nobody had spent the money to produce a hand optimized library for your hardware (or, they had, but you didn't want to pay what they were asking). e.g. I remember vaguely people using it on AMD cpus because they couldn't use icc/mkl effectively...
  - bee_rider 3y ago
    > Rather that it was expensive to automate for all the hardware variations that were around Is the “automate” that I’ve italicized supposed to be hand-tune or something along those lines? If so, I think I agree. I think this was the justification for it, and it made sense at the time, and it might make sense for future architectures or exotic ones now. But eventually Kazushige Gotō wrote gotoBLAS, which really emphasized the importance of the matrix-multiplication kernel and showed how much performance could come from getting it right. Since it is only a handful of routines, it is possible to hand-tune them. BLIS went on to formalize this in a nice open-source library, so any vendor can drop their kernels in. What is special about a particular CPU from this point of view is the SIMD extension and the cache hierarchy/memory system. SIMD extensions, somebody will write a matrix-multiply kernel in assembly or intrinsics before optimizing compilers get really good at using that extension (if they ever get to it!). Cache hierarchy is more abstractable I guess, but there are lots of rectangles and other nice shapes that dedicated tuners can think about, i.e, (PDF warning) https://www.cs.utexas.edu/~flame/pubs/GotoTOMS_revision.pdf https://www.cs.utexas.edu/~flame/pubs/GotoTOMS_revision.pdf I’m sure one could search across all the possible sizes, but it is easier to just have a human identify the right ones. Maybe someone can train an ML model to identify good sizes. Machine learning uses lots of linear algebra, so I guess we’ll have a nice feedback loop, maybe that will kick off the singularity, haha.
    - ska 3y ago
      > Is the “automate” that I’ve italicized supposed to be hand-tune or something along those lines? I Gah, yes, typing faster than coffee. Fixed now, thanks. Agree on your points about more recent work.
- owlbite 3y ago
  Also worth observing BLAS is an even weirder case as odds are the CPU architecture designed to ensure they could hit a high % of peak FMA throughput for that kernel, so there's a lot of SW/HW co-design going on there too.

mattkrause 3y ago

The Fourier transform library FFTW does this too. Its "wisdom" system will plan/measure the best way to do transforms of a particular size/type and cache it for future use. More here: https://www.fftw.org/fftw3_doc/Words-of-Wisdom_002dSaving-Plans.html https://www.fftw.org/fftw3_doc/Words-of-Wisdom_002dSaving-Pl...