3 ms·

My admittedly less charitable take is that they may have been sitting on this 1 operation optimization for a while, noticed it works on the new method, and only

• kcexn 4y ago
  More likely, I think it's that the first principles approach they have used to find this optimisation has applications in domains far outside of optimising matrix multiplications. Only because the Google paper crossed their desk did they even think to try it on matrix multiplications. And then more or less by luck, it happened to turn out to be slightly better. If it had produced exactly the machine learning algorithm, it would have been less interesting. The speed of the response makes me feel that the authors are extremely good at their domain, and that finding this algorithm was a reasonably trivial problem for them. The slowness of the follow up paper, I believe is that the authors understand that their mathematical domain is wildly far outside of what most people can easily understand, so they're going to take their time to write it up well, instead of drop a whole bunch of maths definitions that nobody else in the world understands.
  • mattkrause 4y ago
    The authors were already working on other small-matrix optimizations. They have a 2020 preprint about choosing good pivots for up to 8x8 matrices. https://arxiv.org/abs/2006.01623 https://arxiv.org/abs/2006.01623
• woopwoop 4y ago
  That seems fine to me. What's so sinister about that?
  • robbomacrae 4y ago
    Well there could be an argument that computational saving info should be shared to reduce energy usage but I also think it's fine to sit on it a while.