4 ms·
Ironic thing is large part of how ML works is via matrix multiplication.
by TrainedMonkey 5y ago
Ironic thing is large part of how ML works is via matrix multiplication.
- r00fus 5y agoThat's not ironic, it could be revolutionary for this algorithm and ML in general.
- uluyol 5y agoLots of things are like this. I program my network using a network. I run a server (BMC) on my server. I start my engine with an engine (https://en.m.wikipedia.org/wiki/Lockheed_SR-71_Blackbird https://en.m.wikipedia.org/wiki/Lockheed_SR-71_Blackbird).
- deleted 5y ago[deleted]
- noobermin 5y agoIt important to remember dense matrix multiplication (doing it by hand) is a O(N^3) operation, this is about approximations to multiplication that beat that already harsh complexity. There is a whole field that develops approximations to matrix multiplication of large matrices, I'm assuming this article is about using ML to find good approximations. To the replies, the very act of evaluting a prediction from a NN is matrix multiplication (linear transform of a column vector is matrix multiplication). This doesn't replace matrix multiplication wholesale, lol. This is about multiplication in a specific case.
- Nevermark 5y agoAs long as their is enough consistency between approximating a multiply X*W and Z*Wt (Wt = W transpose), then it is possible it could be used in NN training. Y = X*W is the forward propagation. If Z is an error or derivative of error, Z*Wt is the back propagation. Its an interesting question as to how well that would work. Anything that speeds up matrix multiply in NN and deep learning in general would be a big deal.
- amelius 5y agoBut ML relies on GPUs with lots of parallelism, so e.g. O(N^3) becomes O(N^2) for N smaller than the number of cores (roughly speaking).