51 ms·

Could you break down a bit more about why you can skip so many calculations ?

kolinko 3y ago
It’s explained in detail here: https://kolinko.github.io/effort/equations.html https://kolinko.github.io/effort/equations.html Long story short - I kind of sort them and pick only the top % that would give a highest result. One part is choosing them though - I think this was done before in some papers. But the second part was an implementation of multiplication that is efficient on both gpus and cpus when choosing weights almost at will. All explained on the site, but I just got feedback that it may not be easy enough to read, so I’ll push it through gpt for grmamar fixes soon :) It’s also a bit complicated as an algorithm.
- throwaway2562 3y ago
  Hmmm. Very interesting. I wonder if you could speed up your clever approximation still further with this approach https://arxiv.org/pdf/2205.09120.pdf https://arxiv.org/pdf/2205.09120.pdf
  - kolinko 3y ago
    Nah, sadly this most likely will not work with Q1/Q1.5 implementations - initially I was playing with monte carlo approximations (before I arrived at bucketMul), and the convergence was very slow for binary/ternary networks. Or, in simpler terms - if you have just ones and zeroes, and minus ones, you can remove zeroes from calculations, but that's it. No good method to figure out which ones are more important than the other ones. Also, there are no bits left to store positional information when bucketing. There are some paths that could be explored in this fashion, but it would require a redesign of the algorithm from the ground up.
    - throwaway2562 2y ago
      Late response: thank you. Saved me some time thinking further on this.
punnerud 3y ago
A bit like calculating the fastest route for a car, you probably don’t need to calculate the distances for the opposite side of earth if you will not drive there. Then multiply that by a billion, but the optimization still holds.
- kolinko 3y ago
  Nice analogy, thanks! Yes - you still need a map, because one day you need this road, another day you need info about another road, but you're not using info about all the roads all of the time.
deleted 3y ago
[deleted]

It’s explained in detail here: https://kolinko.github.io/effort/equations.html https://kolinko.github.io/effort/equations.html Long story short - I kind of sort them and pick only the top % that would give a highest result. One part is choosing them though - I think this was done before in some papers. But the second part was an implementation of multiplication that is efficient on both gpus and cpus when choosing weights almost at will. All explained on the site, but I just got feedback that it may not be easy enough to read, so I’ll push it through gpt for grmamar fixes soon :) It’s also a bit complicated as an algorithm.

throwaway2562 3y ago
Hmmm. Very interesting. I wonder if you could speed up your clever approximation still further with this approach https://arxiv.org/pdf/2205.09120.pdf https://arxiv.org/pdf/2205.09120.pdf
- kolinko 3y ago
  Nah, sadly this most likely will not work with Q1/Q1.5 implementations - initially I was playing with monte carlo approximations (before I arrived at bucketMul), and the convergence was very slow for binary/ternary networks. Or, in simpler terms - if you have just ones and zeroes, and minus ones, you can remove zeroes from calculations, but that's it. No good method to figure out which ones are more important than the other ones. Also, there are no bits left to store positional information when bucketing. There are some paths that could be explored in this fashion, but it would require a redesign of the algorithm from the ground up.
  - throwaway2562 2y ago
    Late response: thank you. Saved me some time thinking further on this.

punnerud 3y ago

A bit like calculating the fastest route for a car, you probably don’t need to calculate the distances for the opposite side of earth if you will not drive there. Then multiply that by a billion, but the optimization still holds.

kolinko 3y ago
Nice analogy, thanks! Yes - you still need a map, because one day you need this road, another day you need info about another road, but you're not using info about all the roads all of the time.

deleted 3y ago

[deleted]