martinloretz

5 ms·

1. 1.▲
   by martinloretz 2y ago

   I think this paper is the key to the next speedup in local LLM inference. By making the model sparse (using the ReLU activation), we can save around 80% of memory accesses and computations of the Feed Forward Layers. ReLU sets the output of
2. 2.▲
   ReLU Strikes Back: Exploiting Activation Sparsity in Large Language Models(2023)(arxiv.org)

   2 points by martinloretz 2y ago|1 comments
3. 3.▲
   by martinloretz 2y ago

   Great work. Can you elaborate on how the radix selection works and how to get that working with float's and inner product distance? I just quickly checked the code, I'm not familiar with radix selection, but really interested in m
4. 4.▲
   A three body problem simulator(three-body-problem.martinloretz.com)

   2 points by martinloretz 2y ago|0 comments