amindiro

4 ms·

1.
by amindiro 9mo ago
Hi, thanks a lot for the feedback! I'm glad you enjoyed the profiling sections. You've hit the nail on the head regarding the missing pieces. I actually hit a bit of a wall with my current hardware; using an RTX 2070 made it diffi
2.
by amindiro 9mo ago
One reason is clearly the fast past at which nvidia is evolving the hardware. I would consider cuda a very well documented platform in general. What they lack is low level tutorials, but this is where posts like this one can be a good resou
3.
by amindiro 9mo ago
Agreed, this observation holds true for both decode and prefill. Thanks for sharing the code
4.
by amindiro 10mo ago
Very interesting and Would love to see the experiments. Quick question: what do you mean about kernel dependent ?
5.
by amindiro 10mo ago
Hi thanks for feedback! That’s a good point I did compare to torch but at a high enough sequence length (~1024) torch version starts OOM because it has to materialize the S^2 in global mem. On small sequence length, torch does win solely on
6.
by amindiro 10mo ago
I’ve spent the last few weeks deconstructing FlashAttention. While the original paper is brilliant, I found that just reading it didn't give me a "gut feeling" for why certain engineering choices were made (the transition fro
7.
I rebuilt FlashAttention in Triton to understand the performance archaeology(aminediro.com)
95 points by amindiro 10mo ago|19 comments
8.
Ferrules: A fast document parser written in Rust(github.com)
1 points by amindiro 2y ago|1 comments
9.
by amindiro 2y ago
After battling Python dependencies, slow processing, and deployment headaches with tools like unstructured, I finally snapped—and built Ferrules, a blazing-fast document parser in Rust. Why Ferrules? - Speed – Native PDF parsing (pdfium), h
10.
Clean Code, Horrible performance Rust edition(medium.com)
2 points by amindiro 3y ago|1 comments
11.
by amindiro 3y ago
Hello, Just reimplemented the infamous video "Clean" Code, Horrible Performance" in Rust! Casey Muratori showed how trying to be cute with your code and introducing unnecessary indirection can hurt performance. The enum versi
12.
by amindiro 4y ago
I started working on a distributed task queue library a few months back. The library is available as a python package to install a start using : daskqueue - pypi package For all its greatness, Dask implements a central scheduler (basically
13.
Daskqueue: Dask-based distributed task queue(github.com)
3 points by amindiro 4y ago|1 comments