5 ms·
I've been doing web related engineering for the past few years and recently been interested in machine learning. Taking a random perusal through the code for th
by triyambakam 3y ago
I've been doing web related engineering for the past few years and recently been interested in machine learning. Taking a random perusal through the code for this [1] feels daunting. So many single letter variables, it looks like old JS soup code. It makes me hesitant to leave Typescript and even Rust (for web tooling).
[1] https://github.com/state-spaces/mamba/blob/main/mamba_ssm/ops/selective_scan_interface.py https://github.com/state-spaces/mamba/blob/main/mamba_ssm/op...
- ekelsen 3y agoThe variable names A, B, C, D and related assume knowledge of the state space model / formulation where these are the common names of matrices in the state space equations.
- triyambakam 3y agoThanks, that's helpful.
- junipertea 3y agoIt corresponds to the paper formulas as a courtesy. While descriptive comments or function signature might be nicer, it still is reasonably easy to follow along from the paper description! (Although, most of the complexity is hidden away in selective_scan_cuda.fwd())
- triyambakam 3y agoAha, thanks, that is helpful. Any chance you know of a guide that walks through a paper and code side by side?
- ttul 3y agoHailing Jeremy Howard and his team at Fast.ai to make it so!
- telotortium 3y agoImplementation of selective_scan_cuda: https://github.com/state-spaces/mamba/blob/6dbfc4553a98c81e1e93b8fd2d5abf387f5c09ee/csrc/selective_scan/selective_scan_fwd_kernel.cuh https://github.com/state-spaces/mamba/blob/6dbfc4553a98c81e1...
- filterfiber 3y ago> It makes me hesitant to leave Typescript and even Rust (for web tooling). To be clear this is not what a "typical" python application/script looks like at all. While there's definitely an argument to be made about python's immature type support (especially compared to typescript/rust), don't use this as an example of python's readability. This was written to closely follow the math equations, not to be a maintainable piece of software. It makes a lot more sense from a mathematics/academic perspective, but not from a software development perspective. Strong typing support would do nothing for readability but make this harder to read. Typing support doesn't help your naming, commenting, formatting, etc. that makes this so hard to read.
- vaillant 3y agoWell, this is cutting edge ML code written in PyTorch. I wouldn't worry about understanding something like this - you start with scikit-learn first.