smaddrellmander

3 ms·

1.
Fast weights and sparse attention in GLM-5.3-Flash(idlemachines.co.uk)
7 points by smaddrellmander 1mo ago|0 comments
2.
Tokenisation: Who decides what a token is anyway?(idlemachines.co.uk)
2 points by smaddrellmander 2mo ago|0 comments
3.
Adam and AdamW: adaptive optimisation and weight decay(idlemachines.co.uk)
1 points by smaddrellmander 2mo ago|0 comments
4.
Thinking Machines dropped RoPE, and it's a good idea(idlemachines.co.uk)
6 points by smaddrellmander 3mo ago|0 comments
5.
Nesso-1: Accelerating Open-Source Binding Affinity Predictions [pdf](valencelabs.com)
2 points by smaddrellmander 3mo ago|1 comments
6.
by smaddrellmander 3mo ago
Curious what people think? Decent binding affinity model out of Valence Labs.
7.
by smaddrellmander 3mo ago
Love this! Cute graphics and the way the balls combine is really fun. Great job.
8.
The annotated PyTorch training loop(idlemachines.co.uk)
81 points by smaddrellmander 4mo ago|19 comments
9.
Mae vs. MSE: more than just the mean vs. median debate(idlemachines.co.uk)
1 points by smaddrellmander 4mo ago|0 comments
10.
DiffusionGemma: Discrete diffusion in a large language model(idlemachines.co.uk)
3 points by smaddrellmander 4mo ago|1 comments
11.
Heaven knows I'm perplexed now(idlemachines.co.uk)
2 points by smaddrellmander 4mo ago|0 comments
12.
Reading MAI's efficiency gain. How to pick architectures like serious people(idlemachines.co.uk)
9 points by smaddrellmander 4mo ago|0 comments
13.
MAI-Thinking-1: Building a Hill-Climbing Machine [pdf](microsoft.ai)
3 points by smaddrellmander 4mo ago|0 comments
14.
Are contrastive losses just cross entropy all along?(idlemachines.co.uk)
2 points by smaddrellmander 4mo ago|0 comments
15.
Every token, everywhere, all at once(idlemachines.co.uk)
2 points by smaddrellmander 4mo ago|0 comments
16.
by smaddrellmander 4mo ago
A similar vein, asking why in PRs. Most people don’t do the above, and asking why when it doesn’t make sense really helps understanding what your colleagues are actually thinking. Helps create a unified perspective on the codebase.
17.
The cut in the Mixture of Experts compute graph(idlemachines.co.uk)
1 points by smaddrellmander 5mo ago|0 comments
18.
DeepSeek V4 from the Inside(idlemachines.co.uk)
2 points by smaddrellmander 5mo ago|0 comments
19.
Softmax, can you derive the Jacobian? And should you care?(idlemachines.co.uk)
134 points by smaddrellmander 5mo ago|47 comments
20.
Gemma 4 is not your standard transformer(idlemachines.co.uk)
2 points by smaddrellmander 6mo ago|0 comments