oraziorillo

13 ms·

1.
by oraziorillo 3mo ago
Following up on my post on microcrad — many of you liked and supported the project, thank you for that — I published a long-form piece on automatic differentiation, where I explain backpropagation in simple terms so that everyone will (hope
2.
What loss.backward() actually does(oraziorillo.com)
18 points by oraziorillo 3mo ago|3 comments
3.
by oraziorillo 4mo ago
My experience on this project was that the checks proved to be useful way before I could run the program for the first time. The first version of the code had relatively few null checks. Later I went through and added them consistently at f
4.
by oraziorillo 4mo ago
I haven’t really looked into performance comparisons with tensor-based frameworks. This was mainly a learning project, so my goal wasn’t to make it fast. My guess is that a scalar engine might have an advantage on tiny optimization problems
5.
by oraziorillo 4mo ago
I love the idea! The next step is to add role play games in town squares.
6.
by oraziorillo 4mo ago
Thank you for pointing this out, and to those adding clarifications to this statement as well. I've fixed the issue by renaming the reserved identifiers.
7.
by oraziorillo 4mo ago
I re-read the second part with more calm once I got back home. There were at least a couple of things I wasn't familiar with (and therefore didn't even consider), specifically FAM and variably modified types. Thanks for the point
8.
by oraziorillo 4mo ago
I see, I agree that especially checking for null really comes to styles and opinions, I still don't have one I can call mine. Thanks for the explanation!
9.
by oraziorillo 4mo ago
[deprecated post-edit] I guess I used function names beginning with underscore as it didn’t occur to me that it might be un-idiomatic. The intention was to make clear to myself that those functions are private and meant to be only used only
10.
by oraziorillo 4mo ago
ngl, it distracts from backprop itself a little, but teaches a lot about memory management. I did it this way because in parallel I wanted to get better at C, but if your aim is to purely work on ML fundamentals, it’s probably better to do
11.
by oraziorillo 4mo ago
In general yes, but not if you are disciplined. I’d put it this way: though the idea of ref counting felt very natural at the beginning for my use case, at some point I realized there were probably better techniques to achieve the same goal
12.
by oraziorillo 4mo ago
Mainly did it for learning, as dgellow correctly presumed, but also there’s something intrinsically beautiful in writing code with zero dependencies
13.
by oraziorillo 4mo ago
Hey HN, I'm Orazio. I built microcrad (with a 'c'), a tiny scalar-valued automatic differentiation engine, with a small multi-layer perceptron implementation on top. It's reimplementation of Andrej Karpathy's microg
14.
Show HN: Microcrad – Micrograd Reimplemented in C(github.com)
79 points by oraziorillo 4mo ago|28 comments