cgadski

7 ms·

31.
by cgadski 2y ago
Tensor network notation is really useful for differentiating with respect to tensors. For example, where F is a real function of a matrix variable, think of how you'd differentiate F(A X) with respect to X. Conceptually this is easy, b
32.
Factor Graphs and Tensor Networks(rajatvd.github.io)
1 points by cgadski 2y ago|1 comments
33.
by cgadski 3y ago
Haha yeah, took some love. I have a scrappy little "framework" that I've been adjusting since I started making interactive posts last year. Writing my interactive widgets feels a bit like doing a game jam now: just copy a tem
34.
by cgadski 3y ago
Thank you! In the beginning, I used kognise's water.css [1], so most of the smart decisions (background/text color, margins, line spacing I think) probably come from there. Since then it's been some amount of little adjustmen
35.
by cgadski 3y ago
Thanks so much! And yes, that's quite true. When parameter gradients don't quite vanish, then the equation <g_x, d x / d eps> = <g_y, d y / d eps> becomes <g_x, d x / d eps> = <g_y, d y / d
36.
by cgadski 3y ago
Hi, thanks! In that sentence I was only talking about the translations and rotations of the plane as a group of invariances for the action of the two-body problem. This group is generated by one-parameter subgroups producing vertical transl
37.
by cgadski 3y ago
That's also a neat result! I'd just like to highlight that the conservation laws proved in that paper are functions of the parameters that hold over the course of gradient descent, whereas my post is talking about functions of the
38.
by cgadski 3y ago
Exactly right! In fact, because that symmetry does not include an action on the parameters of the layer, your conserved quantity <gx, dx> should hold whether or not the network is stationary for a loss. This means that it'll be s
39.
Where is Noether's principle in machine learning?(cgad.ski)
296 points by cgadski 3y ago|76 comments
40.
by cgadski 3y ago
But it _is_ possible to make a ball out of hexagons if you use infinitely many of them!
41.
by cgadski 3y ago
In my experience, the discrete version of the Gauss-Bonnet theorem is a really great conversation starter to talk about the kinds of things mathematicians like. Everyone knows that the exterior angles in a polygon has to sum up to 2 pi, but
42.
Curvature of Polyhedra (2021)(nicf.net)
74 points by cgadski 3y ago|18 comments
43.
by cgadski 3y ago
That's a neat trick, "cancelling" out catastrophic cancellation. Of course it doesn't work anymore once x is smaller than machine epsilon, whereas expm1(x) / x will continue to work. In general herbie is pretty good
44.
Backprop Isn't Just for Calculus(cgad.ski)
3 points by cgadski 3y ago|0 comments
45.
by cgadski 3y ago
Dynamical systems definitely don't have to preserve volume!
46.
Backprop isn't just for calculus(cgad.ski)
3 points by cgadski 3y ago|0 comments
47.
3D Shape Matching with Quaternions(lisyarus.github.io)
2 points by cgadski 3y ago|1 comments
48.
Stable Fiddusion: Frequency-domain blue noise generator(acko.net)
107 points by cgadski 3y ago|17 comments
49.
by cgadski 3y ago
It's great to see so much interest in this post on HN! In view of the attention, I feel like I need to emphasize that although my writing and illustrations are original, the "big idea" certainly is not. In fact, my goal was j
50.
by cgadski 3y ago
Yes indeed, my blog stylesheet started out as water.css. It was really helpful to get good typesetting and color defaults.
51.
by cgadski 3y ago
My guess would be that I have to take another look at optimizing the javascript I use for drawing the interactive widgets. It's just bare-bones javascript and canvas, but a previous version managed to freeze some browsers for unknown r
52.
by cgadski 3y ago
Took a preliminary look through and it feels pretty exciting. Thanks for sharing. My own post basically covers the simplest possible case where gradient descent does something related to kernels. The problem is that the "tangent kernel
53.
by cgadski 3y ago
I had Chapter 6 open while working on this post :) I should probably cite it too. As far as I could see that book doesn't talk about kernels arising from gradient descent, but it does say many other things about GP kernels. Certainly i
54.
When gradient descent is a kernel method(cgad.ski)
201 points by cgadski 3y ago|37 comments
55.
Four Points on the Circle(cgad.ski)
2 points by cgadski 3y ago|0 comments
56.
A Note on Scale-Free Networks(cgad.ski)
5 points by cgadski 3y ago|0 comments
57.
A Coin Flip by Any Other Name...(cgad.ski)
2 points by cgadski 3y ago|0 comments
58.
by cgadski 3y ago
As the author of that other post, I'm happy to see Terry Tao's post on the front page now too. I learned a lot from it when I was researching for my own. The two articles end up going in somewhat different directions---Terry doesn
59.
by cgadski 3y ago
In the formalism of both this article and https://news.ycombinator.com/item?id=37517118 , "base units" are specially designated physical quantities, while "dimensions" describe how physical quantities sca
60.
by cgadski 3y ago
"Dimensions" in dimensional analysis are not the same thing as dimensions in linear algebra! I'm not sure what a scalar quantity with an infinite dimension would be. (To reiterate, the dimensions of dimensional analysis tell

More ›