patrickkidger

8 ms·

31.
by patrickkidger 3y ago
If you like PyTorch then you might like Equinox, by the way. ( https://github.com/patrick-kidger/equinox ; 1.4k GitHub stars now!) Basically designed to offer PyTorch-like syntax for working with JAX. The latter is exce
32.
by patrickkidger 3y ago
You can do JAX->TF->ONNX I believe: https://github.com/patrick-kidger/equinox/blob/main/equinox/...
33.
by patrickkidger 3y ago
Indeed, it is much better to use jit(grad(f)) in general. Supporting the opposite composition is still useful in some edge cases -- for example when debugging, you want to step through a computation without jit, and simply not crash when di
34.
by patrickkidger 3y ago
I'm going to disagree here! Classes and functional programming can go very well together, just don't expect to do in-place mutation. (I.e. OO-style programming.) You might like Equinox ( https://github.com/patrick-k
35.
by patrickkidger 3y ago
I feel like I see the opposite -- that everything scientific computing is getting rewritten in something autodifferentiable! Whether that's JAX or something else. My experience might be biased though: shameless advert for Equinox ( htt
36.
Show HN: Equinox (1.3k stars), a JAX library for neural networks and sciML(github.com)
1 points by patrickkidger 3y ago|0 comments
37.
by patrickkidger 3y ago
FWIW, I've come to regard this (cooperative multiple inheritance) as a failed experiment. It's just been too confusing, and hasn't seen adoption. Instead, I've come to prefer a style I took from Julia: every class is eit
38.
by patrickkidger 3y ago
Incidentally `diffrax.PIDController` also has a `dtmin` argument that could probably be used here instead of re-running things. :)
39.
by patrickkidger 3y ago
This is awesome to hear! And if you're doing implicit methods, then you may be interested to hear that today's release of Diffrax now includes IMEX solvers! Sil3, KenCarp3, KenCarp4, KenCarp5.
40.
by patrickkidger 3y ago
You're thinking of `jax.closure_convert`. :) (Although technically that works by tracing and extracting all constants from the jaxpr, rather than introspecting the function's closure cells -- it sounds like your trick is the latte
41.
by patrickkidger 3y ago
Shameless advert -- Equinox is a neural network library for JAX based entirely around pytrees: https://github.com/patrick-kidger/equinox (Now on 1.1k stars so it's achieved some popularity!) This makes model-build
42.
by patrickkidger 3y ago
Try using jaxtyping: https://github.com/google/jaxtyping . It also supports numpy/pytorch/etc.
43.
by patrickkidger 4y ago
I think of Python as being a pretty big language. At a feature level there are things like descriptors and metaclasses, which are complex and rarely-used. There's a huge number of obscure magic methods/attributes: __origin__, __or
44.
by patrickkidger 4y ago
+1 to the sibling comments for a split keyboard option. (Preferably a proper version with thumb clusters.) At least for me, that would make this laptop a must-buy.
45.
by patrickkidger 4y ago
Heads-up that soft-wrap is now supported at HEAD! https://docs.helix-editor.com/master/configuration.html#edit...
46.
by patrickkidger 4y ago
There is the option to `auto-format` on save, if that helps. (FWIW I use neither, and format via pre-commit hooks instead of in-editor.)
47.
by patrickkidger 4y ago
FWIW, Helix does have tabs. (Set `editor.bufferline = True` in `config.toml` in order to have them listed.)
48.
by patrickkidger 4y ago
This can be changed: I use zellij and just rewrote its config file to place almost all the keys in a single mode, basically 1-1 with tmux. That said, I must admit that zellij doesn't add a whole lot to my workflow relative to tmux. I t
49.
by patrickkidger 4y ago
It sounds like you're concerned about how downstream libraries tend to wrap JAX transformations to handle their own thing? (E.g. `haiku.grad`.) If so, then allow me to make my usual advert here for Equinox: https://github.co
50.
by patrickkidger 4y ago
Aha, I have probably read that sentence literally hundreds of times, and never spotted the typo. I will never be able to unsee that. Geron is good but now a bit out-of-date. No transformers (just CNNs/RNNs/etc.) and the coding com
51.
by patrickkidger 4y ago
I think those are all things you need for life . But what you do need for specifically a PhD ? I argue that "knowing stuff" is what is necessary -- and that indeed it's essentially the purpose of the whole academic institut
52.
by patrickkidger 4y ago
As the author of this article... I have read maybe one textbook cover-to-cover in my life. :D (Hands-on machine learning, by Geron, back when I made the jump math->ML.)
53.
by patrickkidger 4y ago
Right! This was the situation for me.
54.
by patrickkidger 4y ago
So HPC means like ten different things. For example another commentor mentions low-latency concerns in finance, and that's something I have zero experience with. HPC has often also meant writing a lot of C++ to do e.g. MD or something.
55.
by patrickkidger 4y ago
Completely agreed!
56.
by patrickkidger 4y ago
Haha, so I actually have an atrocious memory! Famously so amongst my friends, I never remember what we've discussed. When I wrote this list I certainly wasn't expecting/recommending all of this to stay in the reader's he
57.
by patrickkidger 4y ago
Ooh, that's a great suggestion! So one thing I learned a lot of in my PhD (for literally a whole year), that I literally never needed, was functional analytic methods for PDEs. Stuff like Moser iterations / the De Giorgi-Nash-Mo
58.
by patrickkidger 4y ago
This is a great description, thank you! What you've said is precisely the reason I emphasised knowing a bit of foundational math, e.g. topology.
59.
by patrickkidger 4y ago
I'm sorry it came across this way for you! Rather, I'm just outlining why folks seem to keep asking me this question. :)
60.
by patrickkidger 4y ago
I'm sorry to hear that things didn't go so well for you! FWIW being able to "program like nobody's business" is still really really valuable. It's why I dedicate such a large chunk of the post to software dev s

More ›